wget-1.21.2/0000755000000000000000000000000014115733702007515 500000000000000wget-1.21.2/README0000644000000000000000000000756514057006176010335 00000000000000 -*- text -*- GNU Wget ======== Current Web home: https://www.gnu.org/software/wget/ GNU Wget is a free utility for non-interactive download of files from the Web. It supports HTTP, HTTPS, and FTP protocols, as well as retrieval through HTTP proxies. It can follow links in HTML pages and create local versions of remote web sites, fully recreating the directory structure of the original site. This is sometimes referred to as "recursive downloading." While doing that, Wget respects the Robot Exclusion Standard (/robots.txt). Wget can be instructed to convert the links in downloaded HTML files to the local files for offline viewing. Recursive downloading also works with FTP, where Wget can retrieve a hierarchy of directories and files. With both HTTP and FTP, Wget can check whether a remote file has changed on the server since the previous run, and only download the newer files. Wget has been designed for robustness over slow or unstable network connections; if a download fails due to a network problem, it will keep retrying until the whole file has been retrieved. If the server supports regetting, it will instruct the server to continue the download from where it left off. If you are behind a firewall that requires the use of a socks style gateway, you can get the socks library and compile wget with support for socks. Most of the features are configurable, either through command-line options, or via initialization file .wgetrc. Wget allows you to install a global startup file (/usr/local/etc/wgetrc by default) for site settings. Wget works under almost all Unix variants in use today and, unlike many of its historical predecessors, is written entirely in C, thus requiring no additional software, such as Perl. The external software it does work with, such as OpenSSL, is optional. As Wget uses the GNU Autoconf, it is easily built on and ported to new Unix-like systems. The installation procedure is described in the INSTALL file. As with other GNU software, the latest version of Wget can be found at the master GNU archive site ftp.gnu.org, and its mirrors. Wget resides at . Please report bugs in Wget to . See the file `MAILING-LIST' for information about Wget mailing lists. Wget's home page is at . If you would like to contribute code for Wget, please read CONTRIBUTING.md. Wget was originally written and mainained by Hrvoje Niksic. Please see the file AUTHORS for a list of major contributors, and the ChangeLogs for a detailed listing of all contributions. Copyright (C) 1995-2021 Free Software Foundation, Inc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. wget-1.21.2/.tarball-version0000644000000000000000000000000714115733702012537 000000000000001.21.2 wget-1.21.2/aclocal.m40000644000000000000000000031673214115733306011311 00000000000000# generated automatically by aclocal 1.16.4 -*- Autoconf -*- # Copyright (C) 1996-2021 Free Software Foundation, Inc. # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY, to the extent permitted by law; without # even the implied warranty of MERCHANTABILITY or FITNESS FOR A # PARTICULAR PURPOSE. m4_ifndef([AC_CONFIG_MACRO_DIRS], [m4_defun([_AM_CONFIG_MACRO_DIRS], [])m4_defun([AC_CONFIG_MACRO_DIRS], [_AM_CONFIG_MACRO_DIRS($@)])]) m4_ifndef([AC_AUTOCONF_VERSION], [m4_copy([m4_PACKAGE_VERSION], [AC_AUTOCONF_VERSION])])dnl m4_if(m4_defn([AC_AUTOCONF_VERSION]), [2.71],, [m4_warning([this file was generated for autoconf 2.71. You have another version of autoconf. It may work, but is not guaranteed to. If you have problems, you may need to regenerate the build system entirely. To do so, use the procedure documented by the package, typically 'autoreconf'.])]) # =========================================================================== # https://www.gnu.org/software/autoconf-archive/ax_ac_append_to_file.html # =========================================================================== # # SYNOPSIS # # AX_AC_APPEND_TO_FILE([FILE],[DATA]) # # DESCRIPTION # # Appends the specified data to the specified Autoconf is run. If you want # to append to a file when configure is run use AX_APPEND_TO_FILE instead. # # LICENSE # # Copyright (c) 2009 Allan Caffee # # Copying and distribution of this file, with or without modification, are # permitted in any medium without royalty provided the copyright notice # and this notice are preserved. This file is offered as-is, without any # warranty. #serial 10 AC_DEFUN([AX_AC_APPEND_TO_FILE],[ AC_REQUIRE([AX_FILE_ESCAPES]) m4_esyscmd( AX_FILE_ESCAPES [ printf "%s" "$2" >> "$1" ]) ]) # =========================================================================== # https://www.gnu.org/software/autoconf-archive/ax_ac_print_to_file.html # =========================================================================== # # SYNOPSIS # # AX_AC_PRINT_TO_FILE([FILE],[DATA]) # # DESCRIPTION # # Writes the specified data to the specified file when Autoconf is run. If # you want to print to a file when configure is run use AX_PRINT_TO_FILE # instead. # # LICENSE # # Copyright (c) 2009 Allan Caffee # # Copying and distribution of this file, with or without modification, are # permitted in any medium without royalty provided the copyright notice # and this notice are preserved. This file is offered as-is, without any # warranty. #serial 10 AC_DEFUN([AX_AC_PRINT_TO_FILE],[ m4_esyscmd( AC_REQUIRE([AX_FILE_ESCAPES]) [ printf "%s" "$2" > "$1" ]) ]) # =========================================================================== # https://www.gnu.org/software/autoconf-archive/ax_add_am_macro_static.html # =========================================================================== # # SYNOPSIS # # AX_ADD_AM_MACRO_STATIC([RULE]) # # DESCRIPTION # # Adds the specified rule to $AMINCLUDE. # # LICENSE # # Copyright (c) 2009 Tom Howard # Copyright (c) 2009 Allan Caffee # # Copying and distribution of this file, with or without modification, are # permitted in any medium without royalty provided the copyright notice # and this notice are preserved. This file is offered as-is, without any # warranty. #serial 8 AC_DEFUN([AX_ADD_AM_MACRO_STATIC],[ AC_REQUIRE([AX_AM_MACROS_STATIC]) AX_AC_APPEND_TO_FILE(AMINCLUDE_STATIC,[$1]) ]) # =========================================================================== # https://www.gnu.org/software/autoconf-archive/ax_am_macros_static.html # =========================================================================== # # SYNOPSIS # # AX_AM_MACROS_STATIC # # DESCRIPTION # # Adds support for macros that create Automake rules. You must manually # add the following line # # include $(top_srcdir)/aminclude_static.am # # to your Makefile.am files. # # LICENSE # # Copyright (c) 2009 Tom Howard # Copyright (c) 2009 Allan Caffee # # Copying and distribution of this file, with or without modification, are # permitted in any medium without royalty provided the copyright notice # and this notice are preserved. This file is offered as-is, without any # warranty. #serial 11 AC_DEFUN([AMINCLUDE_STATIC],[aminclude_static.am]) AC_DEFUN([AX_AM_MACROS_STATIC], [ AX_AC_PRINT_TO_FILE(AMINCLUDE_STATIC,[ # ]AMINCLUDE_STATIC[ generated automatically by Autoconf # from AX_AM_MACROS_STATIC on ]m4_esyscmd([LC_ALL=C date])[ ]) ]) # =========================================================================== # https://www.gnu.org/software/autoconf-archive/ax_check_gnu_make.html # =========================================================================== # # SYNOPSIS # # AX_CHECK_GNU_MAKE([run-if-true],[run-if-false]) # # DESCRIPTION # # This macro searches for a GNU version of make. If a match is found: # # * The makefile variable `ifGNUmake' is set to the empty string, otherwise # it is set to "#". This is useful for including a special features in a # Makefile, which cannot be handled by other versions of make. # * The makefile variable `ifnGNUmake' is set to #, otherwise # it is set to the empty string. This is useful for including a special # features in a Makefile, which can be handled # by other versions of make or to specify else like clause. # * The variable `_cv_gnu_make_command` is set to the command to invoke # GNU make if it exists, the empty string otherwise. # * The variable `ax_cv_gnu_make_command` is set to the command to invoke # GNU make by copying `_cv_gnu_make_command`, otherwise it is unset. # * If GNU Make is found, its version is extracted from the output of # `make --version` as the last field of a record of space-separated # columns and saved into the variable `ax_check_gnu_make_version`. # * Additionally if GNU Make is found, run shell code run-if-true # else run shell code run-if-false. # # Here is an example of its use: # # Makefile.in might contain: # # # A failsafe way of putting a dependency rule into a makefile # $(DEPEND): # $(CC) -MM $(srcdir)/*.c > $(DEPEND) # # @ifGNUmake@ ifeq ($(DEPEND),$(wildcard $(DEPEND))) # @ifGNUmake@ include $(DEPEND) # @ifGNUmake@ else # fallback code # @ifGNUmake@ endif # # Then configure.in would normally contain: # # AX_CHECK_GNU_MAKE() # AC_OUTPUT(Makefile) # # Then perhaps to cause gnu make to override any other make, we could do # something like this (note that GNU make always looks for GNUmakefile # first): # # if ! test x$_cv_gnu_make_command = x ; then # mv Makefile GNUmakefile # echo .DEFAULT: > Makefile ; # echo \ $_cv_gnu_make_command \$@ >> Makefile; # fi # # Then, if any (well almost any) other make is called, and GNU make also # exists, then the other make wraps the GNU make. # # LICENSE # # Copyright (c) 2008 John Darrington # Copyright (c) 2015 Enrico M. Crisostomo # # Copying and distribution of this file, with or without modification, are # permitted in any medium without royalty provided the copyright notice # and this notice are preserved. This file is offered as-is, without any # warranty. #serial 12 AC_DEFUN([AX_CHECK_GNU_MAKE],dnl [AC_PROG_AWK AC_CACHE_CHECK([for GNU make],[_cv_gnu_make_command],[dnl _cv_gnu_make_command="" ; dnl Search all the common names for GNU make for a in "$MAKE" make gmake gnumake ; do if test -z "$a" ; then continue ; fi ; if "$a" --version 2> /dev/null | grep GNU 2>&1 > /dev/null ; then _cv_gnu_make_command=$a ; AX_CHECK_GNU_MAKE_HEADLINE=$("$a" --version 2> /dev/null | grep "GNU Make") ax_check_gnu_make_version=$(echo ${AX_CHECK_GNU_MAKE_HEADLINE} | ${AWK} -F " " '{ print $(NF); }') break ; fi done ;]) dnl If there was a GNU version, then set @ifGNUmake@ to the empty string, '#' otherwise AS_VAR_IF([_cv_gnu_make_command], [""], [AS_VAR_SET([ifGNUmake], ["#"])], [AS_VAR_SET([ifGNUmake], [""])]) AS_VAR_IF([_cv_gnu_make_command], [""], [AS_VAR_SET([ifnGNUmake], [""])], [AS_VAR_SET([ifnGNUmake], ["#"])]) AS_VAR_IF([_cv_gnu_make_command], [""], [AS_UNSET(ax_cv_gnu_make_command)], [AS_VAR_SET([ax_cv_gnu_make_command], [${_cv_gnu_make_command}])]) AS_VAR_IF([_cv_gnu_make_command], [""],[$2],[$1]) AC_SUBST([ifGNUmake]) AC_SUBST([ifnGNUmake]) ]) # =========================================================================== # https://www.gnu.org/software/autoconf-archive/ax_code_coverage.html # =========================================================================== # # SYNOPSIS # # AX_CODE_COVERAGE() # # DESCRIPTION # # Defines CODE_COVERAGE_CPPFLAGS, CODE_COVERAGE_CFLAGS, # CODE_COVERAGE_CXXFLAGS and CODE_COVERAGE_LIBS which should be included # in the CPPFLAGS, CFLAGS CXXFLAGS and LIBS/LIBADD variables of every # build target (program or library) which should be built with code # coverage support. Also add rules using AX_ADD_AM_MACRO_STATIC; and # $enable_code_coverage which can be used in subsequent configure output. # CODE_COVERAGE_ENABLED is defined and substituted, and corresponds to the # value of the --enable-code-coverage option, which defaults to being # disabled. # # Test also for gcov program and create GCOV variable that could be # substituted. # # Note that all optimization flags in CFLAGS must be disabled when code # coverage is enabled. # # Usage example: # # configure.ac: # # AX_CODE_COVERAGE # # Makefile.am: # # include $(top_srcdir)/aminclude_static.am # # my_program_LIBS = ... $(CODE_COVERAGE_LIBS) ... # my_program_CPPFLAGS = ... $(CODE_COVERAGE_CPPFLAGS) ... # my_program_CFLAGS = ... $(CODE_COVERAGE_CFLAGS) ... # my_program_CXXFLAGS = ... $(CODE_COVERAGE_CXXFLAGS) ... # # clean-local: code-coverage-clean # distclean-local: code-coverage-dist-clean # # This results in a "check-code-coverage" rule being added to any # Makefile.am which do "include $(top_srcdir)/aminclude_static.am" # (assuming the module has been configured with --enable-code-coverage). # Running `make check-code-coverage` in that directory will run the # module's test suite (`make check`) and build a code coverage report # detailing the code which was touched, then print the URI for the report. # # This code was derived from Makefile.decl in GLib, originally licensed # under LGPLv2.1+. # # LICENSE # # Copyright (c) 2012, 2016 Philip Withnall # Copyright (c) 2012 Xan Lopez # Copyright (c) 2012 Christian Persch # Copyright (c) 2012 Paolo Borelli # Copyright (c) 2012 Dan Winship # Copyright (c) 2015,2018 Bastien ROUCARIES # # This library is free software; you can redistribute it and/or modify it # under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation; either version 2.1 of the License, or (at # your option) any later version. # # This library is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser # General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this program. If not, see . #serial 34 m4_define(_AX_CODE_COVERAGE_RULES,[ AX_ADD_AM_MACRO_STATIC([ # Code coverage # # Optional: # - CODE_COVERAGE_DIRECTORY: Top-level directory for code coverage reporting. # Multiple directories may be specified, separated by whitespace. # (Default: \$(top_builddir)) # - CODE_COVERAGE_OUTPUT_FILE: Filename and path for the .info file generated # by lcov for code coverage. (Default: # \$(PACKAGE_NAME)-\$(PACKAGE_VERSION)-coverage.info) # - CODE_COVERAGE_OUTPUT_DIRECTORY: Directory for generated code coverage # reports to be created. (Default: # \$(PACKAGE_NAME)-\$(PACKAGE_VERSION)-coverage) # - CODE_COVERAGE_BRANCH_COVERAGE: Set to 1 to enforce branch coverage, # set to 0 to disable it and leave empty to stay with the default. # (Default: empty) # - CODE_COVERAGE_LCOV_SHOPTS_DEFAULT: Extra options shared between both lcov # instances. (Default: based on $CODE_COVERAGE_BRANCH_COVERAGE) # - CODE_COVERAGE_LCOV_SHOPTS: Extra options to shared between both lcov # instances. (Default: $CODE_COVERAGE_LCOV_SHOPTS_DEFAULT) # - CODE_COVERAGE_LCOV_OPTIONS_GCOVPATH: --gcov-tool pathtogcov # - CODE_COVERAGE_LCOV_OPTIONS_DEFAULT: Extra options to pass to the # collecting lcov instance. (Default: $CODE_COVERAGE_LCOV_OPTIONS_GCOVPATH) # - CODE_COVERAGE_LCOV_OPTIONS: Extra options to pass to the collecting lcov # instance. (Default: $CODE_COVERAGE_LCOV_OPTIONS_DEFAULT) # - CODE_COVERAGE_LCOV_RMOPTS_DEFAULT: Extra options to pass to the filtering # lcov instance. (Default: empty) # - CODE_COVERAGE_LCOV_RMOPTS: Extra options to pass to the filtering lcov # instance. (Default: $CODE_COVERAGE_LCOV_RMOPTS_DEFAULT) # - CODE_COVERAGE_GENHTML_OPTIONS_DEFAULT: Extra options to pass to the # genhtml instance. (Default: based on $CODE_COVERAGE_BRANCH_COVERAGE) # - CODE_COVERAGE_GENHTML_OPTIONS: Extra options to pass to the genhtml # instance. (Default: $CODE_COVERAGE_GENHTML_OPTIONS_DEFAULT) # - CODE_COVERAGE_IGNORE_PATTERN: Extra glob pattern of files to ignore # # The generated report will be titled using the \$(PACKAGE_NAME) and # \$(PACKAGE_VERSION). In order to add the current git hash to the title, # use the git-version-gen script, available online. # Optional variables # run only on top dir if CODE_COVERAGE_ENABLED ifeq (\$(abs_builddir), \$(abs_top_builddir)) CODE_COVERAGE_DIRECTORY ?= \$(top_builddir) CODE_COVERAGE_OUTPUT_FILE ?= \$(PACKAGE_NAME)-\$(PACKAGE_VERSION)-coverage.info CODE_COVERAGE_OUTPUT_DIRECTORY ?= \$(PACKAGE_NAME)-\$(PACKAGE_VERSION)-coverage CODE_COVERAGE_BRANCH_COVERAGE ?= CODE_COVERAGE_LCOV_SHOPTS_DEFAULT ?= \$(if \$(CODE_COVERAGE_BRANCH_COVERAGE),\ --rc lcov_branch_coverage=\$(CODE_COVERAGE_BRANCH_COVERAGE)) CODE_COVERAGE_LCOV_SHOPTS ?= \$(CODE_COVERAGE_LCOV_SHOPTS_DEFAULT) CODE_COVERAGE_LCOV_OPTIONS_GCOVPATH ?= --gcov-tool \"\$(GCOV)\" CODE_COVERAGE_LCOV_OPTIONS_DEFAULT ?= \$(CODE_COVERAGE_LCOV_OPTIONS_GCOVPATH) CODE_COVERAGE_LCOV_OPTIONS ?= \$(CODE_COVERAGE_LCOV_OPTIONS_DEFAULT) CODE_COVERAGE_LCOV_RMOPTS_DEFAULT ?= CODE_COVERAGE_LCOV_RMOPTS ?= \$(CODE_COVERAGE_LCOV_RMOPTS_DEFAULT) CODE_COVERAGE_GENHTML_OPTIONS_DEFAULT ?=\ \$(if \$(CODE_COVERAGE_BRANCH_COVERAGE),\ --rc genhtml_branch_coverage=\$(CODE_COVERAGE_BRANCH_COVERAGE)) CODE_COVERAGE_GENHTML_OPTIONS ?= \$(CODE_COVERAGE_GENHTML_OPTIONS_DEFAULT) CODE_COVERAGE_IGNORE_PATTERN ?= GITIGNOREFILES := \$(GITIGNOREFILES) \$(CODE_COVERAGE_OUTPUT_FILE) \$(CODE_COVERAGE_OUTPUT_DIRECTORY) code_coverage_v_lcov_cap = \$(code_coverage_v_lcov_cap_\$(V)) code_coverage_v_lcov_cap_ = \$(code_coverage_v_lcov_cap_\$(AM_DEFAULT_VERBOSITY)) code_coverage_v_lcov_cap_0 = @echo \" LCOV --capture\" \$(CODE_COVERAGE_OUTPUT_FILE); code_coverage_v_lcov_ign = \$(code_coverage_v_lcov_ign_\$(V)) code_coverage_v_lcov_ign_ = \$(code_coverage_v_lcov_ign_\$(AM_DEFAULT_VERBOSITY)) code_coverage_v_lcov_ign_0 = @echo \" LCOV --remove /tmp/*\" \$(CODE_COVERAGE_IGNORE_PATTERN); code_coverage_v_genhtml = \$(code_coverage_v_genhtml_\$(V)) code_coverage_v_genhtml_ = \$(code_coverage_v_genhtml_\$(AM_DEFAULT_VERBOSITY)) code_coverage_v_genhtml_0 = @echo \" GEN \" \"\$(CODE_COVERAGE_OUTPUT_DIRECTORY)\"; code_coverage_quiet = \$(code_coverage_quiet_\$(V)) code_coverage_quiet_ = \$(code_coverage_quiet_\$(AM_DEFAULT_VERBOSITY)) code_coverage_quiet_0 = --quiet # sanitizes the test-name: replaces with underscores: dashes and dots code_coverage_sanitize = \$(subst -,_,\$(subst .,_,\$(1))) # Use recursive makes in order to ignore errors during check check-code-coverage: -\$(AM_V_at)\$(MAKE) \$(AM_MAKEFLAGS) -k check \$(AM_V_at)\$(MAKE) \$(AM_MAKEFLAGS) code-coverage-capture # Capture code coverage data code-coverage-capture: code-coverage-capture-hook \$(code_coverage_v_lcov_cap)\$(LCOV) \$(code_coverage_quiet) \$(addprefix --directory ,\$(CODE_COVERAGE_DIRECTORY)) --capture --output-file \"\$(CODE_COVERAGE_OUTPUT_FILE).tmp\" --test-name \"\$(call code_coverage_sanitize,\$(PACKAGE_NAME)-\$(PACKAGE_VERSION))\" --no-checksum --compat-libtool \$(CODE_COVERAGE_LCOV_SHOPTS) \$(CODE_COVERAGE_LCOV_OPTIONS) \$(code_coverage_v_lcov_ign)\$(LCOV) \$(code_coverage_quiet) \$(addprefix --directory ,\$(CODE_COVERAGE_DIRECTORY)) --remove \"\$(CODE_COVERAGE_OUTPUT_FILE).tmp\" \"/tmp/*\" \$(CODE_COVERAGE_IGNORE_PATTERN) --output-file \"\$(CODE_COVERAGE_OUTPUT_FILE)\" \$(CODE_COVERAGE_LCOV_SHOPTS) \$(CODE_COVERAGE_LCOV_RMOPTS) -@rm -f \"\$(CODE_COVERAGE_OUTPUT_FILE).tmp\" \$(code_coverage_v_genhtml)LANG=C \$(GENHTML) \$(code_coverage_quiet) \$(addprefix --prefix ,\$(CODE_COVERAGE_DIRECTORY)) --output-directory \"\$(CODE_COVERAGE_OUTPUT_DIRECTORY)\" --title \"\$(PACKAGE_NAME)-\$(PACKAGE_VERSION) Code Coverage\" --legend --show-details \"\$(CODE_COVERAGE_OUTPUT_FILE)\" \$(CODE_COVERAGE_GENHTML_OPTIONS) @echo \"file://\$(abs_builddir)/\$(CODE_COVERAGE_OUTPUT_DIRECTORY)/index.html\" code-coverage-clean: -\$(LCOV) --directory \$(top_builddir) -z -rm -rf \"\$(CODE_COVERAGE_OUTPUT_FILE)\" \"\$(CODE_COVERAGE_OUTPUT_FILE).tmp\" \"\$(CODE_COVERAGE_OUTPUT_DIRECTORY)\" -find . \\( -name \"*.gcda\" -o -name \"*.gcno\" -o -name \"*.gcov\" \\) -delete code-coverage-dist-clean: A][M_DISTCHECK_CONFIGURE_FLAGS := \$(A][M_DISTCHECK_CONFIGURE_FLAGS) --disable-code-coverage else # ifneq (\$(abs_builddir), \$(abs_top_builddir)) check-code-coverage: code-coverage-capture: code-coverage-capture-hook code-coverage-clean: code-coverage-dist-clean: endif # ifeq (\$(abs_builddir), \$(abs_top_builddir)) else #! CODE_COVERAGE_ENABLED # Use recursive makes in order to ignore errors during check check-code-coverage: @echo \"Need to reconfigure with --enable-code-coverage\" # Capture code coverage data code-coverage-capture: code-coverage-capture-hook @echo \"Need to reconfigure with --enable-code-coverage\" code-coverage-clean: code-coverage-dist-clean: endif #CODE_COVERAGE_ENABLED # Hook rule executed before code-coverage-capture, overridable by the user code-coverage-capture-hook: .PHONY: check-code-coverage code-coverage-capture code-coverage-dist-clean code-coverage-clean code-coverage-capture-hook ]) ]) AC_DEFUN([_AX_CODE_COVERAGE_ENABLED],[ AX_CHECK_GNU_MAKE([],[AC_MSG_ERROR([not using GNU make that is needed for coverage])]) AC_REQUIRE([AX_ADD_AM_MACRO_STATIC]) # check for gcov AC_CHECK_TOOL([GCOV], [$_AX_CODE_COVERAGE_GCOV_PROG_WITH], [:]) AS_IF([test "X$GCOV" = "X:"], [AC_MSG_ERROR([gcov is needed to do coverage])]) AC_SUBST([GCOV]) dnl Check if gcc is being used AS_IF([ test "$GCC" = "no" ], [ AC_MSG_ERROR([not compiling with gcc, which is required for gcov code coverage]) ]) AC_CHECK_PROG([LCOV], [lcov], [lcov]) AC_CHECK_PROG([GENHTML], [genhtml], [genhtml]) AS_IF([ test x"$LCOV" = x ], [ AC_MSG_ERROR([To enable code coverage reporting you must have lcov installed]) ]) AS_IF([ test x"$GENHTML" = x ], [ AC_MSG_ERROR([Could not find genhtml from the lcov package]) ]) dnl Build the code coverage flags dnl Define CODE_COVERAGE_LDFLAGS for backwards compatibility CODE_COVERAGE_CPPFLAGS="-DNDEBUG" CODE_COVERAGE_CFLAGS="-O0 -g -fprofile-arcs -ftest-coverage" CODE_COVERAGE_CXXFLAGS="-O0 -g -fprofile-arcs -ftest-coverage" CODE_COVERAGE_LIBS="-lgcov" AC_SUBST([CODE_COVERAGE_CPPFLAGS]) AC_SUBST([CODE_COVERAGE_CFLAGS]) AC_SUBST([CODE_COVERAGE_CXXFLAGS]) AC_SUBST([CODE_COVERAGE_LIBS]) ]) AC_DEFUN([AX_CODE_COVERAGE],[ dnl Check for --enable-code-coverage # allow to override gcov location AC_ARG_WITH([gcov], [AS_HELP_STRING([--with-gcov[=GCOV]], [use given GCOV for coverage (GCOV=gcov).])], [_AX_CODE_COVERAGE_GCOV_PROG_WITH=$with_gcov], [_AX_CODE_COVERAGE_GCOV_PROG_WITH=gcov]) AC_MSG_CHECKING([whether to build with code coverage support]) AC_ARG_ENABLE([code-coverage], AS_HELP_STRING([--enable-code-coverage], [Whether to enable code coverage support]),, enable_code_coverage=no) AM_CONDITIONAL([CODE_COVERAGE_ENABLED], [test "x$enable_code_coverage" = xyes]) AC_SUBST([CODE_COVERAGE_ENABLED], [$enable_code_coverage]) AC_MSG_RESULT($enable_code_coverage) AS_IF([ test "x$enable_code_coverage" = xyes ], [ _AX_CODE_COVERAGE_ENABLED ]) _AX_CODE_COVERAGE_RULES ]) # =========================================================================== # https://www.gnu.org/software/autoconf-archive/ax_file_escapes.html # =========================================================================== # # SYNOPSIS # # AX_FILE_ESCAPES # # DESCRIPTION # # Writes the specified data to the specified file. # # LICENSE # # Copyright (c) 2008 Tom Howard # # Copying and distribution of this file, with or without modification, are # permitted in any medium without royalty provided the copyright notice # and this notice are preserved. This file is offered as-is, without any # warranty. #serial 8 AC_DEFUN([AX_FILE_ESCAPES],[ AX_DOLLAR="\$" AX_SRB="\\135" AX_SLB="\\133" AX_BS="\\\\" AX_DQ="\"" ]) # gpgme.m4 - autoconf macro to detect GPGME. # Copyright (C) 2002, 2003, 2004, 2014, 2018 g10 Code GmbH # # This file is free software; as a special exception the author gives # unlimited permission to copy and/or distribute it, with or without # modifications, as long as this notice is preserved. # # This file is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY, to the extent permitted by law; without even the # implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # Last-changed: 2020-11-20 AC_DEFUN([_AM_PATH_GPGME_CONFIG], [ AC_ARG_WITH(gpgme-prefix, AS_HELP_STRING([--with-gpgme-prefix=PFX], [prefix where GPGME is installed (optional)]), gpgme_config_prefix="$withval", gpgme_config_prefix="") if test x"${GPGME_CONFIG}" = x ; then if test x"${gpgme_config_prefix}" != x ; then GPGME_CONFIG="${gpgme_config_prefix}/bin/gpgme-config" else case "${SYSROOT}" in /*) if test -x "${SYSROOT}/bin/gpgme-config" ; then GPGME_CONFIG="${SYSROOT}/bin/gpgme-config" fi ;; '') ;; *) AC_MSG_WARN([Ignoring \$SYSROOT as it is not an absolute path.]) ;; esac fi fi use_gpgrt_config="" if test x"${GPGME_CONFIG}" = x -a x"$GPGRT_CONFIG" != x -a "$GPGRT_CONFIG" != "no"; then if $GPGRT_CONFIG gpgme --exists; then GPGME_CONFIG="$GPGRT_CONFIG gpgme" AC_MSG_NOTICE([Use gpgrt-config as gpgme-config]) use_gpgrt_config=yes fi fi if test -z "$use_gpgrt_config"; then AC_PATH_PROG(GPGME_CONFIG, gpgme-config, no) fi if test "$GPGME_CONFIG" != "no" ; then if test -z "$use_gpgrt_config"; then gpgme_version=`$GPGME_CONFIG --version` else gpgme_version=`$GPGME_CONFIG --modversion` fi fi gpgme_version_major=`echo $gpgme_version | \ sed 's/\([[0-9]]*\)\.\([[0-9]]*\)\.\([[0-9]]*\).*/\1/'` gpgme_version_minor=`echo $gpgme_version | \ sed 's/\([[0-9]]*\)\.\([[0-9]]*\)\.\([[0-9]]*\).*/\2/'` gpgme_version_micro=`echo $gpgme_version | \ sed 's/\([[0-9]]*\)\.\([[0-9]]*\)\.\([[0-9]]*\).*/\3/'` ]) AC_DEFUN([_AM_PATH_GPGME_CONFIG_HOST_CHECK], [ if test -z "$use_gpgrt_config"; then gpgme_config_host=`$GPGME_CONFIG --host 2>/dev/null || echo none` else gpgme_config_host=`$GPGME_CONFIG --variable=host 2>/dev/null || echo none` fi if test x"$gpgme_config_host" != xnone ; then if test x"$gpgme_config_host" != x"$host" ; then AC_MSG_WARN([[ *** *** The config script "$GPGME_CONFIG" was *** built for $gpgme_config_host and thus may not match the *** used host $host. *** You may want to use the configure option --with-gpgme-prefix *** to specify a matching config script or use \$SYSROOT. ***]]) gpg_config_script_warn="$gpg_config_script_warn gpgme" fi fi ]) dnl AM_PATH_GPGME([MINIMUM-VERSION, dnl [ACTION-IF-FOUND [, ACTION-IF-NOT-FOUND ]]]) dnl Test for libgpgme and define GPGME_CFLAGS and GPGME_LIBS. dnl dnl If a prefix option is not used, the config script is first dnl searched in $SYSROOT/bin and then along $PATH. If the used dnl config script does not match the host specification the script dnl is added to the gpg_config_script_warn variable. dnl AC_DEFUN([AM_PATH_GPGME], [ AC_REQUIRE([_AM_PATH_GPGME_CONFIG])dnl tmp=ifelse([$1], ,1:0.4.2,$1) if echo "$tmp" | grep ':' >/dev/null 2>/dev/null ; then req_gpgme_api=`echo "$tmp" | sed 's/\(.*\):\(.*\)/\1/'` min_gpgme_version=`echo "$tmp" | sed 's/\(.*\):\(.*\)/\2/'` else req_gpgme_api=0 min_gpgme_version="$tmp" fi AC_MSG_CHECKING(for GPGME - version >= $min_gpgme_version) ok=no if test "$GPGME_CONFIG" != "no" ; then req_major=`echo $min_gpgme_version | \ sed 's/\([[0-9]]*\)\.\([[0-9]]*\)\.\([[0-9]]*\)/\1/'` req_minor=`echo $min_gpgme_version | \ sed 's/\([[0-9]]*\)\.\([[0-9]]*\)\.\([[0-9]]*\)/\2/'` req_micro=`echo $min_gpgme_version | \ sed 's/\([[0-9]]*\)\.\([[0-9]]*\)\.\([[0-9]]*\)/\3/'` if test "$gpgme_version_major" -gt "$req_major"; then ok=yes else if test "$gpgme_version_major" -eq "$req_major"; then if test "$gpgme_version_minor" -gt "$req_minor"; then ok=yes else if test "$gpgme_version_minor" -eq "$req_minor"; then if test "$gpgme_version_micro" -ge "$req_micro"; then ok=yes fi fi fi fi fi fi if test $ok = yes; then # If we have a recent GPGME, we should also check that the # API is compatible. if test "$req_gpgme_api" -gt 0 ; then if test -z "$use_gpgrt_config"; then tmp=`$GPGME_CONFIG --api-version 2>/dev/null || echo 0` else tmp=`$GPGME_CONFIG --variable=api_version 2>/dev/null || echo 0` fi if test "$tmp" -gt 0 ; then if test "$req_gpgme_api" -ne "$tmp" ; then ok=no fi fi fi fi if test $ok = yes; then GPGME_CFLAGS=`$GPGME_CONFIG --cflags` GPGME_LIBS=`$GPGME_CONFIG --libs` AC_MSG_RESULT(yes) ifelse([$2], , :, [$2]) _AM_PATH_GPGME_CONFIG_HOST_CHECK else GPGME_CFLAGS="" GPGME_LIBS="" AC_MSG_RESULT(no) ifelse([$3], , :, [$3]) fi AC_SUBST(GPGME_CFLAGS) AC_SUBST(GPGME_LIBS) ]) dnl AM_PATH_GPGME_PTHREAD([MINIMUM-VERSION, dnl [ACTION-IF-FOUND [, ACTION-IF-NOT-FOUND ]]]) dnl Test for libgpgme and define GPGME_PTHREAD_CFLAGS dnl and GPGME_PTHREAD_LIBS. dnl AC_DEFUN([AM_PATH_GPGME_PTHREAD], [ AC_REQUIRE([_AM_PATH_GPGME_CONFIG])dnl tmp=ifelse([$1], ,1:0.4.2,$1) if echo "$tmp" | grep ':' >/dev/null 2>/dev/null ; then req_gpgme_api=`echo "$tmp" | sed 's/\(.*\):\(.*\)/\1/'` min_gpgme_version=`echo "$tmp" | sed 's/\(.*\):\(.*\)/\2/'` else req_gpgme_api=0 min_gpgme_version="$tmp" fi AC_MSG_CHECKING(for GPGME pthread - version >= $min_gpgme_version) ok=no if test "$GPGME_CONFIG" != "no" ; then if `$GPGME_CONFIG --thread=pthread 2> /dev/null` ; then req_major=`echo $min_gpgme_version | \ sed 's/\([[0-9]]*\)\.\([[0-9]]*\)\.\([[0-9]]*\)/\1/'` req_minor=`echo $min_gpgme_version | \ sed 's/\([[0-9]]*\)\.\([[0-9]]*\)\.\([[0-9]]*\)/\2/'` req_micro=`echo $min_gpgme_version | \ sed 's/\([[0-9]]*\)\.\([[0-9]]*\)\.\([[0-9]]*\)/\3/'` if test "$gpgme_version_major" -gt "$req_major"; then ok=yes else if test "$gpgme_version_major" -eq "$req_major"; then if test "$gpgme_version_minor" -gt "$req_minor"; then ok=yes else if test "$gpgme_version_minor" -eq "$req_minor"; then if test "$gpgme_version_micro" -ge "$req_micro"; then ok=yes fi fi fi fi fi fi fi if test $ok = yes; then # If we have a recent GPGME, we should also check that the # API is compatible. if test "$req_gpgme_api" -gt 0 ; then tmp=`$GPGME_CONFIG --api-version 2>/dev/null || echo 0` if test "$tmp" -gt 0 ; then if test "$req_gpgme_api" -ne "$tmp" ; then ok=no fi fi fi fi if test $ok = yes; then GPGME_PTHREAD_CFLAGS=`$GPGME_CONFIG --thread=pthread --cflags` GPGME_PTHREAD_LIBS=`$GPGME_CONFIG --thread=pthread --libs` AC_MSG_RESULT(yes) ifelse([$2], , :, [$2]) _AM_PATH_GPGME_CONFIG_HOST_CHECK else GPGME_PTHREAD_CFLAGS="" GPGME_PTHREAD_LIBS="" AC_MSG_RESULT(no) ifelse([$3], , :, [$3]) fi AC_SUBST(GPGME_PTHREAD_CFLAGS) AC_SUBST(GPGME_PTHREAD_LIBS) ]) dnl AM_PATH_GPGME_GLIB([MINIMUM-VERSION, dnl [ACTION-IF-FOUND [, ACTION-IF-NOT-FOUND ]]]) dnl Test for libgpgme-glib and define GPGME_GLIB_CFLAGS and GPGME_GLIB_LIBS. dnl AC_DEFUN([AM_PATH_GPGME_GLIB], [ AC_REQUIRE([_AM_PATH_GPGME_CONFIG])dnl tmp=ifelse([$1], ,1:0.4.2,$1) if echo "$tmp" | grep ':' >/dev/null 2>/dev/null ; then req_gpgme_api=`echo "$tmp" | sed 's/\(.*\):\(.*\)/\1/'` min_gpgme_version=`echo "$tmp" | sed 's/\(.*\):\(.*\)/\2/'` else req_gpgme_api=0 min_gpgme_version="$tmp" fi AC_MSG_CHECKING(for GPGME - version >= $min_gpgme_version) ok=no if test "$GPGME_CONFIG" != "no" ; then req_major=`echo $min_gpgme_version | \ sed 's/\([[0-9]]*\)\.\([[0-9]]*\)\.\([[0-9]]*\)/\1/'` req_minor=`echo $min_gpgme_version | \ sed 's/\([[0-9]]*\)\.\([[0-9]]*\)\.\([[0-9]]*\)/\2/'` req_micro=`echo $min_gpgme_version | \ sed 's/\([[0-9]]*\)\.\([[0-9]]*\)\.\([[0-9]]*\)/\3/'` if test "$gpgme_version_major" -gt "$req_major"; then ok=yes else if test "$gpgme_version_major" -eq "$req_major"; then if test "$gpgme_version_minor" -gt "$req_minor"; then ok=yes else if test "$gpgme_version_minor" -eq "$req_minor"; then if test "$gpgme_version_micro" -ge "$req_micro"; then ok=yes fi fi fi fi fi fi if test $ok = yes; then # If we have a recent GPGME, we should also check that the # API is compatible. if test "$req_gpgme_api" -gt 0 ; then if test -z "$use_gpgrt_config"; then tmp=`$GPGME_CONFIG --api-version 2>/dev/null || echo 0` else tmp=`$GPGME_CONFIG --variable=api_version 2>/dev/null || echo 0` fi if test "$tmp" -gt 0 ; then if test "$req_gpgme_api" -ne "$tmp" ; then ok=no fi fi fi fi if test $ok = yes; then if test -z "$use_gpgrt_config"; then GPGME_GLIB_CFLAGS=`$GPGME_CONFIG --glib --cflags` GPGME_GLIB_LIBS=`$GPGME_CONFIG --glib --libs` else if $GPGRT_CONFIG gpgme-glib --exists; then GPGME_CONFIG="$GPGRT_CONFIG gpgme-glib" GPGME_GLIB_CFLAGS=`$GPGME_CONFIG --cflags` GPGME_GLIB_LIBS=`$GPGME_CONFIG --libs` else ok = no fi fi fi if test $ok = yes; then AC_MSG_RESULT(yes) ifelse([$2], , :, [$2]) _AM_PATH_GPGME_CONFIG_HOST_CHECK else GPGME_GLIB_CFLAGS="" GPGME_GLIB_LIBS="" AC_MSG_RESULT(no) ifelse([$3], , :, [$3]) fi AC_SUBST(GPGME_GLIB_CFLAGS) AC_SUBST(GPGME_GLIB_LIBS) ]) # pkg.m4 - Macros to locate and utilise pkg-config. -*- Autoconf -*- # serial 11 (pkg-config-0.29.1) dnl Copyright © 2004 Scott James Remnant . dnl Copyright © 2012-2015 Dan Nicholson dnl dnl This program is free software; you can redistribute it and/or modify dnl it under the terms of the GNU General Public License as published by dnl the Free Software Foundation; either version 2 of the License, or dnl (at your option) any later version. dnl dnl This program is distributed in the hope that it will be useful, but dnl WITHOUT ANY WARRANTY; without even the implied warranty of dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU dnl General Public License for more details. dnl dnl You should have received a copy of the GNU General Public License dnl along with this program; if not, write to the Free Software dnl Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA dnl 02111-1307, USA. dnl dnl As a special exception to the GNU General Public License, if you dnl distribute this file as part of a program that contains a dnl configuration script generated by Autoconf, you may include it under dnl the same distribution terms that you use for the rest of that dnl program. dnl PKG_PREREQ(MIN-VERSION) dnl ----------------------- dnl Since: 0.29 dnl dnl Verify that the version of the pkg-config macros are at least dnl MIN-VERSION. Unlike PKG_PROG_PKG_CONFIG, which checks the user's dnl installed version of pkg-config, this checks the developer's version dnl of pkg.m4 when generating configure. dnl dnl To ensure that this macro is defined, also add: dnl m4_ifndef([PKG_PREREQ], dnl [m4_fatal([must install pkg-config 0.29 or later before running autoconf/autogen])]) dnl dnl See the "Since" comment for each macro you use to see what version dnl of the macros you require. m4_defun([PKG_PREREQ], [m4_define([PKG_MACROS_VERSION], [0.29.1]) m4_if(m4_version_compare(PKG_MACROS_VERSION, [$1]), -1, [m4_fatal([pkg.m4 version $1 or higher is required but ]PKG_MACROS_VERSION[ found])]) ])dnl PKG_PREREQ dnl PKG_PROG_PKG_CONFIG([MIN-VERSION]) dnl ---------------------------------- dnl Since: 0.16 dnl dnl Search for the pkg-config tool and set the PKG_CONFIG variable to dnl first found in the path. Checks that the version of pkg-config found dnl is at least MIN-VERSION. If MIN-VERSION is not specified, 0.9.0 is dnl used since that's the first version where most current features of dnl pkg-config existed. AC_DEFUN([PKG_PROG_PKG_CONFIG], [m4_pattern_forbid([^_?PKG_[A-Z_]+$]) m4_pattern_allow([^PKG_CONFIG(_(PATH|LIBDIR|SYSROOT_DIR|ALLOW_SYSTEM_(CFLAGS|LIBS)))?$]) m4_pattern_allow([^PKG_CONFIG_(DISABLE_UNINSTALLED|TOP_BUILD_DIR|DEBUG_SPEW)$]) AC_ARG_VAR([PKG_CONFIG], [path to pkg-config utility]) AC_ARG_VAR([PKG_CONFIG_PATH], [directories to add to pkg-config's search path]) AC_ARG_VAR([PKG_CONFIG_LIBDIR], [path overriding pkg-config's built-in search path]) if test "x$ac_cv_env_PKG_CONFIG_set" != "xset"; then AC_PATH_TOOL([PKG_CONFIG], [pkg-config]) fi if test -n "$PKG_CONFIG"; then _pkg_min_version=m4_default([$1], [0.9.0]) AC_MSG_CHECKING([pkg-config is at least version $_pkg_min_version]) if $PKG_CONFIG --atleast-pkgconfig-version $_pkg_min_version; then AC_MSG_RESULT([yes]) else AC_MSG_RESULT([no]) PKG_CONFIG="" fi fi[]dnl ])dnl PKG_PROG_PKG_CONFIG dnl PKG_CHECK_EXISTS(MODULES, [ACTION-IF-FOUND], [ACTION-IF-NOT-FOUND]) dnl ------------------------------------------------------------------- dnl Since: 0.18 dnl dnl Check to see whether a particular set of modules exists. Similar to dnl PKG_CHECK_MODULES(), but does not set variables or print errors. dnl dnl Please remember that m4 expands AC_REQUIRE([PKG_PROG_PKG_CONFIG]) dnl only at the first occurence in configure.ac, so if the first place dnl it's called might be skipped (such as if it is within an "if", you dnl have to call PKG_CHECK_EXISTS manually AC_DEFUN([PKG_CHECK_EXISTS], [AC_REQUIRE([PKG_PROG_PKG_CONFIG])dnl if test -n "$PKG_CONFIG" && \ AC_RUN_LOG([$PKG_CONFIG --exists --print-errors "$1"]); then m4_default([$2], [:]) m4_ifvaln([$3], [else $3])dnl fi]) dnl _PKG_CONFIG([VARIABLE], [COMMAND], [MODULES]) dnl --------------------------------------------- dnl Internal wrapper calling pkg-config via PKG_CONFIG and setting dnl pkg_failed based on the result. m4_define([_PKG_CONFIG], [if test -n "$$1"; then pkg_cv_[]$1="$$1" elif test -n "$PKG_CONFIG"; then PKG_CHECK_EXISTS([$3], [pkg_cv_[]$1=`$PKG_CONFIG --[]$2 "$3" 2>/dev/null` test "x$?" != "x0" && pkg_failed=yes ], [pkg_failed=yes]) else pkg_failed=untried fi[]dnl ])dnl _PKG_CONFIG dnl _PKG_SHORT_ERRORS_SUPPORTED dnl --------------------------- dnl Internal check to see if pkg-config supports short errors. AC_DEFUN([_PKG_SHORT_ERRORS_SUPPORTED], [AC_REQUIRE([PKG_PROG_PKG_CONFIG]) if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then _pkg_short_errors_supported=yes else _pkg_short_errors_supported=no fi[]dnl ])dnl _PKG_SHORT_ERRORS_SUPPORTED dnl PKG_CHECK_MODULES(VARIABLE-PREFIX, MODULES, [ACTION-IF-FOUND], dnl [ACTION-IF-NOT-FOUND]) dnl -------------------------------------------------------------- dnl Since: 0.4.0 dnl dnl Note that if there is a possibility the first call to dnl PKG_CHECK_MODULES might not happen, you should be sure to include an dnl explicit call to PKG_PROG_PKG_CONFIG in your configure.ac AC_DEFUN([PKG_CHECK_MODULES], [AC_REQUIRE([PKG_PROG_PKG_CONFIG])dnl AC_ARG_VAR([$1][_CFLAGS], [C compiler flags for $1, overriding pkg-config])dnl AC_ARG_VAR([$1][_LIBS], [linker flags for $1, overriding pkg-config])dnl pkg_failed=no AC_MSG_CHECKING([for $1]) _PKG_CONFIG([$1][_CFLAGS], [cflags], [$2]) _PKG_CONFIG([$1][_LIBS], [libs], [$2]) m4_define([_PKG_TEXT], [Alternatively, you may set the environment variables $1[]_CFLAGS and $1[]_LIBS to avoid the need to call pkg-config. See the pkg-config man page for more details.]) if test $pkg_failed = yes; then AC_MSG_RESULT([no]) _PKG_SHORT_ERRORS_SUPPORTED if test $_pkg_short_errors_supported = yes; then $1[]_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "$2" 2>&1` else $1[]_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "$2" 2>&1` fi # Put the nasty error message in config.log where it belongs echo "$$1[]_PKG_ERRORS" >&AS_MESSAGE_LOG_FD m4_default([$4], [AC_MSG_ERROR( [Package requirements ($2) were not met: $$1_PKG_ERRORS Consider adjusting the PKG_CONFIG_PATH environment variable if you installed software in a non-standard prefix. _PKG_TEXT])[]dnl ]) elif test $pkg_failed = untried; then AC_MSG_RESULT([no]) m4_default([$4], [AC_MSG_FAILURE( [The pkg-config script could not be found or is too old. Make sure it is in your PATH or set the PKG_CONFIG environment variable to the full path to pkg-config. _PKG_TEXT To get pkg-config, see .])[]dnl ]) else $1[]_CFLAGS=$pkg_cv_[]$1[]_CFLAGS $1[]_LIBS=$pkg_cv_[]$1[]_LIBS AC_MSG_RESULT([yes]) $3 fi[]dnl ])dnl PKG_CHECK_MODULES dnl PKG_CHECK_MODULES_STATIC(VARIABLE-PREFIX, MODULES, [ACTION-IF-FOUND], dnl [ACTION-IF-NOT-FOUND]) dnl --------------------------------------------------------------------- dnl Since: 0.29 dnl dnl Checks for existence of MODULES and gathers its build flags with dnl static libraries enabled. Sets VARIABLE-PREFIX_CFLAGS from --cflags dnl and VARIABLE-PREFIX_LIBS from --libs. dnl dnl Note that if there is a possibility the first call to dnl PKG_CHECK_MODULES_STATIC might not happen, you should be sure to dnl include an explicit call to PKG_PROG_PKG_CONFIG in your dnl configure.ac. AC_DEFUN([PKG_CHECK_MODULES_STATIC], [AC_REQUIRE([PKG_PROG_PKG_CONFIG])dnl _save_PKG_CONFIG=$PKG_CONFIG PKG_CONFIG="$PKG_CONFIG --static" PKG_CHECK_MODULES($@) PKG_CONFIG=$_save_PKG_CONFIG[]dnl ])dnl PKG_CHECK_MODULES_STATIC dnl PKG_INSTALLDIR([DIRECTORY]) dnl ------------------------- dnl Since: 0.27 dnl dnl Substitutes the variable pkgconfigdir as the location where a module dnl should install pkg-config .pc files. By default the directory is dnl $libdir/pkgconfig, but the default can be changed by passing dnl DIRECTORY. The user can override through the --with-pkgconfigdir dnl parameter. AC_DEFUN([PKG_INSTALLDIR], [m4_pushdef([pkg_default], [m4_default([$1], ['${libdir}/pkgconfig'])]) m4_pushdef([pkg_description], [pkg-config installation directory @<:@]pkg_default[@:>@]) AC_ARG_WITH([pkgconfigdir], [AS_HELP_STRING([--with-pkgconfigdir], pkg_description)],, [with_pkgconfigdir=]pkg_default) AC_SUBST([pkgconfigdir], [$with_pkgconfigdir]) m4_popdef([pkg_default]) m4_popdef([pkg_description]) ])dnl PKG_INSTALLDIR dnl PKG_NOARCH_INSTALLDIR([DIRECTORY]) dnl -------------------------------- dnl Since: 0.27 dnl dnl Substitutes the variable noarch_pkgconfigdir as the location where a dnl module should install arch-independent pkg-config .pc files. By dnl default the directory is $datadir/pkgconfig, but the default can be dnl changed by passing DIRECTORY. The user can override through the dnl --with-noarch-pkgconfigdir parameter. AC_DEFUN([PKG_NOARCH_INSTALLDIR], [m4_pushdef([pkg_default], [m4_default([$1], ['${datadir}/pkgconfig'])]) m4_pushdef([pkg_description], [pkg-config arch-independent installation directory @<:@]pkg_default[@:>@]) AC_ARG_WITH([noarch-pkgconfigdir], [AS_HELP_STRING([--with-noarch-pkgconfigdir], pkg_description)],, [with_noarch_pkgconfigdir=]pkg_default) AC_SUBST([noarch_pkgconfigdir], [$with_noarch_pkgconfigdir]) m4_popdef([pkg_default]) m4_popdef([pkg_description]) ])dnl PKG_NOARCH_INSTALLDIR dnl PKG_CHECK_VAR(VARIABLE, MODULE, CONFIG-VARIABLE, dnl [ACTION-IF-FOUND], [ACTION-IF-NOT-FOUND]) dnl ------------------------------------------- dnl Since: 0.28 dnl dnl Retrieves the value of the pkg-config variable for the given module. AC_DEFUN([PKG_CHECK_VAR], [AC_REQUIRE([PKG_PROG_PKG_CONFIG])dnl AC_ARG_VAR([$1], [value of $3 for $2, overriding pkg-config])dnl _PKG_CONFIG([$1], [variable="][$3]["], [$2]) AS_VAR_COPY([$1], [pkg_cv_][$1]) AS_VAR_IF([$1], [""], [$5], [$4])dnl ])dnl PKG_CHECK_VAR dnl PKG_WITH_MODULES(VARIABLE-PREFIX, MODULES, dnl [ACTION-IF-FOUND],[ACTION-IF-NOT-FOUND], dnl [DESCRIPTION], [DEFAULT]) dnl ------------------------------------------ dnl dnl Prepare a "--with-" configure option using the lowercase dnl [VARIABLE-PREFIX] name, merging the behaviour of AC_ARG_WITH and dnl PKG_CHECK_MODULES in a single macro. AC_DEFUN([PKG_WITH_MODULES], [ m4_pushdef([with_arg], m4_tolower([$1])) m4_pushdef([description], [m4_default([$5], [build with ]with_arg[ support])]) m4_pushdef([def_arg], [m4_default([$6], [auto])]) m4_pushdef([def_action_if_found], [AS_TR_SH([with_]with_arg)=yes]) m4_pushdef([def_action_if_not_found], [AS_TR_SH([with_]with_arg)=no]) m4_case(def_arg, [yes],[m4_pushdef([with_without], [--without-]with_arg)], [m4_pushdef([with_without],[--with-]with_arg)]) AC_ARG_WITH(with_arg, AS_HELP_STRING(with_without, description[ @<:@default=]def_arg[@:>@]),, [AS_TR_SH([with_]with_arg)=def_arg]) AS_CASE([$AS_TR_SH([with_]with_arg)], [yes],[PKG_CHECK_MODULES([$1],[$2],$3,$4)], [auto],[PKG_CHECK_MODULES([$1],[$2], [m4_n([def_action_if_found]) $3], [m4_n([def_action_if_not_found]) $4])]) m4_popdef([with_arg]) m4_popdef([description]) m4_popdef([def_arg]) ])dnl PKG_WITH_MODULES dnl PKG_HAVE_WITH_MODULES(VARIABLE-PREFIX, MODULES, dnl [DESCRIPTION], [DEFAULT]) dnl ----------------------------------------------- dnl dnl Convenience macro to trigger AM_CONDITIONAL after PKG_WITH_MODULES dnl check._[VARIABLE-PREFIX] is exported as make variable. AC_DEFUN([PKG_HAVE_WITH_MODULES], [ PKG_WITH_MODULES([$1],[$2],,,[$3],[$4]) AM_CONDITIONAL([HAVE_][$1], [test "$AS_TR_SH([with_]m4_tolower([$1]))" = "yes"]) ])dnl PKG_HAVE_WITH_MODULES dnl PKG_HAVE_DEFINE_WITH_MODULES(VARIABLE-PREFIX, MODULES, dnl [DESCRIPTION], [DEFAULT]) dnl ------------------------------------------------------ dnl dnl Convenience macro to run AM_CONDITIONAL and AC_DEFINE after dnl PKG_WITH_MODULES check. HAVE_[VARIABLE-PREFIX] is exported as make dnl and preprocessor variable. AC_DEFUN([PKG_HAVE_DEFINE_WITH_MODULES], [ PKG_HAVE_WITH_MODULES([$1],[$2],[$3],[$4]) AS_IF([test "$AS_TR_SH([with_]m4_tolower([$1]))" = "yes"], [AC_DEFINE([HAVE_][$1], 1, [Enable ]m4_tolower([$1])[ support])]) ])dnl PKG_HAVE_DEFINE_WITH_MODULES # Copyright (C) 2002-2021 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # AM_AUTOMAKE_VERSION(VERSION) # ---------------------------- # Automake X.Y traces this macro to ensure aclocal.m4 has been # generated from the m4 files accompanying Automake X.Y. # (This private macro should not be called outside this file.) AC_DEFUN([AM_AUTOMAKE_VERSION], [am__api_version='1.16' dnl Some users find AM_AUTOMAKE_VERSION and mistake it for a way to dnl require some minimum version. Point them to the right macro. m4_if([$1], [1.16.4], [], [AC_FATAL([Do not call $0, use AM_INIT_AUTOMAKE([$1]).])])dnl ]) # _AM_AUTOCONF_VERSION(VERSION) # ----------------------------- # aclocal traces this macro to find the Autoconf version. # This is a private macro too. Using m4_define simplifies # the logic in aclocal, which can simply ignore this definition. m4_define([_AM_AUTOCONF_VERSION], []) # AM_SET_CURRENT_AUTOMAKE_VERSION # ------------------------------- # Call AM_AUTOMAKE_VERSION and AM_AUTOMAKE_VERSION so they can be traced. # This function is AC_REQUIREd by AM_INIT_AUTOMAKE. AC_DEFUN([AM_SET_CURRENT_AUTOMAKE_VERSION], [AM_AUTOMAKE_VERSION([1.16.4])dnl m4_ifndef([AC_AUTOCONF_VERSION], [m4_copy([m4_PACKAGE_VERSION], [AC_AUTOCONF_VERSION])])dnl _AM_AUTOCONF_VERSION(m4_defn([AC_AUTOCONF_VERSION]))]) # AM_AUX_DIR_EXPAND -*- Autoconf -*- # Copyright (C) 2001-2021 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # For projects using AC_CONFIG_AUX_DIR([foo]), Autoconf sets # $ac_aux_dir to '$srcdir/foo'. In other projects, it is set to # '$srcdir', '$srcdir/..', or '$srcdir/../..'. # # Of course, Automake must honor this variable whenever it calls a # tool from the auxiliary directory. The problem is that $srcdir (and # therefore $ac_aux_dir as well) can be either absolute or relative, # depending on how configure is run. This is pretty annoying, since # it makes $ac_aux_dir quite unusable in subdirectories: in the top # source directory, any form will work fine, but in subdirectories a # relative path needs to be adjusted first. # # $ac_aux_dir/missing # fails when called from a subdirectory if $ac_aux_dir is relative # $top_srcdir/$ac_aux_dir/missing # fails if $ac_aux_dir is absolute, # fails when called from a subdirectory in a VPATH build with # a relative $ac_aux_dir # # The reason of the latter failure is that $top_srcdir and $ac_aux_dir # are both prefixed by $srcdir. In an in-source build this is usually # harmless because $srcdir is '.', but things will broke when you # start a VPATH build or use an absolute $srcdir. # # So we could use something similar to $top_srcdir/$ac_aux_dir/missing, # iff we strip the leading $srcdir from $ac_aux_dir. That would be: # am_aux_dir='\$(top_srcdir)/'`expr "$ac_aux_dir" : "$srcdir//*\(.*\)"` # and then we would define $MISSING as # MISSING="\${SHELL} $am_aux_dir/missing" # This will work as long as MISSING is not called from configure, because # unfortunately $(top_srcdir) has no meaning in configure. # However there are other variables, like CC, which are often used in # configure, and could therefore not use this "fixed" $ac_aux_dir. # # Another solution, used here, is to always expand $ac_aux_dir to an # absolute PATH. The drawback is that using absolute paths prevent a # configured tree to be moved without reconfiguration. AC_DEFUN([AM_AUX_DIR_EXPAND], [AC_REQUIRE([AC_CONFIG_AUX_DIR_DEFAULT])dnl # Expand $ac_aux_dir to an absolute path. am_aux_dir=`cd "$ac_aux_dir" && pwd` ]) # AM_CONDITIONAL -*- Autoconf -*- # Copyright (C) 1997-2021 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # AM_CONDITIONAL(NAME, SHELL-CONDITION) # ------------------------------------- # Define a conditional. AC_DEFUN([AM_CONDITIONAL], [AC_PREREQ([2.52])dnl m4_if([$1], [TRUE], [AC_FATAL([$0: invalid condition: $1])], [$1], [FALSE], [AC_FATAL([$0: invalid condition: $1])])dnl AC_SUBST([$1_TRUE])dnl AC_SUBST([$1_FALSE])dnl _AM_SUBST_NOTMAKE([$1_TRUE])dnl _AM_SUBST_NOTMAKE([$1_FALSE])dnl m4_define([_AM_COND_VALUE_$1], [$2])dnl if $2; then $1_TRUE= $1_FALSE='#' else $1_TRUE='#' $1_FALSE= fi AC_CONFIG_COMMANDS_PRE( [if test -z "${$1_TRUE}" && test -z "${$1_FALSE}"; then AC_MSG_ERROR([[conditional "$1" was never defined. Usually this means the macro was only invoked conditionally.]]) fi])]) # Copyright (C) 1999-2021 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # There are a few dirty hacks below to avoid letting 'AC_PROG_CC' be # written in clear, in which case automake, when reading aclocal.m4, # will think it sees a *use*, and therefore will trigger all it's # C support machinery. Also note that it means that autoscan, seeing # CC etc. in the Makefile, will ask for an AC_PROG_CC use... # _AM_DEPENDENCIES(NAME) # ---------------------- # See how the compiler implements dependency checking. # NAME is "CC", "CXX", "OBJC", "OBJCXX", "UPC", or "GJC". # We try a few techniques and use that to set a single cache variable. # # We don't AC_REQUIRE the corresponding AC_PROG_CC since the latter was # modified to invoke _AM_DEPENDENCIES(CC); we would have a circular # dependency, and given that the user is not expected to run this macro, # just rely on AC_PROG_CC. AC_DEFUN([_AM_DEPENDENCIES], [AC_REQUIRE([AM_SET_DEPDIR])dnl AC_REQUIRE([AM_OUTPUT_DEPENDENCY_COMMANDS])dnl AC_REQUIRE([AM_MAKE_INCLUDE])dnl AC_REQUIRE([AM_DEP_TRACK])dnl m4_if([$1], [CC], [depcc="$CC" am_compiler_list=], [$1], [CXX], [depcc="$CXX" am_compiler_list=], [$1], [OBJC], [depcc="$OBJC" am_compiler_list='gcc3 gcc'], [$1], [OBJCXX], [depcc="$OBJCXX" am_compiler_list='gcc3 gcc'], [$1], [UPC], [depcc="$UPC" am_compiler_list=], [$1], [GCJ], [depcc="$GCJ" am_compiler_list='gcc3 gcc'], [depcc="$$1" am_compiler_list=]) AC_CACHE_CHECK([dependency style of $depcc], [am_cv_$1_dependencies_compiler_type], [if test -z "$AMDEP_TRUE" && test -f "$am_depcomp"; then # We make a subdir and do the tests there. Otherwise we can end up # making bogus files that we don't know about and never remove. For # instance it was reported that on HP-UX the gcc test will end up # making a dummy file named 'D' -- because '-MD' means "put the output # in D". rm -rf conftest.dir mkdir conftest.dir # Copy depcomp to subdir because otherwise we won't find it if we're # using a relative directory. cp "$am_depcomp" conftest.dir cd conftest.dir # We will build objects and dependencies in a subdirectory because # it helps to detect inapplicable dependency modes. For instance # both Tru64's cc and ICC support -MD to output dependencies as a # side effect of compilation, but ICC will put the dependencies in # the current directory while Tru64 will put them in the object # directory. mkdir sub am_cv_$1_dependencies_compiler_type=none if test "$am_compiler_list" = ""; then am_compiler_list=`sed -n ['s/^#*\([a-zA-Z0-9]*\))$/\1/p'] < ./depcomp` fi am__universal=false m4_case([$1], [CC], [case " $depcc " in #( *\ -arch\ *\ -arch\ *) am__universal=true ;; esac], [CXX], [case " $depcc " in #( *\ -arch\ *\ -arch\ *) am__universal=true ;; esac]) for depmode in $am_compiler_list; do # Setup a source with many dependencies, because some compilers # like to wrap large dependency lists on column 80 (with \), and # we should not choose a depcomp mode which is confused by this. # # We need to recreate these files for each test, as the compiler may # overwrite some of them when testing with obscure command lines. # This happens at least with the AIX C compiler. : > sub/conftest.c for i in 1 2 3 4 5 6; do echo '#include "conftst'$i'.h"' >> sub/conftest.c # Using ": > sub/conftst$i.h" creates only sub/conftst1.h with # Solaris 10 /bin/sh. echo '/* dummy */' > sub/conftst$i.h done echo "${am__include} ${am__quote}sub/conftest.Po${am__quote}" > confmf # We check with '-c' and '-o' for the sake of the "dashmstdout" # mode. It turns out that the SunPro C++ compiler does not properly # handle '-M -o', and we need to detect this. Also, some Intel # versions had trouble with output in subdirs. am__obj=sub/conftest.${OBJEXT-o} am__minus_obj="-o $am__obj" case $depmode in gcc) # This depmode causes a compiler race in universal mode. test "$am__universal" = false || continue ;; nosideeffect) # After this tag, mechanisms are not by side-effect, so they'll # only be used when explicitly requested. if test "x$enable_dependency_tracking" = xyes; then continue else break fi ;; msvc7 | msvc7msys | msvisualcpp | msvcmsys) # This compiler won't grok '-c -o', but also, the minuso test has # not run yet. These depmodes are late enough in the game, and # so weak that their functioning should not be impacted. am__obj=conftest.${OBJEXT-o} am__minus_obj= ;; none) break ;; esac if depmode=$depmode \ source=sub/conftest.c object=$am__obj \ depfile=sub/conftest.Po tmpdepfile=sub/conftest.TPo \ $SHELL ./depcomp $depcc -c $am__minus_obj sub/conftest.c \ >/dev/null 2>conftest.err && grep sub/conftst1.h sub/conftest.Po > /dev/null 2>&1 && grep sub/conftst6.h sub/conftest.Po > /dev/null 2>&1 && grep $am__obj sub/conftest.Po > /dev/null 2>&1 && ${MAKE-make} -s -f confmf > /dev/null 2>&1; then # icc doesn't choke on unknown options, it will just issue warnings # or remarks (even with -Werror). So we grep stderr for any message # that says an option was ignored or not supported. # When given -MP, icc 7.0 and 7.1 complain thusly: # icc: Command line warning: ignoring option '-M'; no argument required # The diagnosis changed in icc 8.0: # icc: Command line remark: option '-MP' not supported if (grep 'ignoring option' conftest.err || grep 'not supported' conftest.err) >/dev/null 2>&1; then :; else am_cv_$1_dependencies_compiler_type=$depmode break fi fi done cd .. rm -rf conftest.dir else am_cv_$1_dependencies_compiler_type=none fi ]) AC_SUBST([$1DEPMODE], [depmode=$am_cv_$1_dependencies_compiler_type]) AM_CONDITIONAL([am__fastdep$1], [ test "x$enable_dependency_tracking" != xno \ && test "$am_cv_$1_dependencies_compiler_type" = gcc3]) ]) # AM_SET_DEPDIR # ------------- # Choose a directory name for dependency files. # This macro is AC_REQUIREd in _AM_DEPENDENCIES. AC_DEFUN([AM_SET_DEPDIR], [AC_REQUIRE([AM_SET_LEADING_DOT])dnl AC_SUBST([DEPDIR], ["${am__leading_dot}deps"])dnl ]) # AM_DEP_TRACK # ------------ AC_DEFUN([AM_DEP_TRACK], [AC_ARG_ENABLE([dependency-tracking], [dnl AS_HELP_STRING( [--enable-dependency-tracking], [do not reject slow dependency extractors]) AS_HELP_STRING( [--disable-dependency-tracking], [speeds up one-time build])]) if test "x$enable_dependency_tracking" != xno; then am_depcomp="$ac_aux_dir/depcomp" AMDEPBACKSLASH='\' am__nodep='_no' fi AM_CONDITIONAL([AMDEP], [test "x$enable_dependency_tracking" != xno]) AC_SUBST([AMDEPBACKSLASH])dnl _AM_SUBST_NOTMAKE([AMDEPBACKSLASH])dnl AC_SUBST([am__nodep])dnl _AM_SUBST_NOTMAKE([am__nodep])dnl ]) # Generate code to set up dependency tracking. -*- Autoconf -*- # Copyright (C) 1999-2021 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # _AM_OUTPUT_DEPENDENCY_COMMANDS # ------------------------------ AC_DEFUN([_AM_OUTPUT_DEPENDENCY_COMMANDS], [{ # Older Autoconf quotes --file arguments for eval, but not when files # are listed without --file. Let's play safe and only enable the eval # if we detect the quoting. # TODO: see whether this extra hack can be removed once we start # requiring Autoconf 2.70 or later. AS_CASE([$CONFIG_FILES], [*\'*], [eval set x "$CONFIG_FILES"], [*], [set x $CONFIG_FILES]) shift # Used to flag and report bootstrapping failures. am_rc=0 for am_mf do # Strip MF so we end up with the name of the file. am_mf=`AS_ECHO(["$am_mf"]) | sed -e 's/:.*$//'` # Check whether this is an Automake generated Makefile which includes # dependency-tracking related rules and includes. # Grep'ing the whole file directly is not great: AIX grep has a line # limit of 2048, but all sed's we know have understand at least 4000. sed -n 's,^am--depfiles:.*,X,p' "$am_mf" | grep X >/dev/null 2>&1 \ || continue am_dirpart=`AS_DIRNAME(["$am_mf"])` am_filepart=`AS_BASENAME(["$am_mf"])` AM_RUN_LOG([cd "$am_dirpart" \ && sed -e '/# am--include-marker/d' "$am_filepart" \ | $MAKE -f - am--depfiles]) || am_rc=$? done if test $am_rc -ne 0; then AC_MSG_FAILURE([Something went wrong bootstrapping makefile fragments for automatic dependency tracking. If GNU make was not used, consider re-running the configure script with MAKE="gmake" (or whatever is necessary). You can also try re-running configure with the '--disable-dependency-tracking' option to at least be able to build the package (albeit without support for automatic dependency tracking).]) fi AS_UNSET([am_dirpart]) AS_UNSET([am_filepart]) AS_UNSET([am_mf]) AS_UNSET([am_rc]) rm -f conftest-deps.mk } ])# _AM_OUTPUT_DEPENDENCY_COMMANDS # AM_OUTPUT_DEPENDENCY_COMMANDS # ----------------------------- # This macro should only be invoked once -- use via AC_REQUIRE. # # This code is only required when automatic dependency tracking is enabled. # This creates each '.Po' and '.Plo' makefile fragment that we'll need in # order to bootstrap the dependency handling code. AC_DEFUN([AM_OUTPUT_DEPENDENCY_COMMANDS], [AC_CONFIG_COMMANDS([depfiles], [test x"$AMDEP_TRUE" != x"" || _AM_OUTPUT_DEPENDENCY_COMMANDS], [AMDEP_TRUE="$AMDEP_TRUE" MAKE="${MAKE-make}"])]) # Do all the work for Automake. -*- Autoconf -*- # Copyright (C) 1996-2021 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # This macro actually does too much. Some checks are only needed if # your package does certain things. But this isn't really a big deal. dnl Redefine AC_PROG_CC to automatically invoke _AM_PROG_CC_C_O. m4_define([AC_PROG_CC], m4_defn([AC_PROG_CC]) [_AM_PROG_CC_C_O ]) # AM_INIT_AUTOMAKE(PACKAGE, VERSION, [NO-DEFINE]) # AM_INIT_AUTOMAKE([OPTIONS]) # ----------------------------------------------- # The call with PACKAGE and VERSION arguments is the old style # call (pre autoconf-2.50), which is being phased out. PACKAGE # and VERSION should now be passed to AC_INIT and removed from # the call to AM_INIT_AUTOMAKE. # We support both call styles for the transition. After # the next Automake release, Autoconf can make the AC_INIT # arguments mandatory, and then we can depend on a new Autoconf # release and drop the old call support. AC_DEFUN([AM_INIT_AUTOMAKE], [AC_PREREQ([2.65])dnl dnl Autoconf wants to disallow AM_ names. We explicitly allow dnl the ones we care about. m4_pattern_allow([^AM_[A-Z]+FLAGS$])dnl AC_REQUIRE([AM_SET_CURRENT_AUTOMAKE_VERSION])dnl AC_REQUIRE([AC_PROG_INSTALL])dnl if test "`cd $srcdir && pwd`" != "`pwd`"; then # Use -I$(srcdir) only when $(srcdir) != ., so that make's output # is not polluted with repeated "-I." AC_SUBST([am__isrc], [' -I$(srcdir)'])_AM_SUBST_NOTMAKE([am__isrc])dnl # test to see if srcdir already configured if test -f $srcdir/config.status; then AC_MSG_ERROR([source directory already configured; run "make distclean" there first]) fi fi # test whether we have cygpath if test -z "$CYGPATH_W"; then if (cygpath --version) >/dev/null 2>/dev/null; then CYGPATH_W='cygpath -w' else CYGPATH_W=echo fi fi AC_SUBST([CYGPATH_W]) # Define the identity of the package. dnl Distinguish between old-style and new-style calls. m4_ifval([$2], [AC_DIAGNOSE([obsolete], [$0: two- and three-arguments forms are deprecated.]) m4_ifval([$3], [_AM_SET_OPTION([no-define])])dnl AC_SUBST([PACKAGE], [$1])dnl AC_SUBST([VERSION], [$2])], [_AM_SET_OPTIONS([$1])dnl dnl Diagnose old-style AC_INIT with new-style AM_AUTOMAKE_INIT. m4_if( m4_ifset([AC_PACKAGE_NAME], [ok]):m4_ifset([AC_PACKAGE_VERSION], [ok]), [ok:ok],, [m4_fatal([AC_INIT should be called with package and version arguments])])dnl AC_SUBST([PACKAGE], ['AC_PACKAGE_TARNAME'])dnl AC_SUBST([VERSION], ['AC_PACKAGE_VERSION'])])dnl _AM_IF_OPTION([no-define],, [AC_DEFINE_UNQUOTED([PACKAGE], ["$PACKAGE"], [Name of package]) AC_DEFINE_UNQUOTED([VERSION], ["$VERSION"], [Version number of package])])dnl # Some tools Automake needs. AC_REQUIRE([AM_SANITY_CHECK])dnl AC_REQUIRE([AC_ARG_PROGRAM])dnl AM_MISSING_PROG([ACLOCAL], [aclocal-${am__api_version}]) AM_MISSING_PROG([AUTOCONF], [autoconf]) AM_MISSING_PROG([AUTOMAKE], [automake-${am__api_version}]) AM_MISSING_PROG([AUTOHEADER], [autoheader]) AM_MISSING_PROG([MAKEINFO], [makeinfo]) AC_REQUIRE([AM_PROG_INSTALL_SH])dnl AC_REQUIRE([AM_PROG_INSTALL_STRIP])dnl AC_REQUIRE([AC_PROG_MKDIR_P])dnl # For better backward compatibility. To be removed once Automake 1.9.x # dies out for good. For more background, see: # # AC_SUBST([mkdir_p], ['$(MKDIR_P)']) # We need awk for the "check" target (and possibly the TAP driver). The # system "awk" is bad on some platforms. AC_REQUIRE([AC_PROG_AWK])dnl AC_REQUIRE([AC_PROG_MAKE_SET])dnl AC_REQUIRE([AM_SET_LEADING_DOT])dnl _AM_IF_OPTION([tar-ustar], [_AM_PROG_TAR([ustar])], [_AM_IF_OPTION([tar-pax], [_AM_PROG_TAR([pax])], [_AM_PROG_TAR([v7])])]) _AM_IF_OPTION([no-dependencies],, [AC_PROVIDE_IFELSE([AC_PROG_CC], [_AM_DEPENDENCIES([CC])], [m4_define([AC_PROG_CC], m4_defn([AC_PROG_CC])[_AM_DEPENDENCIES([CC])])])dnl AC_PROVIDE_IFELSE([AC_PROG_CXX], [_AM_DEPENDENCIES([CXX])], [m4_define([AC_PROG_CXX], m4_defn([AC_PROG_CXX])[_AM_DEPENDENCIES([CXX])])])dnl AC_PROVIDE_IFELSE([AC_PROG_OBJC], [_AM_DEPENDENCIES([OBJC])], [m4_define([AC_PROG_OBJC], m4_defn([AC_PROG_OBJC])[_AM_DEPENDENCIES([OBJC])])])dnl AC_PROVIDE_IFELSE([AC_PROG_OBJCXX], [_AM_DEPENDENCIES([OBJCXX])], [m4_define([AC_PROG_OBJCXX], m4_defn([AC_PROG_OBJCXX])[_AM_DEPENDENCIES([OBJCXX])])])dnl ]) # Variables for tags utilities; see am/tags.am if test -z "$CTAGS"; then CTAGS=ctags fi AC_SUBST([CTAGS]) if test -z "$ETAGS"; then ETAGS=etags fi AC_SUBST([ETAGS]) if test -z "$CSCOPE"; then CSCOPE=cscope fi AC_SUBST([CSCOPE]) AC_REQUIRE([AM_SILENT_RULES])dnl dnl The testsuite driver may need to know about EXEEXT, so add the dnl 'am__EXEEXT' conditional if _AM_COMPILER_EXEEXT was seen. This dnl macro is hooked onto _AC_COMPILER_EXEEXT early, see below. AC_CONFIG_COMMANDS_PRE(dnl [m4_provide_if([_AM_COMPILER_EXEEXT], [AM_CONDITIONAL([am__EXEEXT], [test -n "$EXEEXT"])])])dnl # POSIX will say in a future version that running "rm -f" with no argument # is OK; and we want to be able to make that assumption in our Makefile # recipes. So use an aggressive probe to check that the usage we want is # actually supported "in the wild" to an acceptable degree. # See automake bug#10828. # To make any issue more visible, cause the running configure to be aborted # by default if the 'rm' program in use doesn't match our expectations; the # user can still override this though. if rm -f && rm -fr && rm -rf; then : OK; else cat >&2 <<'END' Oops! Your 'rm' program seems unable to run without file operands specified on the command line, even when the '-f' option is present. This is contrary to the behaviour of most rm programs out there, and not conforming with the upcoming POSIX standard: Please tell bug-automake@gnu.org about your system, including the value of your $PATH and any error possibly output before this message. This can help us improve future automake versions. END if test x"$ACCEPT_INFERIOR_RM_PROGRAM" = x"yes"; then echo 'Configuration will proceed anyway, since you have set the' >&2 echo 'ACCEPT_INFERIOR_RM_PROGRAM variable to "yes"' >&2 echo >&2 else cat >&2 <<'END' Aborting the configuration process, to ensure you take notice of the issue. You can download and install GNU coreutils to get an 'rm' implementation that behaves properly: . If you want to complete the configuration process using your problematic 'rm' anyway, export the environment variable ACCEPT_INFERIOR_RM_PROGRAM to "yes", and re-run configure. END AC_MSG_ERROR([Your 'rm' program is bad, sorry.]) fi fi dnl The trailing newline in this macro's definition is deliberate, for dnl backward compatibility and to allow trailing 'dnl'-style comments dnl after the AM_INIT_AUTOMAKE invocation. See automake bug#16841. ]) dnl Hook into '_AC_COMPILER_EXEEXT' early to learn its expansion. Do not dnl add the conditional right here, as _AC_COMPILER_EXEEXT may be further dnl mangled by Autoconf and run in a shell conditional statement. m4_define([_AC_COMPILER_EXEEXT], m4_defn([_AC_COMPILER_EXEEXT])[m4_provide([_AM_COMPILER_EXEEXT])]) # When config.status generates a header, we must update the stamp-h file. # This file resides in the same directory as the config header # that is generated. The stamp files are numbered to have different names. # Autoconf calls _AC_AM_CONFIG_HEADER_HOOK (when defined) in the # loop where config.status creates the headers, so we can generate # our stamp files there. AC_DEFUN([_AC_AM_CONFIG_HEADER_HOOK], [# Compute $1's index in $config_headers. _am_arg=$1 _am_stamp_count=1 for _am_header in $config_headers :; do case $_am_header in $_am_arg | $_am_arg:* ) break ;; * ) _am_stamp_count=`expr $_am_stamp_count + 1` ;; esac done echo "timestamp for $_am_arg" >`AS_DIRNAME(["$_am_arg"])`/stamp-h[]$_am_stamp_count]) # Copyright (C) 2001-2021 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # AM_PROG_INSTALL_SH # ------------------ # Define $install_sh. AC_DEFUN([AM_PROG_INSTALL_SH], [AC_REQUIRE([AM_AUX_DIR_EXPAND])dnl if test x"${install_sh+set}" != xset; then case $am_aux_dir in *\ * | *\ *) install_sh="\${SHELL} '$am_aux_dir/install-sh'" ;; *) install_sh="\${SHELL} $am_aux_dir/install-sh" esac fi AC_SUBST([install_sh])]) # Copyright (C) 2003-2021 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # Check whether the underlying file-system supports filenames # with a leading dot. For instance MS-DOS doesn't. AC_DEFUN([AM_SET_LEADING_DOT], [rm -rf .tst 2>/dev/null mkdir .tst 2>/dev/null if test -d .tst; then am__leading_dot=. else am__leading_dot=_ fi rmdir .tst 2>/dev/null AC_SUBST([am__leading_dot])]) # Check to see how 'make' treats includes. -*- Autoconf -*- # Copyright (C) 2001-2021 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # AM_MAKE_INCLUDE() # ----------------- # Check whether make has an 'include' directive that can support all # the idioms we need for our automatic dependency tracking code. AC_DEFUN([AM_MAKE_INCLUDE], [AC_MSG_CHECKING([whether ${MAKE-make} supports the include directive]) cat > confinc.mk << 'END' am__doit: @echo this is the am__doit target >confinc.out .PHONY: am__doit END am__include="#" am__quote= # BSD make does it like this. echo '.include "confinc.mk" # ignored' > confmf.BSD # Other make implementations (GNU, Solaris 10, AIX) do it like this. echo 'include confinc.mk # ignored' > confmf.GNU _am_result=no for s in GNU BSD; do AM_RUN_LOG([${MAKE-make} -f confmf.$s && cat confinc.out]) AS_CASE([$?:`cat confinc.out 2>/dev/null`], ['0:this is the am__doit target'], [AS_CASE([$s], [BSD], [am__include='.include' am__quote='"'], [am__include='include' am__quote=''])]) if test "$am__include" != "#"; then _am_result="yes ($s style)" break fi done rm -f confinc.* confmf.* AC_MSG_RESULT([${_am_result}]) AC_SUBST([am__include])]) AC_SUBST([am__quote])]) # Fake the existence of programs that GNU maintainers use. -*- Autoconf -*- # Copyright (C) 1997-2021 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # AM_MISSING_PROG(NAME, PROGRAM) # ------------------------------ AC_DEFUN([AM_MISSING_PROG], [AC_REQUIRE([AM_MISSING_HAS_RUN]) $1=${$1-"${am_missing_run}$2"} AC_SUBST($1)]) # AM_MISSING_HAS_RUN # ------------------ # Define MISSING if not defined so far and test if it is modern enough. # If it is, set am_missing_run to use it, otherwise, to nothing. AC_DEFUN([AM_MISSING_HAS_RUN], [AC_REQUIRE([AM_AUX_DIR_EXPAND])dnl AC_REQUIRE_AUX_FILE([missing])dnl if test x"${MISSING+set}" != xset; then MISSING="\${SHELL} '$am_aux_dir/missing'" fi # Use eval to expand $SHELL if eval "$MISSING --is-lightweight"; then am_missing_run="$MISSING " else am_missing_run= AC_MSG_WARN(['missing' script is too old or missing]) fi ]) # Helper functions for option handling. -*- Autoconf -*- # Copyright (C) 2001-2021 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # _AM_MANGLE_OPTION(NAME) # ----------------------- AC_DEFUN([_AM_MANGLE_OPTION], [[_AM_OPTION_]m4_bpatsubst($1, [[^a-zA-Z0-9_]], [_])]) # _AM_SET_OPTION(NAME) # -------------------- # Set option NAME. Presently that only means defining a flag for this option. AC_DEFUN([_AM_SET_OPTION], [m4_define(_AM_MANGLE_OPTION([$1]), [1])]) # _AM_SET_OPTIONS(OPTIONS) # ------------------------ # OPTIONS is a space-separated list of Automake options. AC_DEFUN([_AM_SET_OPTIONS], [m4_foreach_w([_AM_Option], [$1], [_AM_SET_OPTION(_AM_Option)])]) # _AM_IF_OPTION(OPTION, IF-SET, [IF-NOT-SET]) # ------------------------------------------- # Execute IF-SET if OPTION is set, IF-NOT-SET otherwise. AC_DEFUN([_AM_IF_OPTION], [m4_ifset(_AM_MANGLE_OPTION([$1]), [$2], [$3])]) # Copyright (C) 1999-2021 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # _AM_PROG_CC_C_O # --------------- # Like AC_PROG_CC_C_O, but changed for automake. We rewrite AC_PROG_CC # to automatically call this. AC_DEFUN([_AM_PROG_CC_C_O], [AC_REQUIRE([AM_AUX_DIR_EXPAND])dnl AC_REQUIRE_AUX_FILE([compile])dnl AC_LANG_PUSH([C])dnl AC_CACHE_CHECK( [whether $CC understands -c and -o together], [am_cv_prog_cc_c_o], [AC_LANG_CONFTEST([AC_LANG_PROGRAM([])]) # Make sure it works both with $CC and with simple cc. # Following AC_PROG_CC_C_O, we do the test twice because some # compilers refuse to overwrite an existing .o file with -o, # though they will create one. am_cv_prog_cc_c_o=yes for am_i in 1 2; do if AM_RUN_LOG([$CC -c conftest.$ac_ext -o conftest2.$ac_objext]) \ && test -f conftest2.$ac_objext; then : OK else am_cv_prog_cc_c_o=no break fi done rm -f core conftest* unset am_i]) if test "$am_cv_prog_cc_c_o" != yes; then # Losing compiler, so override with the script. # FIXME: It is wrong to rewrite CC. # But if we don't then we get into trouble of one sort or another. # A longer-term fix would be to have automake use am__CC in this case, # and then we could set am__CC="\$(top_srcdir)/compile \$(CC)" CC="$am_aux_dir/compile $CC" fi AC_LANG_POP([C])]) # For backward compatibility. AC_DEFUN_ONCE([AM_PROG_CC_C_O], [AC_REQUIRE([AC_PROG_CC])]) # Copyright (C) 1999-2021 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # AM_PATH_PYTHON([MINIMUM-VERSION], [ACTION-IF-FOUND], [ACTION-IF-NOT-FOUND]) # --------------------------------------------------------------------------- # Adds support for distributing Python modules and packages. To # install modules, copy them to $(pythondir), using the python_PYTHON # automake variable. To install a package with the same name as the # automake package, install to $(pkgpythondir), or use the # pkgpython_PYTHON automake variable. # # The variables $(pyexecdir) and $(pkgpyexecdir) are provided as # locations to install python extension modules (shared libraries). # Another macro is required to find the appropriate flags to compile # extension modules. # # If your package is configured with a different prefix to python, # users will have to add the install directory to the PYTHONPATH # environment variable, or create a .pth file (see the python # documentation for details). # # If the MINIMUM-VERSION argument is passed, AM_PATH_PYTHON will # cause an error if the version of python installed on the system # doesn't meet the requirement. MINIMUM-VERSION should consist of # numbers and dots only. AC_DEFUN([AM_PATH_PYTHON], [ dnl Find a Python interpreter. Python versions prior to 2.0 are not dnl supported. (2.0 was released on October 16, 2000). m4_define_default([_AM_PYTHON_INTERPRETER_LIST], [python python2 python3 dnl python3.9 python3.8 python3.7 python3.6 python3.5 python3.4 python3.3 dnl python3.2 python3.1 python3.0 dnl python2.7 python2.6 python2.5 python2.4 python2.3 python2.2 python2.1 dnl python2.0]) AC_ARG_VAR([PYTHON], [the Python interpreter]) m4_if([$1],[],[ dnl No version check is needed. # Find any Python interpreter. if test -z "$PYTHON"; then AC_PATH_PROGS([PYTHON], _AM_PYTHON_INTERPRETER_LIST, :) fi am_display_PYTHON=python ], [ dnl A version check is needed. if test -n "$PYTHON"; then # If the user set $PYTHON, use it and don't search something else. AC_MSG_CHECKING([whether $PYTHON version is >= $1]) AM_PYTHON_CHECK_VERSION([$PYTHON], [$1], [AC_MSG_RESULT([yes])], [AC_MSG_RESULT([no]) AC_MSG_ERROR([Python interpreter is too old])]) am_display_PYTHON=$PYTHON else # Otherwise, try each interpreter until we find one that satisfies # VERSION. AC_CACHE_CHECK([for a Python interpreter with version >= $1], [am_cv_pathless_PYTHON],[ for am_cv_pathless_PYTHON in _AM_PYTHON_INTERPRETER_LIST none; do test "$am_cv_pathless_PYTHON" = none && break AM_PYTHON_CHECK_VERSION([$am_cv_pathless_PYTHON], [$1], [break]) done]) # Set $PYTHON to the absolute path of $am_cv_pathless_PYTHON. if test "$am_cv_pathless_PYTHON" = none; then PYTHON=: else AC_PATH_PROG([PYTHON], [$am_cv_pathless_PYTHON]) fi am_display_PYTHON=$am_cv_pathless_PYTHON fi ]) if test "$PYTHON" = :; then dnl Run any user-specified action, or abort. m4_default([$3], [AC_MSG_ERROR([no suitable Python interpreter found])]) else dnl Query Python for its version number. Although site.py simply uses dnl sys.version[:3], printing that failed with Python 3.10, since the dnl trailing zero was eliminated. So now we output just the major dnl and minor version numbers, as numbers. Apparently the tertiary dnl version is not of interest. AC_CACHE_CHECK([for $am_display_PYTHON version], [am_cv_python_version], [am_cv_python_version=`$PYTHON -c "import sys; print ('%u.%u' % sys.version_info[[:2]])"`]) AC_SUBST([PYTHON_VERSION], [$am_cv_python_version]) dnl Use the values of sys.prefix and sys.exec_prefix for the corresponding dnl values of PYTHON_PREFIX and PYTHON_EXEC_PREFIX. These are made dnl distinct variables so they can be overridden if need be. However, dnl general consensus is that you shouldn't need this ability. dnl Also allow directly setting the prefixes via configure args. if test "x$prefix" = xNONE then am__usable_prefix=$ac_default_prefix else am__usable_prefix=$prefix fi AC_ARG_WITH([python_prefix], [AS_HELP_STRING([--with-python_prefix], [override the default PYTHON_PREFIX])], [ am_python_prefix_subst="$withval" am_cv_python_prefix="$withval" AC_MSG_CHECKING([for $am_display_PYTHON prefix]) AC_MSG_RESULT([$am_cv_python_prefix])], [ AC_CACHE_CHECK([for $am_display_PYTHON prefix], [am_cv_python_prefix], [am_cv_python_prefix=`$PYTHON -c "import sys; sys.stdout.write(sys.prefix)"`]) dnl If sys.prefix is a subdir of $prefix, replace the literal value of $prefix dnl with a variable reference so it can be overridden. case $am_cv_python_prefix in $am__usable_prefix*) am__strip_prefix=`echo "$am__usable_prefix" | sed 's|.|.|g'` am_python_prefix_subst=`echo "$am_cv_python_prefix" | sed "s,^$am__strip_prefix,\\${prefix},"` ;; *) am_python_prefix_subst=$am_cv_python_prefix ;; esac ]) AC_SUBST([PYTHON_PREFIX], [$am_python_prefix_subst]) AC_ARG_WITH([python_exec_prefix], [AS_HELP_STRING([--with-python_exec_prefix], [override the default PYTHON_EXEC_PREFIX])], [ am_python_exec_prefix_subst="$withval" am_cv_python_exec_prefix="$withval" AC_MSG_CHECKING([for $am_display_PYTHON exec_prefix]) AC_MSG_RESULT([$am_cv_python_exec_prefix])], [ dnl --with-python_prefix was given - use its value for python_exec_prefix too AS_IF([test -n "$with_python_prefix"], [am_python_exec_prefix_subst="$with_python_prefix" am_cv_python_exec_prefix="$with_python_prefix" AC_MSG_CHECKING([for $am_display_PYTHON exec_prefix]) AC_MSG_RESULT([$am_cv_python_exec_prefix])], [ AC_CACHE_CHECK([for $am_display_PYTHON exec_prefix], [am_cv_python_exec_prefix], [am_cv_python_exec_prefix=`$PYTHON -c "import sys; sys.stdout.write(sys.exec_prefix)"`]) dnl If sys.exec_prefix is a subdir of $exec_prefix, replace the dnl literal value of $exec_prefix with a variable reference so it can dnl be overridden. if test "x$exec_prefix" = xNONE then am__usable_exec_prefix=$am__usable_prefix else am__usable_exec_prefix=$exec_prefix fi case $am_cv_python_exec_prefix in $am__usable_exec_prefix*) am__strip_prefix=`echo "$am__usable_exec_prefix" | sed 's|.|.|g'` am_python_exec_prefix_subst=`echo "$am_cv_python_exec_prefix" | sed "s,^$am__strip_prefix,\\${exec_prefix},"` ;; *) am_python_exec_prefix_subst=$am_cv_python_exec_prefix ;; esac ])]) AC_SUBST([PYTHON_EXEC_PREFIX], [$am_python_exec_prefix_subst]) dnl At times (like when building shared libraries) you may want dnl to know which OS platform Python thinks this is. AC_CACHE_CHECK([for $am_display_PYTHON platform], [am_cv_python_platform], [am_cv_python_platform=`$PYTHON -c "import sys; sys.stdout.write(sys.platform)"`]) AC_SUBST([PYTHON_PLATFORM], [$am_cv_python_platform]) # Just factor out some code duplication. am_python_setup_sysconfig="\ import sys # Prefer sysconfig over distutils.sysconfig, for better compatibility # with python 3.x. See automake bug#10227. try: import sysconfig except ImportError: can_use_sysconfig = 0 else: can_use_sysconfig = 1 # Can't use sysconfig in CPython 2.7, since it's broken in virtualenvs: # try: from platform import python_implementation if python_implementation() == 'CPython' and sys.version[[:3]] == '2.7': can_use_sysconfig = 0 except ImportError: pass" dnl Set up 4 directories: dnl pythondir -- where to install python scripts. This is the dnl site-packages directory, not the python standard library dnl directory like in previous automake betas. This behavior dnl is more consistent with lispdir.m4 for example. dnl Query distutils for this directory. AC_CACHE_CHECK([for $am_display_PYTHON script directory], [am_cv_python_pythondir], [if test "x$am_cv_python_prefix" = x then am_py_prefix=$am__usable_prefix else am_py_prefix=$am_cv_python_prefix fi am_cv_python_pythondir=`$PYTHON -c " $am_python_setup_sysconfig if can_use_sysconfig: sitedir = sysconfig.get_path('purelib', vars={'base':'$am_py_prefix'}) else: from distutils import sysconfig sitedir = sysconfig.get_python_lib(0, 0, prefix='$am_py_prefix') sys.stdout.write(sitedir)"` case $am_cv_python_pythondir in $am_py_prefix*) am__strip_prefix=`echo "$am_py_prefix" | sed 's|.|.|g'` am_cv_python_pythondir=`echo "$am_cv_python_pythondir" | sed "s,^$am__strip_prefix,\\${PYTHON_PREFIX},"` ;; *) case $am_py_prefix in /usr|/System*) ;; *) am_cv_python_pythondir="\${PYTHON_PREFIX}/lib/python$PYTHON_VERSION/site-packages" ;; esac ;; esac ]) AC_SUBST([pythondir], [$am_cv_python_pythondir]) dnl pkgpythondir -- $PACKAGE directory under pythondir. Was dnl PYTHON_SITE_PACKAGE in previous betas, but this naming is dnl more consistent with the rest of automake. AC_SUBST([pkgpythondir], [\${pythondir}/$PACKAGE]) dnl pyexecdir -- directory for installing python extension modules dnl (shared libraries) dnl Query distutils for this directory. AC_CACHE_CHECK([for $am_display_PYTHON extension module directory], [am_cv_python_pyexecdir], [if test "x$am_cv_python_exec_prefix" = x then am_py_exec_prefix=$am__usable_exec_prefix else am_py_exec_prefix=$am_cv_python_exec_prefix fi am_cv_python_pyexecdir=`$PYTHON -c " $am_python_setup_sysconfig if can_use_sysconfig: sitedir = sysconfig.get_path('platlib', vars={'platbase':'$am_py_exec_prefix'}) else: from distutils import sysconfig sitedir = sysconfig.get_python_lib(1, 0, prefix='$am_py_exec_prefix') sys.stdout.write(sitedir)"` case $am_cv_python_pyexecdir in $am_py_exec_prefix*) am__strip_prefix=`echo "$am_py_exec_prefix" | sed 's|.|.|g'` am_cv_python_pyexecdir=`echo "$am_cv_python_pyexecdir" | sed "s,^$am__strip_prefix,\\${PYTHON_EXEC_PREFIX},"` ;; *) case $am_py_exec_prefix in /usr|/System*) ;; *) am_cv_python_pyexecdir="\${PYTHON_EXEC_PREFIX}/lib/python$PYTHON_VERSION/site-packages" ;; esac ;; esac ]) AC_SUBST([pyexecdir], [$am_cv_python_pyexecdir]) dnl pkgpyexecdir -- $(pyexecdir)/$(PACKAGE) AC_SUBST([pkgpyexecdir], [\${pyexecdir}/$PACKAGE]) dnl Run any user-specified action. $2 fi ]) # AM_PYTHON_CHECK_VERSION(PROG, VERSION, [ACTION-IF-TRUE], [ACTION-IF-FALSE]) # --------------------------------------------------------------------------- # Run ACTION-IF-TRUE if the Python interpreter PROG has version >= VERSION. # Run ACTION-IF-FALSE otherwise. # This test uses sys.hexversion instead of the string equivalent (first # word of sys.version), in order to cope with versions such as 2.2c1. # This supports Python 2.0 or higher. (2.0 was released on October 16, 2000). AC_DEFUN([AM_PYTHON_CHECK_VERSION], [prog="import sys # split strings by '.' and convert to numeric. Append some zeros # because we need at least 4 digits for the hex conversion. # map returns an iterator in Python 3.0 and a list in 2.x minver = list(map(int, '$2'.split('.'))) + [[0, 0, 0]] minverhex = 0 # xrange is not present in Python 3.0 and range returns an iterator for i in list(range(0, 4)): minverhex = (minverhex << 8) + minver[[i]] sys.exit(sys.hexversion < minverhex)" AS_IF([AM_RUN_LOG([$1 -c "$prog"])], [$3], [$4])]) # Copyright (C) 2001-2021 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # AM_RUN_LOG(COMMAND) # ------------------- # Run COMMAND, save the exit status in ac_status, and log it. # (This has been adapted from Autoconf's _AC_RUN_LOG macro.) AC_DEFUN([AM_RUN_LOG], [{ echo "$as_me:$LINENO: $1" >&AS_MESSAGE_LOG_FD ($1) >&AS_MESSAGE_LOG_FD 2>&AS_MESSAGE_LOG_FD ac_status=$? echo "$as_me:$LINENO: \$? = $ac_status" >&AS_MESSAGE_LOG_FD (exit $ac_status); }]) # Check to make sure that the build environment is sane. -*- Autoconf -*- # Copyright (C) 1996-2021 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # AM_SANITY_CHECK # --------------- AC_DEFUN([AM_SANITY_CHECK], [AC_MSG_CHECKING([whether build environment is sane]) # Reject unsafe characters in $srcdir or the absolute working directory # name. Accept space and tab only in the latter. am_lf=' ' case `pwd` in *[[\\\"\#\$\&\'\`$am_lf]]*) AC_MSG_ERROR([unsafe absolute working directory name]);; esac case $srcdir in *[[\\\"\#\$\&\'\`$am_lf\ \ ]]*) AC_MSG_ERROR([unsafe srcdir value: '$srcdir']);; esac # Do 'set' in a subshell so we don't clobber the current shell's # arguments. Must try -L first in case configure is actually a # symlink; some systems play weird games with the mod time of symlinks # (eg FreeBSD returns the mod time of the symlink's containing # directory). if ( am_has_slept=no for am_try in 1 2; do echo "timestamp, slept: $am_has_slept" > conftest.file set X `ls -Lt "$srcdir/configure" conftest.file 2> /dev/null` if test "$[*]" = "X"; then # -L didn't work. set X `ls -t "$srcdir/configure" conftest.file` fi if test "$[*]" != "X $srcdir/configure conftest.file" \ && test "$[*]" != "X conftest.file $srcdir/configure"; then # If neither matched, then we have a broken ls. This can happen # if, for instance, CONFIG_SHELL is bash and it inherits a # broken ls alias from the environment. This has actually # happened. Such a system could not be considered "sane". AC_MSG_ERROR([ls -t appears to fail. Make sure there is not a broken alias in your environment]) fi if test "$[2]" = conftest.file || test $am_try -eq 2; then break fi # Just in case. sleep 1 am_has_slept=yes done test "$[2]" = conftest.file ) then # Ok. : else AC_MSG_ERROR([newly created file is older than distributed files! Check your system clock]) fi AC_MSG_RESULT([yes]) # If we didn't sleep, we still need to ensure time stamps of config.status and # generated files are strictly newer. am_sleep_pid= if grep 'slept: no' conftest.file >/dev/null 2>&1; then ( sleep 1 ) & am_sleep_pid=$! fi AC_CONFIG_COMMANDS_PRE( [AC_MSG_CHECKING([that generated files are newer than configure]) if test -n "$am_sleep_pid"; then # Hide warnings about reused PIDs. wait $am_sleep_pid 2>/dev/null fi AC_MSG_RESULT([done])]) rm -f conftest.file ]) # Copyright (C) 2009-2021 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # AM_SILENT_RULES([DEFAULT]) # -------------------------- # Enable less verbose build rules; with the default set to DEFAULT # ("yes" being less verbose, "no" or empty being verbose). AC_DEFUN([AM_SILENT_RULES], [AC_ARG_ENABLE([silent-rules], [dnl AS_HELP_STRING( [--enable-silent-rules], [less verbose build output (undo: "make V=1")]) AS_HELP_STRING( [--disable-silent-rules], [verbose build output (undo: "make V=0")])dnl ]) case $enable_silent_rules in @%:@ ((( yes) AM_DEFAULT_VERBOSITY=0;; no) AM_DEFAULT_VERBOSITY=1;; *) AM_DEFAULT_VERBOSITY=m4_if([$1], [yes], [0], [1]);; esac dnl dnl A few 'make' implementations (e.g., NonStop OS and NextStep) dnl do not support nested variable expansions. dnl See automake bug#9928 and bug#10237. am_make=${MAKE-make} AC_CACHE_CHECK([whether $am_make supports nested variables], [am_cv_make_support_nested_variables], [if AS_ECHO([['TRUE=$(BAR$(V)) BAR0=false BAR1=true V=1 am__doit: @$(TRUE) .PHONY: am__doit']]) | $am_make -f - >/dev/null 2>&1; then am_cv_make_support_nested_variables=yes else am_cv_make_support_nested_variables=no fi]) if test $am_cv_make_support_nested_variables = yes; then dnl Using '$V' instead of '$(V)' breaks IRIX make. AM_V='$(V)' AM_DEFAULT_V='$(AM_DEFAULT_VERBOSITY)' else AM_V=$AM_DEFAULT_VERBOSITY AM_DEFAULT_V=$AM_DEFAULT_VERBOSITY fi AC_SUBST([AM_V])dnl AM_SUBST_NOTMAKE([AM_V])dnl AC_SUBST([AM_DEFAULT_V])dnl AM_SUBST_NOTMAKE([AM_DEFAULT_V])dnl AC_SUBST([AM_DEFAULT_VERBOSITY])dnl AM_BACKSLASH='\' AC_SUBST([AM_BACKSLASH])dnl _AM_SUBST_NOTMAKE([AM_BACKSLASH])dnl ]) # Copyright (C) 2001-2021 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # AM_PROG_INSTALL_STRIP # --------------------- # One issue with vendor 'install' (even GNU) is that you can't # specify the program used to strip binaries. This is especially # annoying in cross-compiling environments, where the build's strip # is unlikely to handle the host's binaries. # Fortunately install-sh will honor a STRIPPROG variable, so we # always use install-sh in "make install-strip", and initialize # STRIPPROG with the value of the STRIP variable (set by the user). AC_DEFUN([AM_PROG_INSTALL_STRIP], [AC_REQUIRE([AM_PROG_INSTALL_SH])dnl # Installed binaries are usually stripped using 'strip' when the user # run "make install-strip". However 'strip' might not be the right # tool to use in cross-compilation environments, therefore Automake # will honor the 'STRIP' environment variable to overrule this program. dnl Don't test for $cross_compiling = yes, because it might be 'maybe'. if test "$cross_compiling" != no; then AC_CHECK_TOOL([STRIP], [strip], :) fi INSTALL_STRIP_PROGRAM="\$(install_sh) -c -s" AC_SUBST([INSTALL_STRIP_PROGRAM])]) # Copyright (C) 2006-2021 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # _AM_SUBST_NOTMAKE(VARIABLE) # --------------------------- # Prevent Automake from outputting VARIABLE = @VARIABLE@ in Makefile.in. # This macro is traced by Automake. AC_DEFUN([_AM_SUBST_NOTMAKE]) # AM_SUBST_NOTMAKE(VARIABLE) # -------------------------- # Public sister of _AM_SUBST_NOTMAKE. AC_DEFUN([AM_SUBST_NOTMAKE], [_AM_SUBST_NOTMAKE($@)]) # Check how to create a tarball. -*- Autoconf -*- # Copyright (C) 2004-2021 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, # with or without modifications, as long as this notice is preserved. # _AM_PROG_TAR(FORMAT) # -------------------- # Check how to create a tarball in format FORMAT. # FORMAT should be one of 'v7', 'ustar', or 'pax'. # # Substitute a variable $(am__tar) that is a command # writing to stdout a FORMAT-tarball containing the directory # $tardir. # tardir=directory && $(am__tar) > result.tar # # Substitute a variable $(am__untar) that extract such # a tarball read from stdin. # $(am__untar) < result.tar # AC_DEFUN([_AM_PROG_TAR], [# Always define AMTAR for backward compatibility. Yes, it's still used # in the wild :-( We should find a proper way to deprecate it ... AC_SUBST([AMTAR], ['$${TAR-tar}']) # We'll loop over all known methods to create a tar archive until one works. _am_tools='gnutar m4_if([$1], [ustar], [plaintar]) pax cpio none' m4_if([$1], [v7], [am__tar='$${TAR-tar} chof - "$$tardir"' am__untar='$${TAR-tar} xf -'], [m4_case([$1], [ustar], [# The POSIX 1988 'ustar' format is defined with fixed-size fields. # There is notably a 21 bits limit for the UID and the GID. In fact, # the 'pax' utility can hang on bigger UID/GID (see automake bug#8343 # and bug#13588). am_max_uid=2097151 # 2^21 - 1 am_max_gid=$am_max_uid # The $UID and $GID variables are not portable, so we need to resort # to the POSIX-mandated id(1) utility. Errors in the 'id' calls # below are definitely unexpected, so allow the users to see them # (that is, avoid stderr redirection). am_uid=`id -u || echo unknown` am_gid=`id -g || echo unknown` AC_MSG_CHECKING([whether UID '$am_uid' is supported by ustar format]) if test $am_uid -le $am_max_uid; then AC_MSG_RESULT([yes]) else AC_MSG_RESULT([no]) _am_tools=none fi AC_MSG_CHECKING([whether GID '$am_gid' is supported by ustar format]) if test $am_gid -le $am_max_gid; then AC_MSG_RESULT([yes]) else AC_MSG_RESULT([no]) _am_tools=none fi], [pax], [], [m4_fatal([Unknown tar format])]) AC_MSG_CHECKING([how to create a $1 tar archive]) # Go ahead even if we have the value already cached. We do so because we # need to set the values for the 'am__tar' and 'am__untar' variables. _am_tools=${am_cv_prog_tar_$1-$_am_tools} for _am_tool in $_am_tools; do case $_am_tool in gnutar) for _am_tar in tar gnutar gtar; do AM_RUN_LOG([$_am_tar --version]) && break done am__tar="$_am_tar --format=m4_if([$1], [pax], [posix], [$1]) -chf - "'"$$tardir"' am__tar_="$_am_tar --format=m4_if([$1], [pax], [posix], [$1]) -chf - "'"$tardir"' am__untar="$_am_tar -xf -" ;; plaintar) # Must skip GNU tar: if it does not support --format= it doesn't create # ustar tarball either. (tar --version) >/dev/null 2>&1 && continue am__tar='tar chf - "$$tardir"' am__tar_='tar chf - "$tardir"' am__untar='tar xf -' ;; pax) am__tar='pax -L -x $1 -w "$$tardir"' am__tar_='pax -L -x $1 -w "$tardir"' am__untar='pax -r' ;; cpio) am__tar='find "$$tardir" -print | cpio -o -H $1 -L' am__tar_='find "$tardir" -print | cpio -o -H $1 -L' am__untar='cpio -i -H $1 -d' ;; none) am__tar=false am__tar_=false am__untar=false ;; esac # If the value was cached, stop now. We just wanted to have am__tar # and am__untar set. test -n "${am_cv_prog_tar_$1}" && break # tar/untar a dummy directory, and stop if the command works. rm -rf conftest.dir mkdir conftest.dir echo GrepMe > conftest.dir/file AM_RUN_LOG([tardir=conftest.dir && eval $am__tar_ >conftest.tar]) rm -rf conftest.dir if test -s conftest.tar; then AM_RUN_LOG([$am__untar /dev/null 2>&1 && break fi done rm -rf conftest.dir AC_CACHE_VAL([am_cv_prog_tar_$1], [am_cv_prog_tar_$1=$_am_tool]) AC_MSG_RESULT([$am_cv_prog_tar_$1])]) AC_SUBST([am__tar]) AC_SUBST([am__untar]) ]) # _AM_PROG_TAR m4_include([m4/00gnulib.m4]) m4_include([m4/__inline.m4]) m4_include([m4/absolute-header.m4]) m4_include([m4/access.m4]) m4_include([m4/af_alg.m4]) m4_include([m4/alloca.m4]) m4_include([m4/arpa_inet_h.m4]) m4_include([m4/asm-underscore.m4]) m4_include([m4/base32.m4]) m4_include([m4/btowc.m4]) m4_include([m4/builtin-expect.m4]) m4_include([m4/byteswap.m4]) m4_include([m4/calloc.m4]) m4_include([m4/canonicalize.m4]) m4_include([m4/chdir-long.m4]) m4_include([m4/clock_time.m4]) m4_include([m4/close.m4]) m4_include([m4/closedir.m4]) m4_include([m4/codeset.m4]) m4_include([m4/ctype_h.m4]) m4_include([m4/d-ino.m4]) m4_include([m4/dirent_h.m4]) m4_include([m4/dirfd.m4]) m4_include([m4/double-slash-root.m4]) m4_include([m4/dup.m4]) m4_include([m4/dup2.m4]) m4_include([m4/eaccess.m4]) m4_include([m4/eealloc.m4]) m4_include([m4/environ.m4]) m4_include([m4/errno_h.m4]) m4_include([m4/error.m4]) m4_include([m4/exponentd.m4]) m4_include([m4/extensions.m4]) m4_include([m4/extern-inline.m4]) m4_include([m4/fatal-signal.m4]) m4_include([m4/fchdir.m4]) m4_include([m4/fcntl-o.m4]) m4_include([m4/fcntl.m4]) m4_include([m4/fcntl_h.m4]) m4_include([m4/fdopendir.m4]) m4_include([m4/fflush.m4]) m4_include([m4/filenamecat.m4]) m4_include([m4/findprog-in.m4]) m4_include([m4/flexmember.m4]) m4_include([m4/float_h.m4]) m4_include([m4/flock.m4]) m4_include([m4/fnmatch.m4]) m4_include([m4/fnmatch_h.m4]) m4_include([m4/fopen.m4]) m4_include([m4/fpurge.m4]) m4_include([m4/freading.m4]) m4_include([m4/free.m4]) m4_include([m4/fseek.m4]) m4_include([m4/fseeko.m4]) m4_include([m4/fstat.m4]) m4_include([m4/fstatat.m4]) m4_include([m4/ftell.m4]) m4_include([m4/ftello.m4]) m4_include([m4/futimens.m4]) m4_include([m4/getaddrinfo.m4]) m4_include([m4/getcwd-abort-bug.m4]) m4_include([m4/getcwd-path-max.m4]) m4_include([m4/getcwd.m4]) m4_include([m4/getdelim.m4]) m4_include([m4/getdtablesize.m4]) m4_include([m4/getgroups.m4]) m4_include([m4/getline.m4]) m4_include([m4/getopt.m4]) m4_include([m4/getpagesize.m4]) m4_include([m4/getpass.m4]) m4_include([m4/getprogname.m4]) m4_include([m4/getrandom.m4]) m4_include([m4/gettext.m4]) m4_include([m4/gettime.m4]) m4_include([m4/gettimeofday.m4]) m4_include([m4/gl-openssl.m4]) m4_include([m4/gnulib-common.m4]) m4_include([m4/gnulib-comp.m4]) m4_include([m4/group-member.m4]) m4_include([m4/host-cpu-c-abi.m4]) m4_include([m4/hostent.m4]) m4_include([m4/iconv.m4]) m4_include([m4/iconv_h.m4]) m4_include([m4/include_next.m4]) m4_include([m4/inet_ntop.m4]) m4_include([m4/inline.m4]) m4_include([m4/intlmacosx.m4]) m4_include([m4/intmax_t.m4]) m4_include([m4/inttypes.m4]) m4_include([m4/inttypes_h.m4]) m4_include([m4/ioctl.m4]) m4_include([m4/isblank.m4]) m4_include([m4/iswblank.m4]) m4_include([m4/iswdigit.m4]) m4_include([m4/iswxdigit.m4]) m4_include([m4/langinfo_h.m4]) m4_include([m4/largefile.m4]) m4_include([m4/lib-ld.m4]) m4_include([m4/lib-link.m4]) m4_include([m4/lib-prefix.m4]) m4_include([m4/libunistring-base.m4]) m4_include([m4/libunistring-optional.m4]) m4_include([m4/libunistring.m4]) m4_include([m4/limits-h.m4]) m4_include([m4/link.m4]) m4_include([m4/localcharset.m4]) m4_include([m4/locale-fr.m4]) m4_include([m4/locale-ja.m4]) m4_include([m4/locale-zh.m4]) m4_include([m4/locale_h.m4]) m4_include([m4/localeconv.m4]) m4_include([m4/lock.m4]) m4_include([m4/lseek.m4]) m4_include([m4/lstat.m4]) m4_include([m4/malloc.m4]) m4_include([m4/malloca.m4]) m4_include([m4/mbchar.m4]) m4_include([m4/mbiter.m4]) m4_include([m4/mbrtowc.m4]) m4_include([m4/mbsinit.m4]) m4_include([m4/mbsrtowcs.m4]) m4_include([m4/mbstate_t.m4]) m4_include([m4/mbtowc.m4]) m4_include([m4/md4.m4]) m4_include([m4/md5.m4]) m4_include([m4/memchr.m4]) m4_include([m4/mempcpy.m4]) m4_include([m4/memrchr.m4]) m4_include([m4/minmax.m4]) m4_include([m4/mkdir.m4]) m4_include([m4/mkostemp.m4]) m4_include([m4/mkstemp.m4]) m4_include([m4/mktime.m4]) m4_include([m4/mmap-anon.m4]) m4_include([m4/mode_t.m4]) m4_include([m4/msvc-inval.m4]) m4_include([m4/msvc-nothrow.m4]) m4_include([m4/multiarch.m4]) m4_include([m4/nanosleep.m4]) m4_include([m4/netdb_h.m4]) m4_include([m4/netinet_in_h.m4]) m4_include([m4/nl_langinfo.m4]) m4_include([m4/nls.m4]) m4_include([m4/nocrash.m4]) m4_include([m4/off_t.m4]) m4_include([m4/open-cloexec.m4]) m4_include([m4/open-slash.m4]) m4_include([m4/open.m4]) m4_include([m4/openat.m4]) m4_include([m4/opendir.m4]) m4_include([m4/pathmax.m4]) m4_include([m4/pipe.m4]) m4_include([m4/pipe2.m4]) m4_include([m4/po.m4]) m4_include([m4/posix_spawn.m4]) m4_include([m4/posix_spawn_faction_addchdir.m4]) m4_include([m4/printf.m4]) m4_include([m4/pthread_rwlock_rdlock.m4]) m4_include([m4/quote.m4]) m4_include([m4/quotearg.m4]) m4_include([m4/raise.m4]) m4_include([m4/rawmemchr.m4]) m4_include([m4/readdir.m4]) m4_include([m4/readlink.m4]) m4_include([m4/realloc.m4]) m4_include([m4/reallocarray.m4]) m4_include([m4/regex.m4]) m4_include([m4/rename.m4]) m4_include([m4/rewinddir.m4]) m4_include([m4/rmdir.m4]) m4_include([m4/save-cwd.m4]) m4_include([m4/sched_h.m4]) m4_include([m4/secure_getenv.m4]) m4_include([m4/select.m4]) m4_include([m4/servent.m4]) m4_include([m4/setlocale_null.m4]) m4_include([m4/sh-filename.m4]) m4_include([m4/sha1.m4]) m4_include([m4/sha256.m4]) m4_include([m4/sha512.m4]) m4_include([m4/sig_atomic_t.m4]) m4_include([m4/sigaction.m4]) m4_include([m4/signal_h.m4]) m4_include([m4/signalblocking.m4]) m4_include([m4/sigpipe.m4]) m4_include([m4/size_max.m4]) m4_include([m4/snprintf.m4]) m4_include([m4/socketlib.m4]) m4_include([m4/sockets.m4]) m4_include([m4/socklen.m4]) m4_include([m4/sockpfaf.m4]) m4_include([m4/spawn-pipe.m4]) m4_include([m4/spawn_h.m4]) m4_include([m4/ssize_t.m4]) m4_include([m4/stat-time.m4]) m4_include([m4/stat.m4]) m4_include([m4/stdalign.m4]) m4_include([m4/stdbool.m4]) m4_include([m4/stddef_h.m4]) m4_include([m4/stdint.m4]) m4_include([m4/stdint_h.m4]) m4_include([m4/stdio_h.m4]) m4_include([m4/stdlib_h.m4]) m4_include([m4/stpcpy.m4]) m4_include([m4/strcase.m4]) m4_include([m4/strchrnul.m4]) m4_include([m4/strdup.m4]) m4_include([m4/strerror.m4]) m4_include([m4/strerror_r.m4]) m4_include([m4/string_h.m4]) m4_include([m4/strings_h.m4]) m4_include([m4/strndup.m4]) m4_include([m4/strnlen.m4]) m4_include([m4/strpbrk.m4]) m4_include([m4/strptime.m4]) m4_include([m4/strtok_r.m4]) m4_include([m4/strtol.m4]) m4_include([m4/strtoll.m4]) m4_include([m4/symlink.m4]) m4_include([m4/sys_file_h.m4]) m4_include([m4/sys_ioctl_h.m4]) m4_include([m4/sys_random_h.m4]) m4_include([m4/sys_select_h.m4]) m4_include([m4/sys_socket_h.m4]) m4_include([m4/sys_stat_h.m4]) m4_include([m4/sys_time_h.m4]) m4_include([m4/sys_types_h.m4]) m4_include([m4/sys_uio_h.m4]) m4_include([m4/sys_wait_h.m4]) m4_include([m4/tempname.m4]) m4_include([m4/threadlib.m4]) m4_include([m4/time_h.m4]) m4_include([m4/time_r.m4]) m4_include([m4/timegm.m4]) m4_include([m4/timespec.m4]) m4_include([m4/tm_gmtoff.m4]) m4_include([m4/tmpdir.m4]) m4_include([m4/ungetc.m4]) m4_include([m4/unistd-safer.m4]) m4_include([m4/unistd_h.m4]) m4_include([m4/unlink.m4]) m4_include([m4/unlocked-io.m4]) m4_include([m4/utime.m4]) m4_include([m4/utime_h.m4]) m4_include([m4/utimens.m4]) m4_include([m4/utimes.m4]) m4_include([m4/vasnprintf.m4]) m4_include([m4/vasprintf.m4]) m4_include([m4/visibility.m4]) m4_include([m4/vsnprintf.m4]) m4_include([m4/wait-process.m4]) m4_include([m4/waitpid.m4]) m4_include([m4/warn-on-use.m4]) m4_include([m4/warnings.m4]) m4_include([m4/wchar_h.m4]) m4_include([m4/wchar_t.m4]) m4_include([m4/wcrtomb.m4]) m4_include([m4/wctype_h.m4]) m4_include([m4/wcwidth.m4]) m4_include([m4/wget.m4]) m4_include([m4/wget_manywarnings.m4]) m4_include([m4/wint_t.m4]) m4_include([m4/wmemchr.m4]) m4_include([m4/wmempcpy.m4]) m4_include([m4/write.m4]) m4_include([m4/xalloc.m4]) m4_include([m4/xsize.m4]) m4_include([m4/xstrndup.m4]) m4_include([m4/year2038.m4]) m4_include([m4/zzgnulib.m4]) wget-1.21.2/src/0000755000000000000000000000000014115733675010315 500000000000000wget-1.21.2/src/spider.h0000644000000000000000000000267314115732710011671 00000000000000/* Declarations for spider.c Copyright (C) 2006-2011, 2015, 2019-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #ifndef SPIDER_H #define SPIDER_H #define visited_url(a,b) void nonexisting_url (const char *); void print_broken_links (void); void spider_cleanup (void); #endif /* SPIDER_H */ wget-1.21.2/src/spider.c0000644000000000000000000000533014115732710011655 00000000000000/* Keep track of visited URLs in spider mode. Copyright (C) 2006-2011, 2015, 2019-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #include "wget.h" #include #include #include #include "spider.h" #include "url.h" #include "utils.h" #include "hash.h" #include "res.h" static struct hash_table *nonexisting_urls_set; /* Cleanup the data structures associated with this file. */ #if defined DEBUG_MALLOC || defined TESTING void spider_cleanup (void) { if (nonexisting_urls_set) string_set_free (nonexisting_urls_set); } #endif /* Remembers broken links. */ void nonexisting_url (const char *url) { /* Ignore robots.txt URLs */ if (is_robots_txt_url (url)) return; if (!nonexisting_urls_set) nonexisting_urls_set = make_string_hash_table (0); string_set_add (nonexisting_urls_set, url); } void print_broken_links (void) { hash_table_iterator iter; int num_elems; if (!nonexisting_urls_set) { logprintf (LOG_NOTQUIET, _("Found no broken links.\n\n")); return; } num_elems = hash_table_count (nonexisting_urls_set); assert (num_elems > 0); logprintf (LOG_NOTQUIET, ngettext("Found %d broken link.\n\n", "Found %d broken links.\n\n", num_elems), num_elems); for (hash_table_iterate (nonexisting_urls_set, &iter); hash_table_iter_next (&iter); ) { /* Struct url_list *list; */ const char *url = (const char *) iter.key; logprintf (LOG_NOTQUIET, _("%s\n"), url); } logputs (LOG_NOTQUIET, "\n"); } /* * vim: et ts=2 sw=2 */ wget-1.21.2/src/iri.c0000644000000000000000000002753014115732710011160 00000000000000/* IRI related functions. Copyright (C) 2008-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #include "wget.h" #include #include #include #include #include #ifdef HAVE_ICONV # include #endif #include #if IDN2_VERSION_NUMBER < 0x00140000 # include # include #endif #include "utils.h" #include "url.h" #include "c-strcase.h" #include "c-strcasestr.h" #include "xstrndup.h" /* Note: locale encoding is kept in options struct (opt.locale) */ /* Given a string containing "charset=XXX", return the encoding if found, or NULL otherwise */ char * parse_charset (const char *str) { const char *end; char *charset; if (!str || !*str) return NULL; str = c_strcasestr (str, "charset="); if (!str) return NULL; str += 8; end = str; /* sXXXav: which chars should be banned ??? */ while (*end && !c_isspace (*end)) end++; /* sXXXav: could strdupdelim return NULL ? */ charset = strdupdelim (str, end); /* Do a minimum check on the charset value */ if (!check_encoding_name (charset)) { xfree (charset); return NULL; } /*logprintf (LOG_VERBOSE, "parse_charset: %s\n", quote (charset));*/ return charset; } /* Find the locale used, or fall back on a default value */ const char * find_locale (void) { const char *encoding = nl_langinfo(CODESET); if (!encoding || !*encoding) return xstrdup("ASCII"); return xstrdup(encoding); } /* Basic check of an encoding name. */ bool check_encoding_name (const char *encoding) { const char *s = encoding; while (*s) { if (!c_isascii (*s) || c_isspace (*s)) { logprintf (LOG_VERBOSE, _("Encoding %s isn't valid\n"), quote (encoding)); return false; } s++; } return true; } #ifdef HAVE_ICONV /* Do the conversion according to the passed conversion descriptor cd. *out will contain the transcoded string on success. *out content is unspecified otherwise. */ static bool do_conversion (const char *tocode, const char *fromcode, char const *in_org, size_t inlen, char **out) { iconv_t cd; /* sXXXav : hummm hard to guess... */ size_t len, done, outlen; int invalid = 0, tooshort = 0; char *s, *in, *in_save; cd = iconv_open (tocode, fromcode); if (cd == (iconv_t)(-1)) { logprintf (LOG_VERBOSE, _("Conversion from %s to %s isn't supported\n"), quote_n (0, fromcode), quote_n (1, tocode)); *out = NULL; return false; } /* iconv() has to work on an unescaped string */ in_save = in = xstrndup (in_org, inlen); url_unescape_except_reserved (in); inlen = strlen(in); len = outlen = inlen * 2; *out = s = xmalloc (outlen + 1); done = 0; for (;;) { if (iconv (cd, (ICONV_CONST char **) &in, &inlen, out, &outlen) != (size_t)(-1) && iconv (cd, NULL, NULL, out, &outlen) != (size_t)(-1)) { *out = s; *(s + len - outlen - done) = '\0'; xfree(in_save); iconv_close(cd); IF_DEBUG { /* not not print out embedded passwords, in_org might be an URL */ if (!strchr(in_org, '@') && !strchr(*out, '@')) debug_logprintf ("converted '%s' (%s) -> '%s' (%s)\n", in_org, fromcode, *out, tocode); else debug_logprintf ("logging suppressed, strings may contain password\n"); } return true; } /* Incomplete or invalid multibyte sequence */ if (errno == EINVAL || errno == EILSEQ) { if (!invalid) logprintf (LOG_VERBOSE, _("Incomplete or invalid multibyte sequence encountered\n")); invalid++; **out = *in; in++; inlen--; (*out)++; outlen--; } else if (errno == E2BIG) /* Output buffer full */ { tooshort++; done = len; len = done + inlen * 2; s = xrealloc (s, len + 1); *out = s + done - outlen; outlen += inlen * 2; } else /* Weird, we got an unspecified error */ { logprintf (LOG_VERBOSE, _("Unhandled errno %d\n"), errno); break; } } xfree(in_save); iconv_close(cd); IF_DEBUG { /* not not print out embedded passwords, in_org might be an URL */ if (!strchr(in_org, '@') && !strchr(*out, '@')) debug_logprintf ("converted '%s' (%s) -> '%s' (%s)\n", in_org, fromcode, *out, tocode); else debug_logprintf ("logging suppressed, strings may contain password\n"); } return false; } #else static bool do_conversion (const char *tocode _GL_UNUSED, const char *fromcode _GL_UNUSED, char const *in_org _GL_UNUSED, size_t inlen _GL_UNUSED, char **out) { *out = NULL; return false; } #endif /* Try converting string str from locale to UTF-8. Return a new string on success, or str on error or if conversion isn't needed. */ const char * locale_to_utf8 (const char *str) { char *new; /* That shouldn't happen, just in case */ if (!opt.locale) { logprintf (LOG_VERBOSE, _("locale_to_utf8: locale is unset\n")); opt.locale = find_locale (); } if (!opt.locale || !c_strcasecmp (opt.locale, "utf-8")) return str; if (do_conversion ("UTF-8", opt.locale, (char *) str, strlen ((char *) str), &new)) return (const char *) new; xfree (new); return str; } /* Try to "ASCII encode" UTF-8 host. Return the new domain on success or NULL on error. */ char * idn_encode (const struct iri *i, const char *host) { int ret; char *ascii_encoded; char *utf8_encoded = NULL; const char *src; #if IDN2_VERSION_NUMBER < 0x00140000 uint8_t *lower; size_t len = 0; #endif /* Encode to UTF-8 if not done */ if (!i->utf8_encode) { if (!remote_to_utf8 (i, host, &utf8_encoded)) return NULL; /* Nothing to encode or an error occurred */ src = utf8_encoded; } else src = host; #if IDN2_VERSION_NUMBER >= 0x00140000 /* IDN2_TRANSITIONAL implies input NFC encoding */ ret = idn2_lookup_u8 ((uint8_t *) src, (uint8_t **) &ascii_encoded, IDN2_NONTRANSITIONAL); if (ret != IDN2_OK) /* fall back to TR46 Transitional mode, max IDNA2003 compatibility */ ret = idn2_lookup_u8 ((uint8_t *) src, (uint8_t **) &ascii_encoded, IDN2_TRANSITIONAL); if (ret != IDN2_OK) logprintf (LOG_VERBOSE, _("idn_encode failed (%d): %s\n"), ret, quote (idn2_strerror (ret))); #else /* we need a conversion to lowercase */ lower = u8_tolower ((uint8_t *) src, u8_strlen ((uint8_t *) src) + 1, 0, UNINORM_NFKC, NULL, &len); if (!lower) { logprintf (LOG_VERBOSE, _("Failed to convert to lower: %d: %s\n"), errno, quote (src)); xfree (utf8_encoded); return NULL; } if ((ret = idn2_lookup_u8 (lower, (uint8_t **) &ascii_encoded, IDN2_NFC_INPUT)) != IDN2_OK) { logprintf (LOG_VERBOSE, _("idn_encode failed (%d): %s\n"), ret, quote (idn2_strerror (ret))); } xfree (lower); #endif xfree (utf8_encoded); if (ret == IDN2_OK && ascii_encoded) { char *tmp = xstrdup (ascii_encoded); idn2_free (ascii_encoded); ascii_encoded = tmp; } return ret == IDN2_OK ? ascii_encoded : NULL; } /* Try to decode an "ASCII encoded" host. Return the new domain in the locale on success or NULL on error. */ char * idn_decode (const char *host) { /* char *new; int ret; ret = idn2_register_u8 (NULL, host, (uint8_t **) &new, 0); if (ret != IDN2_OK) { logprintf (LOG_VERBOSE, _("idn2_register_u8 failed (%d): %s: %s\n"), ret, quote (idn2_strerror (ret)), host); return NULL; } return new; */ /* idn2_register_u8() just works label by label. * That is pretty much overhead for just displaying the original ulabels. * To keep at least the debug output format, return a cloned host. */ return xstrdup(host); } /* Try to transcode string str from remote encoding to UTF-8. On success, *new contains the transcoded string. *new content is unspecified otherwise. */ bool remote_to_utf8 (const struct iri *iri, const char *str, char **new) { bool ret = false; if (!iri->uri_encoding) return false; /* When `i->uri_encoding' == "UTF-8" there is nothing to convert. But we must test for non-ASCII symbols for correct hostname processing in `idn_encode' function. */ if (!c_strcasecmp (iri->uri_encoding, "UTF-8")) { const unsigned char *p; for (p = (unsigned char *) str; *p; p++) if (*p > 127) { *new = strdup (str); return true; } return false; } if (do_conversion ("UTF-8", iri->uri_encoding, str, strlen (str), new)) ret = true; /* Test if something was converted */ if (*new && !strcmp (str, *new)) { xfree (*new); return false; } return ret; } /* Allocate a new iri structure and return a pointer to it. */ struct iri * iri_new (void) { struct iri *i = xmalloc (sizeof *i); i->uri_encoding = opt.encoding_remote ? xstrdup (opt.encoding_remote) : NULL; i->content_encoding = NULL; i->orig_url = NULL; i->utf8_encode = opt.enable_iri; return i; } struct iri *iri_dup (const struct iri *src) { struct iri *i = xmalloc (sizeof *i); i->uri_encoding = src->uri_encoding ? xstrdup (src->uri_encoding) : NULL; i->content_encoding = (src->content_encoding ? xstrdup (src->content_encoding) : NULL); i->orig_url = src->orig_url ? xstrdup (src->orig_url) : NULL; i->utf8_encode = src->utf8_encode; return i; } /* Completely free an iri structure. */ void iri_free (struct iri *i) { if (i) { xfree (i->uri_encoding); xfree (i->content_encoding); xfree (i->orig_url); xfree (i); } } /* Set uri_encoding of struct iri i. If a remote encoding was specified, use it unless force is true. */ void set_uri_encoding (struct iri *i, const char *charset, bool force) { DEBUGP (("URI encoding = %s\n", charset ? quote (charset) : "None")); if (!force && opt.encoding_remote) return; if (i->uri_encoding) { if (charset && !c_strcasecmp (i->uri_encoding, charset)) return; xfree (i->uri_encoding); } i->uri_encoding = charset ? xstrdup (charset) : NULL; } /* Set content_encoding of struct iri i. */ void set_content_encoding (struct iri *i, const char *charset) { DEBUGP (("URI content encoding = %s\n", charset ? quote (charset) : "None")); if (opt.encoding_remote) return; if (i->content_encoding) { if (charset && !c_strcasecmp (i->content_encoding, charset)) return; xfree (i->content_encoding); } i->content_encoding = charset ? xstrdup (charset) : NULL; } wget-1.21.2/src/cookies.c0000644000000000000000000013634714115732710012040 00000000000000/* Support for cookies. Copyright (C) 2001-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ /* Written by Hrvoje Niksic. Parts are loosely inspired by the cookie patch submitted by Tomasz Wegrzanowski. This implements the client-side cookie support, as specified (loosely) by Netscape's "preliminary specification", currently available at: http://wp.netscape.com/newsref/std/cookie_spec.html rfc2109 is not supported because of its incompatibilities with the above widely-used specification. rfc2965 is entirely ignored, since popular client software doesn't implement it, and even the sites that do send Set-Cookie2 also emit Set-Cookie for compatibility. */ #include "wget.h" #include #include #include #include #include #include #include #ifdef HAVE_LIBPSL # include #endif #include "utils.h" #include "hash.h" #include "cookies.h" #include "http.h" /* for http_atotm */ #include "c-strcase.h" /* Declarations of `struct cookie' and the most basic functions. */ /* Cookie jar serves as cookie storage and a means of retrieving cookies efficiently. All cookies with the same domain are stored in a linked list called "chain". A cookie chain can be reached by looking up the domain in the cookie jar's chains_by_domain table. For example, to reach all the cookies under google.com, one must execute hash_table_get(jar->chains_by_domain, "google.com"). Of course, when sending a cookie to `www.google.com', one must search for cookies that belong to either `www.google.com' or `google.com' -- but the point is that the code doesn't need to go through *all* the cookies. */ struct cookie_jar { /* Cookie chains indexed by domain. */ struct hash_table *chains; int cookie_count; /* number of cookies in the jar. */ }; /* Value set by entry point functions, so that the low-level routines don't need to call time() all the time. */ static time_t cookies_now; struct cookie_jar * cookie_jar_new (void) { struct cookie_jar *jar = xnew (struct cookie_jar); jar->chains = make_nocase_string_hash_table (0); jar->cookie_count = 0; return jar; } struct cookie { char *domain; /* domain of the cookie */ int port; /* port number */ char *path; /* path prefix of the cookie */ unsigned discard_requested :1;/* whether cookie was created to request discarding another cookie. */ unsigned secure :1; /* whether cookie should be transmitted over non-https connections. */ unsigned domain_exact :1; /* whether DOMAIN must match as a whole. */ unsigned permanent :1; /* whether the cookie should outlive the session. */ time_t expiry_time; /* time when the cookie expires, 0 means undetermined. */ char *attr; /* cookie attribute name */ char *value; /* cookie attribute value */ struct cookie *next; /* used for chaining of cookies in the same domain. */ }; #define PORT_ANY (-1) /* Allocate and return a new, empty cookie structure. */ static struct cookie * cookie_new (void) { struct cookie *cookie = xnew0 (struct cookie); /* Both cookie->permanent and cookie->expiry_time are now 0. This means that the cookie doesn't expire, but is only valid for this session (i.e. not written out to disk). */ cookie->port = PORT_ANY; return cookie; } /* Non-zero if the cookie has expired. Assumes cookies_now has been set by one of the entry point functions. */ static bool cookie_expired_p (const struct cookie *c) { return c->expiry_time != 0 && c->expiry_time < cookies_now; } /* Deallocate COOKIE and its components. */ static void delete_cookie (struct cookie *cookie) { xfree (cookie->domain); xfree (cookie->path); xfree (cookie->attr); xfree (cookie->value); xfree (cookie); } /* Functions for storing cookies. All cookies can be reached beginning with jar->chains. The key in that table is the domain name, and the value is a linked list of all cookies from that domain. Every new cookie is placed on the head of the list. */ /* Find and return a cookie in JAR whose domain, path, and attribute name correspond to COOKIE. If found, PREVPTR will point to the location of the cookie previous in chain, or NULL if the found cookie is the head of a chain. If no matching cookie is found, return NULL. */ static struct cookie * find_matching_cookie (struct cookie_jar *jar, struct cookie *cookie, struct cookie **prevptr) { struct cookie *chain, *prev; chain = hash_table_get (jar->chains, cookie->domain); if (!chain) goto nomatch; prev = NULL; for (; chain; prev = chain, chain = chain->next) if (0 == strcmp (cookie->path, chain->path) && 0 == strcmp (cookie->attr, chain->attr) && cookie->port == chain->port) { *prevptr = prev; return chain; } nomatch: *prevptr = NULL; return NULL; } /* Store COOKIE to the jar. This is done by placing COOKIE at the head of its chain. However, if COOKIE matches a cookie already in memory, as determined by find_matching_cookie, the old cookie is unlinked and destroyed. The key of each chain's hash table entry is allocated only the first time; next hash_table_put's reuse the same key. */ static void store_cookie (struct cookie_jar *jar, struct cookie *cookie) { struct cookie *chain_head; char *chain_key; if (hash_table_get_pair (jar->chains, cookie->domain, &chain_key, &chain_head)) { /* A chain of cookies in this domain already exists. Check for duplicates -- if an extant cookie exactly matches our domain, port, path, and name, replace it. */ struct cookie *prev; struct cookie *victim = find_matching_cookie (jar, cookie, &prev); if (victim) { /* Remove VICTIM from the chain. COOKIE will be placed at the head. */ if (prev) { prev->next = victim->next; cookie->next = chain_head; } else { /* prev is NULL; apparently VICTIM was at the head of the chain. This place will be taken by COOKIE, so all we need to do is: */ cookie->next = victim->next; } delete_cookie (victim); --jar->cookie_count; DEBUGP (("Deleted old cookie (to be replaced.)\n")); } else cookie->next = chain_head; } else { /* We are now creating the chain. Use a copy of cookie->domain as the key for the life-time of the chain. Using cookie->domain would be unsafe because the life-time of the chain may exceed the life-time of the cookie. (Cookies may be deleted from the chain by this very function.) */ cookie->next = NULL; chain_key = xstrdup (cookie->domain); } hash_table_put (jar->chains, chain_key, cookie); ++jar->cookie_count; IF_DEBUG { time_t exptime = cookie->expiry_time; DEBUGP (("\nStored cookie %s %d%s %s <%s> <%s> [expiry %s] %s %s\n", cookie->domain, cookie->port, cookie->port == PORT_ANY ? " (ANY)" : "", cookie->path, cookie->permanent ? "permanent" : "session", cookie->secure ? "secure" : "insecure", cookie->expiry_time ? datetime_str (exptime) : "none", cookie->attr, cookie->value)); } } /* Discard a cookie matching COOKIE's domain, port, path, and attribute name. This gets called when we encounter a cookie whose expiry date is in the past, or whose max-age is set to 0. The former corresponds to netscape cookie spec, while the latter is specified by rfc2109. */ static void discard_matching_cookie (struct cookie_jar *jar, struct cookie *cookie) { struct cookie *prev, *victim; if (!hash_table_count (jar->chains)) /* No elements == nothing to discard. */ return; victim = find_matching_cookie (jar, cookie, &prev); if (victim) { if (prev) /* Simply unchain the victim. */ prev->next = victim->next; else { /* VICTIM was head of its chain. We need to place a new cookie at the head. */ char *chain_key = NULL; int res; res = hash_table_get_pair (jar->chains, victim->domain, &chain_key, NULL); if (res == 0) { logprintf (LOG_VERBOSE, _("Unable to get cookie for %s\n"), victim->domain); } if (!victim->next) { /* VICTIM was the only cookie in the chain. Destroy the chain and deallocate the chain key. */ hash_table_remove (jar->chains, victim->domain); xfree (chain_key); } else hash_table_put (jar->chains, chain_key, victim->next); } delete_cookie (victim); DEBUGP (("Discarded old cookie.\n")); } } /* Functions for parsing the `Set-Cookie' header, and creating new cookies from the wire. */ #define TOKEN_IS(token, string_literal) \ BOUNDED_EQUAL_NO_CASE (token.b, token.e, string_literal) #define TOKEN_NON_EMPTY(token) (token.b != NULL && token.b != token.e) /* Parse the contents of the `Set-Cookie' header. The header looks like this: name1=value1; name2=value2; ... Trailing semicolon is optional; spaces are allowed between all tokens. Additionally, values may be quoted. A new cookie is returned upon success, NULL otherwise. The first name-value pair will be used to set the cookie's attribute name and value. Subsequent parameters will be checked against field names such as `domain', `path', etc. Recognized fields will be parsed and the corresponding members of COOKIE filled. */ static struct cookie * parse_set_cookie (const char *set_cookie, bool silent) { const char *ptr = set_cookie; struct cookie *cookie = cookie_new (); param_token name, value; if (!extract_param (&ptr, &name, &value, ';', NULL)) goto error; if (!value.b) goto error; /* If the value is quoted, do not modify it. */ if (*(value.b - 1) == '"') value.b--; if (*value.e == '"') value.e++; cookie->attr = strdupdelim (name.b, name.e); cookie->value = strdupdelim (value.b, value.e); while (extract_param (&ptr, &name, &value, ';', NULL)) { if (TOKEN_IS (name, "domain")) { if (!TOKEN_NON_EMPTY (value)) goto error; xfree (cookie->domain); /* Strictly speaking, we should set cookie->domain_exact if the domain doesn't begin with a dot. But many sites set the domain to "foo.com" and expect "subhost.foo.com" to get the cookie, and it apparently works in browsers. */ if (*value.b == '.') ++value.b; cookie->domain = strdupdelim (value.b, value.e); } else if (TOKEN_IS (name, "path")) { if (!TOKEN_NON_EMPTY (value)) goto error; xfree (cookie->path); cookie->path = strdupdelim (value.b, value.e); } else if (TOKEN_IS (name, "expires")) { char value_copy[128]; size_t value_len = value.e - value.b; time_t expires; if (!TOKEN_NON_EMPTY (value) || value_len >= sizeof (value_copy)) goto error; memcpy (value_copy, value.b, value_len); value_copy[value_len] = 0; /* Check if expiration spec is valid. If not, assume default (cookie doesn't expire, but valid only for this session.) */ expires = http_atotm (value_copy); if (expires != (time_t) -1) { cookie->permanent = 1; cookie->expiry_time = expires; /* According to netscape's specification, expiry time in the past means that discarding of a matching cookie is requested. */ if (cookie->expiry_time < cookies_now) cookie->discard_requested = 1; } } else if (TOKEN_IS (name, "max-age")) { double maxage = -1; char value_copy[32]; size_t value_len = value.e - value.b; if (!TOKEN_NON_EMPTY (value) || value_len >= sizeof (value_copy)) goto error; memcpy (value_copy, value.b, value_len); value_copy[value_len] = 0; sscanf (value_copy, "%lf", &maxage); if (maxage == -1) /* something went wrong. */ goto error; cookie->permanent = 1; cookie->expiry_time = cookies_now + (time_t) maxage; /* According to rfc2109, a cookie with max-age of 0 means that discarding of a matching cookie is requested. */ if (maxage == 0) cookie->discard_requested = 1; } else if (TOKEN_IS (name, "secure")) { /* ignore value completely */ cookie->secure = 1; } /* else: Ignore unrecognized attribute. */ } if (*ptr) /* extract_param has encountered a syntax error */ goto error; /* The cookie has been successfully constructed; return it. */ return cookie; error: if (!silent) logprintf (LOG_NOTQUIET, _("Syntax error in Set-Cookie: %s at position %d.\n"), quotearg_style (escape_quoting_style, set_cookie), (int) (ptr - set_cookie)); delete_cookie (cookie); return NULL; } #undef TOKEN_IS #undef TOKEN_NON_EMPTY /* Sanity checks. These are important, otherwise it is possible for mailcious attackers to destroy important cookie information and/or violate your privacy. */ #define REQUIRE_DIGITS(p) do { \ if (!c_isdigit (*p)) \ return false; \ for (++p; c_isdigit (*p); p++) \ ; \ } while (0) #define REQUIRE_DOT(p) do { \ if (*p++ != '.') \ return false; \ } while (0) /* Check whether ADDR matches .... We don't want to call network functions like inet_addr() because all we need is a check, preferably one that is small, fast, and well-defined. */ static bool numeric_address_p (const char *addr) { const char *p = addr; REQUIRE_DIGITS (p); /* A */ REQUIRE_DOT (p); /* . */ REQUIRE_DIGITS (p); /* B */ REQUIRE_DOT (p); /* . */ REQUIRE_DIGITS (p); /* C */ REQUIRE_DOT (p); /* . */ REQUIRE_DIGITS (p); /* D */ if (*p != '\0') return false; return true; } /* Check whether COOKIE_DOMAIN is an appropriate domain for HOST. Originally I tried to make the check compliant with rfc2109, but the sites deviated too often, so I had to fall back to "tail matching", as defined by the original Netscape's cookie spec. Wget now uses libpsl to check domain names against a public suffix list to see if they are valid. However, since we don't provide a psl on our own, if libpsl is compiled without a public suffix list, fall back to using the original "tail matching" heuristic. Also if libpsl is unable to convert the domain to lowercase, which means that it doesn't have any runtime conversion support, we again fall back to "tail matching" since libpsl states the results are unpredictable with upper case strings. */ #ifdef HAVE_LIBPSL static psl_ctx_t *psl; #endif static bool check_domain_match (const char *cookie_domain, const char *host) { #ifdef HAVE_LIBPSL static int init_psl; char *cookie_domain_lower = NULL; char *host_lower = NULL; int is_acceptable; DEBUGP (("cdm: 1\n")); if (!init_psl) { init_psl = 1; #ifdef HAVE_PSL_LATEST if ((psl = psl_latest (NULL))) goto have_psl; DEBUGP (("\nPSL: Failed to load any PSL data. " "Falling back to insecure heuristics.\n")); #else if ((psl = psl_builtin ()) && !psl_builtin_outdated ()) goto have_psl; DEBUGP (("\nPSL: built-in data outdated. " "Trying to load data from %s.\n", quote (psl_builtin_filename ()))); if ((psl = psl_load_file (psl_builtin_filename ()))) goto have_psl; DEBUGP (("\nPSL: %s not found or not readable. " "Falling back to built-in data.\n", quote (psl_builtin_filename ()))); if (!(psl = psl_builtin ())) { DEBUGP (("\nPSL: libpsl not built with a public suffix list. " "Falling back to insecure heuristics.\n")); goto no_psl; } #endif } else if (!psl) goto no_psl; have_psl: if (psl_str_to_utf8lower (cookie_domain, NULL, NULL, &cookie_domain_lower) == PSL_SUCCESS && psl_str_to_utf8lower (host, NULL, NULL, &host_lower) == PSL_SUCCESS) { is_acceptable = psl_is_cookie_domain_acceptable (psl, host_lower, cookie_domain_lower); } else { DEBUGP (("libpsl unable to parse domain name. " "Falling back to simple heuristics.\n")); goto no_psl; } xfree (cookie_domain_lower); xfree (host_lower); return is_acceptable == 1; no_psl: /* Cleanup the PSL pointers first */ xfree (cookie_domain_lower); xfree (host_lower); #endif /* For efficiency make some elementary checks first */ DEBUGP (("cdm: 2\n")); /* For the sake of efficiency, check for exact match first. */ if (0 == strcasecmp (cookie_domain, host)) return true; DEBUGP (("cdm: 3\n")); /* HOST must match the tail of cookie_domain. */ if (!match_tail (host, cookie_domain, true)) return false; /* We know that COOKIE_DOMAIN is a subset of HOST; however, we must make sure that somebody is not trying to set the cookie for a subdomain shared by many entities. For example, "company.co.uk" must not be allowed to set a cookie for ".co.uk". On the other hand, "sso.redhat.de" should be able to set a cookie for ".redhat.de". The only marginally sane way to handle this I can think of is to reject on the basis of the length of the second-level domain name (but when the top-level domain is unknown), with the assumption that those of three or less characters could be reserved. For example: .co.org -> works because the TLD is known .co.uk -> doesn't work because "co" is only two chars long .com.au -> doesn't work because "com" is only 3 chars long .cnn.uk -> doesn't work because "cnn" is also only 3 chars long (ugh) .cnn.de -> doesn't work for the same reason (ugh!!) .abcd.de -> works because "abcd" is 4 chars long .img.cnn.de -> works because it's not trying to set the 2nd level domain .cnn.co.uk -> works for the same reason That should prevent misuse, while allowing reasonable usage. If someone knows of a better way to handle this, please let me know. */ { const char *p = cookie_domain; int dccount = 1; /* number of domain components */ int ldcl = 0; /* last domain component length */ int nldcl = 0; /* next to last domain component length */ int out; if (*p == '.') /* Ignore leading period in this calculation. */ ++p; DEBUGP (("cdm: 4\n")); for (out = 0; !out; p++) switch (*p) { case '\0': out = 1; break; case '.': if (ldcl == 0) /* Empty domain component found -- the domain is invalid. */ return false; if (*(p + 1) == '\0') { /* Tolerate trailing '.' by not treating the domain as one ending with an empty domain component. */ out = 1; break; } nldcl = ldcl; ldcl = 0; ++dccount; break; default: ++ldcl; } DEBUGP (("cdm: 5\n")); if (dccount < 2) return false; DEBUGP (("cdm: 6\n")); if (dccount == 2) { size_t i; int known_toplevel = false; static const char *known_toplevel_domains[] = { ".com", ".edu", ".net", ".org", ".gov", ".mil", ".int" }; for (i = 0; i < countof (known_toplevel_domains); i++) if (match_tail (cookie_domain, known_toplevel_domains[i], true)) { known_toplevel = true; break; } if (!known_toplevel && nldcl <= 3) return false; } } DEBUGP (("cdm: 7\n")); /* Don't allow the host "foobar.com" to set a cookie for domain "bar.com". */ if (*cookie_domain != '.') { int dlen = strlen (cookie_domain); int hlen = strlen (host); /* cookie host: hostname.foobar.com */ /* desired domain: bar.com */ /* '.' must be here in host-> ^ */ if (hlen > dlen && host[hlen - dlen - 1] != '.') return false; } DEBUGP (("cdm: 8\n")); return true; } static int path_matches (const char *, const char *); /* Check whether PATH begins with COOKIE_PATH. */ static bool check_path_match (const char *cookie_path, const char *path) { return path_matches (path, cookie_path) != 0; } /* Process the HTTP `Set-Cookie' header. This results in storing the cookie or discarding a matching one, or ignoring it completely, all depending on the contents. */ void cookie_handle_set_cookie (struct cookie_jar *jar, const char *host, int port, const char *path, const char *set_cookie) { struct cookie *cookie; cookies_now = time (NULL); char buf[1024], *tmp; size_t pathlen = strlen(path); /* Wget's paths don't begin with '/' (blame rfc1808), but cookie usage assumes /-prefixed paths. Until the rest of Wget is fixed, simply prepend slash to PATH. */ if (pathlen < sizeof (buf) - 1) tmp = buf; else tmp = xmalloc (pathlen + 2); *tmp = '/'; memcpy (tmp + 1, path, pathlen + 1); path = tmp; cookie = parse_set_cookie (set_cookie, false); if (!cookie) goto out; /* Sanitize parts of cookie. */ if (!cookie->domain) { cookie->domain = xstrdup (host); cookie->domain_exact = 1; /* Set the port, but only if it's non-default. */ if (port != 80 && port != 443) cookie->port = port; } else { if (!check_domain_match (cookie->domain, host)) { logprintf (LOG_NOTQUIET, _("Cookie coming from %s attempted to set domain to "), quotearg_style (escape_quoting_style, host)); logprintf (LOG_NOTQUIET, _("%s\n"), quotearg_style (escape_quoting_style, cookie->domain)); cookie->discard_requested = true; } } if (!cookie->path) { /* The cookie doesn't set path: set it to the URL path, sans the file part ("/dir/file" truncated to "/dir/"). */ char *trailing_slash = strrchr (path, '/'); if (trailing_slash) cookie->path = strdupdelim (path, trailing_slash + 1); else /* no slash in the string -- can this even happen? */ cookie->path = xstrdup (path); } else { /* The cookie sets its own path; verify that it is legal. */ if (!check_path_match (cookie->path, path)) { DEBUGP (("Attempt to fake the path: %s, %s\n", cookie->path, path)); goto out; } } /* Now store the cookie, or discard an existing cookie, if discarding was requested. */ if (cookie->discard_requested) { discard_matching_cookie (jar, cookie); goto out; } store_cookie (jar, cookie); if (tmp != buf) xfree (tmp); return; out: if (cookie) delete_cookie (cookie); if (tmp != buf) xfree (tmp); } /* Support for sending out cookies in HTTP requests, based on previously stored cookies. Entry point is `build_cookies_request'. */ /* Return a count of how many times CHR occurs in STRING. */ static int count_char (const char *string, char chr) { const char *p; int count = 0; for (p = string; *p; p++) if (*p == chr) ++count; return count; } /* Find the cookie chains whose domains match HOST and store them to DEST. A cookie chain is the head of a list of cookies that belong to a host/domain. Given HOST "img.search.xemacs.org", this function will return the chains for "img.search.xemacs.org", "search.xemacs.org", and "xemacs.org" -- those of them that exist (if any), that is. DEST should be large enough to accept (in the worst case) as many elements as there are domain components of HOST. */ static int find_chains_of_host (struct cookie_jar *jar, const char *host, struct cookie *dest[]) { int dest_count = 0; int passes, passcnt; /* Bail out quickly if there are no cookies in the jar. */ if (!hash_table_count (jar->chains)) return 0; if (numeric_address_p (host)) /* If host is an IP address, only check for the exact match. */ passes = 1; else /* Otherwise, check all the subdomains except the top-level (last) one. As a domain with N components has N-1 dots, the number of passes equals the number of dots. */ passes = count_char (host, '.'); passcnt = 0; /* Find chains that match HOST, starting with exact match and progressing to less specific domains. For instance, given HOST fly.srk.fer.hr, first look for fly.srk.fer.hr's chain, then srk.fer.hr's, then fer.hr's. */ while (1) { struct cookie *chain = hash_table_get (jar->chains, host); if (chain) dest[dest_count++] = chain; if (++passcnt >= passes) break; host = strchr (host, '.') + 1; } return dest_count; } /* If FULL_PATH begins with PREFIX, return the length of PREFIX, zero otherwise. */ static int path_matches (const char *full_path, const char *prefix) { int len = strlen (prefix); if (0 != strncmp (full_path, prefix, len)) /* FULL_PATH doesn't begin with PREFIX. */ return 0; /* Length of PREFIX determines the quality of the match. */ return len + 1; } /* Return true if COOKIE matches the provided parameters of the URL being downloaded: HOST, PORT, PATH, and SECFLAG. If PATH_GOODNESS is non-NULL, store the "path goodness" value there. That value is a measure of how closely COOKIE matches PATH, used for ordering cookies. */ static bool cookie_matches_url (const struct cookie *cookie, const char *host, int port, const char *path, bool secflag, int *path_goodness) { int pg; if (cookie_expired_p (cookie)) /* Ignore stale cookies. Don't bother unchaining the cookie at this point -- Wget is a relatively short-lived application, and stale cookies will not be saved by `save_cookies'. On the other hand, this function should be as efficient as possible. */ return false; if (cookie->secure && !secflag) /* Don't transmit secure cookies over insecure connections. */ return false; if (cookie->port != PORT_ANY && cookie->port != port) return false; /* If exact domain match is required, verify that cookie's domain is equal to HOST. If not, assume success on the grounds of the cookie's chain having been found by find_chains_of_host. */ if (cookie->domain_exact && 0 != strcasecmp (host, cookie->domain)) return false; pg = path_matches (path, cookie->path); if (pg == 0) return false; if (path_goodness) /* If the caller requested path_goodness, we return it. This is an optimization, so that the caller doesn't need to call path_matches() again. */ *path_goodness = pg; return true; } /* A structure that points to a cookie, along with the additional information about the cookie's "goodness". This allows us to sort the cookies when returning them to the server, as required by the spec. */ struct weighed_cookie { struct cookie *cookie; int domain_goodness; int path_goodness; }; /* Comparator used for uniquifying the list. */ static int equality_comparator (const void *p1, const void *p2) { struct weighed_cookie *wc1 = (struct weighed_cookie *)p1; struct weighed_cookie *wc2 = (struct weighed_cookie *)p2; int namecmp = strcmp (wc1->cookie->attr, wc2->cookie->attr); int valuecmp = strcmp (wc1->cookie->value, wc2->cookie->value); /* We only really care whether both name and value are equal. We return them in this order only for consistency... */ return namecmp ? namecmp : valuecmp; } /* Eliminate duplicate cookies. "Duplicate cookies" are any two cookies with the same attr name and value. Whenever a duplicate pair is found, one of the cookies is removed. */ static int eliminate_dups (struct weighed_cookie *outgoing, int count) { struct weighed_cookie *h; /* hare */ struct weighed_cookie *t; /* tortoise */ struct weighed_cookie *end = outgoing + count; /* We deploy a simple uniquify algorithm: first sort the array according to our sort criteria, then copy it to itself, comparing each cookie to its neighbor and ignoring the duplicates. */ qsort (outgoing, count, sizeof (struct weighed_cookie), equality_comparator); /* "Hare" runs through all the entries in the array, followed by "tortoise". If a duplicate is found, the hare skips it. Non-duplicate entries are copied to the tortoise ptr. */ for (h = t = outgoing; h < end; h++) { if (h != end - 1) { struct cookie *c0 = h[0].cookie; struct cookie *c1 = h[1].cookie; if (!strcmp (c0->attr, c1->attr) && !strcmp (c0->value, c1->value)) continue; /* ignore the duplicate */ } /* If the hare has advanced past the tortoise (because of previous dups), make sure the values get copied. Otherwise, no copying is necessary. */ if (h != t) *t++ = *h; else t++; } return t - outgoing; } /* Comparator used for sorting by quality. */ static int goodness_comparator (const void *p1, const void *p2) { struct weighed_cookie *wc1 = (struct weighed_cookie *)p1; struct weighed_cookie *wc2 = (struct weighed_cookie *)p2; /* Subtractions take `wc2' as the first argument becauase we want a sort in *decreasing* order of goodness. */ int dgdiff = wc2->domain_goodness - wc1->domain_goodness; int pgdiff = wc2->path_goodness - wc1->path_goodness; /* Sort by domain goodness; if these are the same, sort by path goodness. (The sorting order isn't really specified; maybe it should be the other way around.) */ return dgdiff ? dgdiff : pgdiff; } /* Generate a `Cookie' header for a request that goes to HOST:PORT and requests PATH from the server. The resulting string is allocated with `malloc', and the caller is responsible for freeing it. If no cookies pertain to this request, i.e. no cookie header should be generated, NULL is returned. */ char * cookie_header (struct cookie_jar *jar, const char *host, int port, const char *path, bool secflag) { struct cookie *chains[32]; int chain_count; struct cookie *cookie; struct weighed_cookie *outgoing; size_t count, i, ocnt; char *result = NULL; int result_size, pos; char pathbuf[1024]; /* First, find the cookie chains whose domains match HOST. */ /* Allocate room for find_chains_of_host to write to. The number of chains can at most equal the number of subdomains, hence 1+. We ignore cookies with more than 32 labels. */ chain_count = 1 + count_char (host, '.'); if (chain_count > (int) countof (chains)) return NULL; chain_count = find_chains_of_host (jar, host, chains); /* No cookies for this host. */ if (chain_count <= 0) return NULL; /* Wget's paths don't begin with '/' (blame rfc1808), but cookie usage assumes /-prefixed paths. Until the rest of Wget is fixed, simply prepend slash to PATH. */ { char *tmp; size_t pathlen = strlen(path); if (pathlen < sizeof (pathbuf) - 1) tmp = pathbuf; else tmp = xmalloc (pathlen + 2); *tmp = '/'; memcpy (tmp + 1, path, pathlen + 1); path = tmp; } cookies_now = time (NULL); /* Now extract from the chains those cookies that match our host (for domain_exact cookies), port (for cookies with port other than PORT_ANY), etc. See matching_cookie for details. */ /* Count the number of matching cookies. */ count = 0; for (i = 0; i < (unsigned) chain_count; i++) for (cookie = chains[i]; cookie; cookie = cookie->next) if (cookie_matches_url (cookie, host, port, path, secflag, NULL)) ++count; if (!count) goto out; /* no cookies matched */ /* Allocate the array. */ if (count > SIZE_MAX / sizeof (struct weighed_cookie)) goto out; /* unable to process so many cookies */ outgoing = xmalloc (count * sizeof (struct weighed_cookie)); /* Fill the array with all the matching cookies from the chains that match HOST. */ ocnt = 0; for (i = 0; i < (unsigned) chain_count; i++) for (cookie = chains[i]; cookie; cookie = cookie->next) { int pg; if (!cookie_matches_url (cookie, host, port, path, secflag, &pg)) continue; outgoing[ocnt].cookie = cookie; outgoing[ocnt].domain_goodness = strlen (cookie->domain); outgoing[ocnt].path_goodness = pg; ++ocnt; } assert (ocnt == count); /* Eliminate duplicate cookies; that is, those whose name and value are the same. */ count = eliminate_dups (outgoing, count); /* Sort the array so that best-matching domains come first, and that, within one domain, best-matching paths come first. */ qsort (outgoing, count, sizeof (struct weighed_cookie), goodness_comparator); /* Count the space the name=value pairs will take. */ result_size = 0; for (i = 0; i < count; i++) { struct cookie *c = outgoing[i].cookie; /* name=value */ result_size += strlen (c->attr) + 1 + strlen (c->value); } /* Allocate output buffer: name=value pairs -- result_size "; " separators -- (count - 1) * 2 \0 terminator -- 1 */ result_size = result_size + (count - 1) * 2 + 1; result = xmalloc (result_size); pos = 0; for (i = 0; i < count; i++) { struct cookie *c = outgoing[i].cookie; int namlen = strlen (c->attr); int vallen = strlen (c->value); memcpy (result + pos, c->attr, namlen); pos += namlen; result[pos++] = '='; memcpy (result + pos, c->value, vallen); pos += vallen; if (i < count - 1) { result[pos++] = ';'; result[pos++] = ' '; } } result[pos++] = '\0'; xfree (outgoing); assert (pos == result_size); out: if (path != pathbuf) xfree (path); return result; } /* Support for loading and saving cookies. The format used for loading and saving should be the format of the `cookies.txt' file used by Netscape and Mozilla, at least the Unix versions. (Apparently IE can export cookies in that format as well.) The format goes like this: DOMAIN DOMAIN-FLAG PATH SECURE-FLAG TIMESTAMP ATTR-NAME ATTR-VALUE DOMAIN -- cookie domain, optionally followed by :PORT DOMAIN-FLAG -- whether all hosts in the domain match PATH -- cookie path SECURE-FLAG -- whether cookie requires secure connection TIMESTAMP -- expiry timestamp, number of seconds since epoch ATTR-NAME -- name of the cookie attribute ATTR-VALUE -- value of the cookie attribute (empty if absent) The fields are separated by TABs. All fields are mandatory, except for ATTR-VALUE. The `-FLAG' fields are boolean, their legal values being "TRUE" and "FALSE'. Empty lines, lines consisting of whitespace only, and comment lines (beginning with # optionally preceded by whitespace) are ignored. Example line from cookies.txt (split in two lines for readability): .google.com TRUE / FALSE 2147368447 \ PREF ID=34bb47565bbcd47b:LD=en:NR=20:TM=985172580:LM=985739012 */ /* If the region [B, E) ends with :, parse the number, return it, and store new boundary (location of the `:') to DOMAIN_E_PTR. If port is not specified, return 0. */ static int domain_port (const char *domain_b, const char *domain_e, const char **domain_e_ptr) { int port = 0; const char *p; const char *colon = memchr (domain_b, ':', domain_e - domain_b); if (!colon) return 0; for (p = colon + 1; p < domain_e && c_isdigit (*p); p++) port = 10 * port + (*p - '0'); if (p < domain_e) /* Garbage following port number. */ return 0; *domain_e_ptr = colon; return port; } #define GET_WORD(p, b, e) do { \ b = p; \ while (*p && *p != '\t') \ ++p; \ e = p; \ if (b == e || !*p) \ goto next; \ ++p; \ } while (0) /* Load cookies from FILE. */ void cookie_jar_load (struct cookie_jar *jar, const char *file) { char *line = NULL; size_t bufsize = 0; FILE *fp = fopen (file, "r"); if (!fp) { logprintf (LOG_NOTQUIET, _("Cannot open cookies file %s: %s\n"), quote (file), strerror (errno)); return; } cookies_now = time (NULL); while (getline (&line, &bufsize, fp) > 0) { struct cookie *cookie; char *p = line; double expiry; int port; char *domain_b = NULL, *domain_e = NULL; char *domflag_b = NULL, *domflag_e = NULL; char *path_b = NULL, *path_e = NULL; char *secure_b = NULL, *secure_e = NULL; char *expires_b = NULL, *expires_e = NULL; char *name_b = NULL, *name_e = NULL; char *value_b = NULL, *value_e = NULL; /* Skip leading white-space. */ while (*p && c_isspace (*p)) ++p; /* Ignore empty lines. */ if (!*p || *p == '#') continue; GET_WORD (p, domain_b, domain_e); GET_WORD (p, domflag_b, domflag_e); GET_WORD (p, path_b, path_e); GET_WORD (p, secure_b, secure_e); GET_WORD (p, expires_b, expires_e); GET_WORD (p, name_b, name_e); /* Don't use GET_WORD for value because it ends with newline, not TAB. */ value_b = p; value_e = p + strlen (p); if (value_e > value_b && value_e[-1] == '\n') --value_e; if (value_e > value_b && value_e[-1] == '\r') --value_e; /* Empty values are legal (I think), so don't bother checking. */ cookie = cookie_new (); cookie->attr = strdupdelim (name_b, name_e); cookie->value = strdupdelim (value_b, value_e); cookie->path = strdupdelim (path_b, path_e); cookie->secure = BOUNDED_EQUAL (secure_b, secure_e, "TRUE"); /* Curl source says, quoting Andre Garcia: "flag: A TRUE/FALSE value indicating if all machines within a given domain can access the variable. This value is set automatically by the browser, depending on the value set for the domain." */ cookie->domain_exact = !BOUNDED_EQUAL (domflag_b, domflag_e, "TRUE"); /* DOMAIN needs special treatment because we might need to extract the port. */ port = domain_port (domain_b, domain_e, (const char **)&domain_e); if (port) cookie->port = port; if (*domain_b == '.') ++domain_b; /* remove leading dot internally */ cookie->domain = strdupdelim (domain_b, domain_e); /* safe default in case EXPIRES field is garbled. */ expiry = (double)cookies_now - 1; /* I don't like changing the line, but it's safe here. (line is malloced.) */ *expires_e = '\0'; sscanf (expires_b, "%lf", &expiry); if (expiry == 0) { /* EXPIRY can be 0 for session cookies saved because the user specified `--keep-session-cookies' in the past. They remain session cookies, and will be saved only if the user has specified `keep-session-cookies' again. */ } else { if (expiry < cookies_now) goto abort_cookie; /* ignore stale cookie. */ cookie->expiry_time = (time_t) expiry; cookie->permanent = 1; } store_cookie (jar, cookie); next: continue; abort_cookie: delete_cookie (cookie); } xfree(line); fclose (fp); } /* Save cookies, in format described above, to FILE. */ void cookie_jar_save (struct cookie_jar *jar, const char *file) { FILE *fp; hash_table_iterator iter; DEBUGP (("Saving cookies to %s.\n", file)); cookies_now = time (NULL); fp = fopen (file, "w"); if (!fp) { logprintf (LOG_NOTQUIET, _("Cannot open cookies file %s: %s\n"), quote (file), strerror (errno)); return; } fputs ("# HTTP Cookie File\n", fp); fprintf (fp, "# Generated by Wget on %s.\n", datetime_str (cookies_now)); fputs ("# Edit at your own risk.\n\n", fp); for (hash_table_iterate (jar->chains, &iter); hash_table_iter_next (&iter); ) { const char *domain = iter.key; struct cookie *cookie = iter.value; for (; cookie; cookie = cookie->next) { if (!cookie->permanent && !opt.keep_session_cookies) continue; if (cookie_expired_p (cookie)) continue; if (!cookie->domain_exact) fputc ('.', fp); fputs (domain, fp); if (cookie->port != PORT_ANY) fprintf (fp, ":%d", cookie->port); fprintf (fp, "\t%s\t%s\t%s\t%.0f\t%s\t%s\n", cookie->domain_exact ? "FALSE" : "TRUE", cookie->path, cookie->secure ? "TRUE" : "FALSE", (double)cookie->expiry_time, cookie->attr, cookie->value); if (ferror (fp)) goto out; } } out: if (ferror (fp)) logprintf (LOG_NOTQUIET, _("Error writing to %s: %s\n"), quote (file), strerror (errno)); if (fclose (fp) < 0) logprintf (LOG_NOTQUIET, _("Error closing %s: %s\n"), quote (file), strerror (errno)); DEBUGP (("Done saving cookies.\n")); } /* Clean up cookie-related data. */ void cookie_jar_delete (struct cookie_jar *jar) { /* Iterate over chains (indexed by domain) and free them. */ hash_table_iterator iter; for (hash_table_iterate (jar->chains, &iter); hash_table_iter_next (&iter); ) { struct cookie *chain = iter.value; xfree (iter.key); /* Then all cookies in this chain. */ while (chain) { struct cookie *next = chain->next; delete_cookie (chain); chain = next; } } hash_table_destroy (jar->chains); xfree (jar); #ifdef HAVE_LIBPSL psl_free (psl); psl = NULL; #endif } /* Test cases. Currently this is only tests parse_set_cookies. To use, recompile Wget with -DTEST_COOKIES and call test_cookies() from main. */ #ifdef TEST_COOKIES void test_cookies (void) { /* Tests expected to succeed: */ static struct { const char *data; const char *results[10]; } tests_succ[] = { { "arg=value", {"arg", "value", NULL} }, { "arg1=value1;arg2=value2", {"arg1", "value1", "arg2", "value2", NULL} }, { "arg1=value1; arg2=value2", {"arg1", "value1", "arg2", "value2", NULL} }, { "arg1=value1; arg2=value2;", {"arg1", "value1", "arg2", "value2", NULL} }, { "arg1=value1; arg2=value2; ", {"arg1", "value1", "arg2", "value2", NULL} }, { "arg1=\"value1\"; arg2=\"\"", {"arg1", "value1", "arg2", "", NULL} }, { "arg=", {"arg", "", NULL} }, { "arg1=; arg2=", {"arg1", "", "arg2", "", NULL} }, { "arg1 = ; arg2= ", {"arg1", "", "arg2", "", NULL} }, }; /* Tests expected to fail: */ static char *tests_fail[] = { ";", "arg=\"unterminated", "=empty-name", "arg1=;=another-empty-name", }; int i; for (i = 0; i < countof (tests_succ); i++) { int ind; const char *data = tests_succ[i].data; const char **expected = tests_succ[i].results; struct cookie *c; c = parse_set_cookie (data, true); if (!c) { printf ("NULL cookie returned for valid data: %s\n", data); continue; } /* Test whether extract_param handles these cases correctly. */ { param_token name, value; const char *ptr = data; int j = 0; while (extract_param (&ptr, &name, &value, ';', NULL)) { char *n = strdupdelim (name.b, name.e); char *v = strdupdelim (value.b, value.e); if (!expected[j]) { printf ("Too many parameters for '%s'\n", data); break; } if (0 != strcmp (expected[j], n)) printf ("Invalid name %d for '%s' (expected '%s', got '%s')\n", j / 2 + 1, data, expected[j], n); if (0 != strcmp (expected[j + 1], v)) printf ("Invalid value %d for '%s' (expected '%s', got '%s')\n", j / 2 + 1, data, expected[j + 1], v); j += 2; xfree (n); xfree (v); } if (expected[j]) printf ("Too few parameters for '%s'\n", data); } } for (i = 0; i < countof (tests_fail); i++) { struct cookie *c; char *data = tests_fail[i]; c = parse_set_cookie (data, true); if (c) printf ("Failed to report error on invalid data: %s\n", data); } } #endif /* TEST_COOKIES */ wget-1.21.2/src/cookies.h0000644000000000000000000000341214115732710012027 00000000000000/* Support for cookies. Copyright (C) 2001-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #ifndef COOKIES_H #define COOKIES_H struct cookie_jar; struct cookie_jar *cookie_jar_new (void); void cookie_jar_delete (struct cookie_jar *); void cookie_handle_set_cookie (struct cookie_jar *, const char *, int, const char *, const char *); char *cookie_header (struct cookie_jar *, const char *, int, const char *, bool); void cookie_jar_load (struct cookie_jar *, const char *); void cookie_jar_save (struct cookie_jar *, const char *); #endif /* COOKIES_H */ wget-1.21.2/src/http-ntlm.c0000644000000000000000000004170514115732710012324 00000000000000/* NTLM code. Copyright (C) 2005-2011, 2015, 2018-2021 Free Software Foundation, Inc. Contributed by Daniel Stenberg. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #include "wget.h" /* NTLM details: http://davenport.sourceforge.net/ntlm.html http://www.innovation.ch/java/ntlm.html */ #include #include #include #include "utils.h" #include "http-ntlm.h" #ifdef HAVE_NETTLE # include # include #else # include # include # include # if OPENSSL_VERSION_NUMBER < 0x00907001L # define DES_key_schedule des_key_schedule # define DES_cblock des_cblock # define DES_set_odd_parity des_set_odd_parity # define DES_set_key des_set_key # define DES_ecb_encrypt des_ecb_encrypt /* This is how things were done in the old days */ # define DESKEY(x) x # define DESKEYARG(x) x # else /* Modern version */ # define DESKEYARG(x) *x # define DESKEY(x) &x # endif #endif /* Define this to make the type-3 message include the NT response message */ #define USE_NTRESPONSES 1 /* Flag bits definitions available at on http://davenport.sourceforge.net/ntlm.html */ #define NTLMFLAG_NEGOTIATE_OEM (1<<1) #define NTLMFLAG_NEGOTIATE_NTLM_KEY (1<<9) /* (*) = A "security buffer" is a triplet consisting of two shorts and one long: 1. a 'short' containing the length of the buffer in bytes 2. a 'short' containing the allocated space for the buffer in bytes 3. a 'long' containing the offset to the start of the buffer from the beginning of the NTLM message, in bytes. */ /* return true on success, false otherwise */ bool ntlm_input (struct ntlmdata *ntlm, const char *header) { if (0 != strncmp (header, "NTLM", 4)) return false; header += 4; while (*header && c_isspace(*header)) header++; if (*header) { /* We got a type-2 message here: Index Description Content 0 NTLMSSP Signature Null-terminated ASCII "NTLMSSP" (0x4e544c4d53535000) 8 NTLM Message Type long (0x02000000) 12 Target Name security buffer(*) 20 Flags long 24 Challenge 8 bytes (32) Context (optional) 8 bytes (two consecutive longs) (40) Target Information (optional) security buffer(*) 32 (48) start of data block */ ssize_t size; char buffer[48]; // decode 48 bytes needs ((48 + 2) / 3) * 4 + 1 bytes DEBUGP (("Received a type-2 NTLM message.\n")); size = wget_base64_decode (header, buffer, sizeof (buffer)); if (size < 0) return false; /* malformed base64 from server */ ntlm->state = NTLMSTATE_TYPE2; /* we got a type-2 */ if ((size_t) size >= sizeof (buffer)) /* the nonce of interest is index [24 .. 31], 8 bytes */ memcpy (ntlm->nonce, &buffer[24], 8); /* at index decimal 20, there's a 32bit NTLM flag field */ } else { if (ntlm->state == NTLMSTATE_LAST) { DEBUGP (("NTLM auth restarted.\n")); /* no return, continue */ } else if (ntlm->state == NTLMSTATE_TYPE3) { DEBUGP (("NTLM handshake rejected.\n")); ntlm->state = NTLMSTATE_NONE; return false; } else if (ntlm->state >= NTLMSTATE_TYPE1) { DEBUGP (("Unexpected empty NTLM message.\n")); return false; /* this is an error */ } DEBUGP (("Empty NTLM message, (re)starting transaction.\n")); ntlm->state = NTLMSTATE_TYPE1; /* we should sent away a type-1 */ } return true; } /* * Turns a 56 bit key into the 64 bit, odd parity key and sets the key. The * key schedule ks is also set. */ #ifdef HAVE_NETTLE static void setup_des_key(unsigned char *key_56, struct des_ctx *des) { unsigned char key[8]; key[0] = key_56[0]; key[1] = ((key_56[0] << 7) & 0xFF) | (key_56[1] >> 1); key[2] = ((key_56[1] << 6) & 0xFF) | (key_56[2] >> 2); key[3] = ((key_56[2] << 5) & 0xFF) | (key_56[3] >> 3); key[4] = ((key_56[3] << 4) & 0xFF) | (key_56[4] >> 4); key[5] = ((key_56[4] << 3) & 0xFF) | (key_56[5] >> 5); key[6] = ((key_56[5] << 2) & 0xFF) | (key_56[6] >> 6); key[7] = (key_56[6] << 1) & 0xFF; nettle_des_set_key(des, key); } #else static void setup_des_key(unsigned char *key_56, DES_key_schedule DESKEYARG(ks)) { DES_cblock key; key[0] = key_56[0]; key[1] = ((key_56[0] << 7) & 0xFF) | (key_56[1] >> 1); key[2] = ((key_56[1] << 6) & 0xFF) | (key_56[2] >> 2); key[3] = ((key_56[2] << 5) & 0xFF) | (key_56[3] >> 3); key[4] = ((key_56[3] << 4) & 0xFF) | (key_56[4] >> 4); key[5] = ((key_56[4] << 3) & 0xFF) | (key_56[5] >> 5); key[6] = ((key_56[5] << 2) & 0xFF) | (key_56[6] >> 6); key[7] = (key_56[6] << 1) & 0xFF; DES_set_odd_parity(&key); DES_set_key(&key, ks); } #endif /* * takes a 21 byte array and treats it as 3 56-bit DES keys. The * 8 byte plaintext is encrypted with each key and the resulting 24 * bytes are stored in the results array. */ static void calc_resp(unsigned char *keys, unsigned char *plaintext, unsigned char *results) { #ifdef HAVE_NETTLE struct des_ctx des; setup_des_key(keys, &des); nettle_des_encrypt(&des, 8, results, plaintext); setup_des_key(keys + 7, &des); nettle_des_encrypt(&des, 8, results + 8, plaintext); setup_des_key(keys + 14, &des); nettle_des_encrypt(&des, 8, results + 16, plaintext); #else DES_key_schedule ks; setup_des_key(keys, DESKEY(ks)); DES_ecb_encrypt((DES_cblock*) plaintext, (DES_cblock*) results, DESKEY(ks), DES_ENCRYPT); setup_des_key(keys+7, DESKEY(ks)); DES_ecb_encrypt((DES_cblock*) plaintext, (DES_cblock*) (results+8), DESKEY(ks), DES_ENCRYPT); setup_des_key(keys+14, DESKEY(ks)); DES_ecb_encrypt((DES_cblock*) plaintext, (DES_cblock*) (results+16), DESKEY(ks), DES_ENCRYPT); #endif } /* * Set up lanmanager and nt hashed passwords */ static void mkhash(const char *password, unsigned char *nonce, /* 8 bytes */ unsigned char *lmresp /* must fit 0x18 bytes */ #ifdef USE_NTRESPONSES , unsigned char *ntresp /* must fit 0x18 bytes */ #endif ) { unsigned char lmbuffer[21]; #ifdef USE_NTRESPONSES unsigned char ntbuffer[21]; #endif unsigned char pw[14]; static const unsigned char magic[] = { 0x4B, 0x47, 0x53, 0x21, 0x40, 0x23, 0x24, 0x25 }; size_t i, len = strlen(password); /* make it fit at least 14 bytes */ if (len > sizeof (pw)) len = sizeof (pw); for (i = 0; i < len; i++) pw[i] = (unsigned char) c_toupper (password[i]); for (; i < sizeof (pw); i++) pw[i] = 0; { /* create LanManager hashed password */ #ifdef HAVE_NETTLE struct des_ctx des; setup_des_key(pw, &des); nettle_des_encrypt(&des, 8, lmbuffer, magic); setup_des_key(pw + 7, &des); nettle_des_encrypt(&des, 8, lmbuffer + 8, magic); #else DES_key_schedule ks; setup_des_key(pw, DESKEY (ks)); DES_ecb_encrypt((DES_cblock *) magic, (DES_cblock *) lmbuffer, DESKEY (ks), DES_ENCRYPT); setup_des_key(pw+7, DESKEY (ks)); DES_ecb_encrypt((DES_cblock *) magic, (DES_cblock *) (lmbuffer + 8), DESKEY (ks), DES_ENCRYPT); #endif memset(lmbuffer + 16, 0, 5); } /* create LM responses */ calc_resp(lmbuffer, nonce, lmresp); #ifdef USE_NTRESPONSES { #ifdef HAVE_NETTLE struct md4_ctx MD4; #else MD4_CTX MD4; #endif unsigned char pw4[64]; len = strlen (password); if (len > sizeof (pw4) / 2) len = sizeof (pw4) / 2; for (i = 0; i < len; i++) { pw4[2 * i] = (unsigned char) password[i]; pw4[2 * i + 1] = 0; } #ifdef HAVE_NETTLE nettle_md4_init(&MD4); nettle_md4_update(&MD4, (unsigned) (2 * len), pw4); nettle_md4_digest(&MD4, MD4_DIGEST_SIZE, ntbuffer); #else /* create NT hashed password */ MD4_Init(&MD4); MD4_Update(&MD4, pw4, 2 * len); MD4_Final(ntbuffer, &MD4); #endif memset(ntbuffer + 16, 0, 5); } calc_resp(ntbuffer, nonce, ntresp); #endif } #define SHORTPAIR(x) (char) ((x) & 0xff), (char) ((x) >> 8) #define LONGQUARTET(x) ((x) & 0xff), (((x) >> 8)&0xff), \ (((x) >>16)&0xff), ((x)>>24) /* this is for creating ntlm header output */ char * ntlm_output (struct ntlmdata *ntlm, const char *user, const char *passwd, bool *ready) { const char *domain = ""; /* empty */ const char *host = ""; /* empty */ size_t domlen = strlen(domain); size_t hostlen = strlen(host); size_t hostoff; /* host name offset */ size_t domoff; /* domain name offset */ size_t size; char ntlmbuf[256]; /* enough, unless the host/domain is very long */ /* point to the address of the pointer that holds the string to sent to the server, which is for a plain host or for a HTTP proxy */ char *output = NULL; *ready = false; /* not set means empty */ if(!user) user=""; if(!passwd) passwd=""; switch(ntlm->state) { case NTLMSTATE_TYPE1: case NTLMSTATE_NONE: case NTLMSTATE_LAST: hostoff = 32; domoff = hostoff + hostlen; DEBUGP (("Creating a type-1 NTLM message.\n")); /* Create and send a type-1 message: Index Description Content 0 NTLMSSP Signature Null-terminated ASCII "NTLMSSP" (0x4e544c4d53535000) 8 NTLM Message Type long (0x01000000) 12 Flags long 16 Supplied Domain security buffer(*) 24 Supplied Workstation security buffer(*) 32 start of data block */ snprintf (ntlmbuf, sizeof(ntlmbuf), "NTLMSSP%c" "\x01%c%c%c" /* 32-bit type = 1 */ "%c%c%c%c" /* 32-bit NTLM flag field */ "%c%c" /* domain length */ "%c%c" /* domain allocated space */ "%c%c" /* domain name offset */ "%c%c" /* 2 zeroes */ "%c%c" /* host length */ "%c%c" /* host allocated space */ "%c%c" /* host name offset */ "%c%c" /* 2 zeroes */ "%s" /* host name */ "%s", /* domain string */ 0, /* trailing zero */ 0,0,0, /* part of type-1 long */ LONGQUARTET( NTLMFLAG_NEGOTIATE_OEM| /* 2 */ NTLMFLAG_NEGOTIATE_NTLM_KEY /* 200 */ /* equals 0x0202 */ ), SHORTPAIR(domlen), SHORTPAIR(domlen), SHORTPAIR(domoff), 0,0, SHORTPAIR(hostlen), SHORTPAIR(hostlen), SHORTPAIR(hostoff), 0,0, host, domain); /* initial packet length */ size = 32 + hostlen + domlen; output = xmalloc(5 + BASE64_LENGTH (size) + 1); memcpy(output, "NTLM ", 5); wget_base64_encode (ntlmbuf, size, output + 5); break; case NTLMSTATE_TYPE2: /* We received the type-2 already, create a type-3 message: Index Description Content 0 NTLMSSP Signature Null-terminated ASCII "NTLMSSP" (0x4e544c4d53535000) 8 NTLM Message Type long (0x03000000) 12 LM/LMv2 Response security buffer(*) 20 NTLM/NTLMv2 Response security buffer(*) 28 Domain Name security buffer(*) 36 User Name security buffer(*) 44 Workstation Name security buffer(*) (52) Session Key (optional) security buffer(*) (60) Flags (optional) long 52 (64) start of data block */ { size_t lmrespoff; size_t ntrespoff; size_t useroff; unsigned char lmresp[0x18]; /* fixed-size */ #ifdef USE_NTRESPONSES unsigned char ntresp[0x18]; /* fixed-size */ #endif const char *usr; size_t userlen; DEBUGP (("Creating a type-3 NTLM message.\n")); usr = strchr(user, '\\'); if(!usr) usr = strchr(user, '/'); if (usr) { domain = user; domlen = (size_t) (usr - domain); usr++; } else usr = user; userlen = strlen(usr); mkhash(passwd, &ntlm->nonce[0], lmresp #ifdef USE_NTRESPONSES , ntresp #endif ); domoff = 64; /* always */ useroff = domoff + domlen; hostoff = useroff + userlen; lmrespoff = hostoff + hostlen; ntrespoff = lmrespoff + 0x18; /* Create the big type-3 message binary blob */ snprintf (ntlmbuf, sizeof (ntlmbuf), "NTLMSSP%c" "\x03%c%c%c" /* type-3, 32 bits */ "%c%c%c%c" /* LanManager length + allocated space */ "%c%c" /* LanManager offset */ "%c%c" /* 2 zeroes */ "%c%c" /* NT-response length */ "%c%c" /* NT-response allocated space */ "%c%c" /* NT-response offset */ "%c%c" /* 2 zeroes */ "%c%c" /* domain length */ "%c%c" /* domain allocated space */ "%c%c" /* domain name offset */ "%c%c" /* 2 zeroes */ "%c%c" /* user length */ "%c%c" /* user allocated space */ "%c%c" /* user offset */ "%c%c" /* 2 zeroes */ "%c%c" /* host length */ "%c%c" /* host allocated space */ "%c%c" /* host offset */ "%c%c%c%c%c%c" /* 6 zeroes */ "\xff\xff" /* message length */ "%c%c" /* 2 zeroes */ "\x01\x82" /* flags */ "%c%c" /* 2 zeroes */ /* domain string */ /* user string */ /* host string */ /* LanManager response */ /* NT response */ , 0, /* zero termination */ 0, 0, 0, /* type-3 long, the 24 upper bits */ SHORTPAIR (0x18), /* LanManager response length, twice */ SHORTPAIR (0x18), SHORTPAIR (lmrespoff), 0x0, 0x0, #ifdef USE_NTRESPONSES SHORTPAIR (0x18), /* NT-response length, twice */ SHORTPAIR (0x18), #else 0x0, 0x0, 0x0, 0x0, #endif SHORTPAIR (ntrespoff), 0x0, 0x0, SHORTPAIR (domlen), SHORTPAIR (domlen), SHORTPAIR (domoff), 0x0, 0x0, SHORTPAIR (userlen), SHORTPAIR (userlen), SHORTPAIR (useroff), 0x0, 0x0, SHORTPAIR (hostlen), SHORTPAIR (hostlen), SHORTPAIR (hostoff), 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0); /* size is now 64 */ size=64; ntlmbuf[62]=ntlmbuf[63]=0; /* Make sure that the user and domain strings fit in the target buffer before we copy them there. */ if((size + userlen + domlen) >= sizeof(ntlmbuf)) return NULL; memcpy(&ntlmbuf[size], domain, domlen); size += domlen; memcpy(&ntlmbuf[size], usr, userlen); size += userlen; /* we append the binary hashes to the end of the blob */ if(size < (sizeof(ntlmbuf) - 0x18)) { memcpy(&ntlmbuf[size], lmresp, 0x18); size += 0x18; } #ifdef USE_NTRESPONSES if(size < (sizeof(ntlmbuf) - 0x18)) { memcpy(&ntlmbuf[size], ntresp, 0x18); size += 0x18; } #endif ntlmbuf[56] = (char) (size & 0xff); ntlmbuf[57] = (char) (size >> 8); /* convert the binary blob into base64 */ output = xmalloc(5 + BASE64_LENGTH (size) + 1); memcpy(output, "NTLM ", 5); wget_base64_encode (ntlmbuf, size, output + 5); ntlm->state = NTLMSTATE_TYPE3; /* we sent a type-3 */ *ready = true; } break; case NTLMSTATE_TYPE3: /* connection is already authenticated, * don't send a header in future requests */ *ready = true; output = NULL; break; } return output; } wget-1.21.2/src/exits.c0000644000000000000000000000607114115732710011526 00000000000000/* Exit status handling. Copyright (C) 2009-2012, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . */ #include "wget.h" #include "exits.h" static int final_exit_status = WGET_EXIT_SUCCESS; /* XXX: I don't like that newly-added uerr_t codes will doubtless fall through the craccks, or the fact that we seem to have way more codes than we know what to do with. Need to go through and sort through the truly essential codes, and merge the rest with those. Quite a few are never even used! Quite a few of the codes below would have no business being returned to retrieve_url's caller, but since it's very difficult to determine which do and which don't, I grab virtually all of them to be safe. */ static int get_status_for_err (uerr_t err) { switch (err) { case RETROK: return WGET_EXIT_SUCCESS; case FOPENERR: case FOPEN_EXCL_ERR: case FWRITEERR: case WRITEFAILED: case UNLINKERR: case CLOSEFAILED: case FILEBADFILE: return WGET_EXIT_IO_FAIL; case NOCONERROR: case HOSTERR: case CONSOCKERR: case CONERROR: case CONSSLERR: case CONIMPOSSIBLE: case FTPRERR: case FTPINVPASV: case READERR: case TRYLIMEXC: return WGET_EXIT_NETWORK_FAIL; case VERIFCERTERR: return WGET_EXIT_SSL_AUTH_FAIL; case FTPLOGINC: case FTPLOGREFUSED: case AUTHFAILED: return WGET_EXIT_SERVER_AUTH_FAIL; case HEOF: case HERR: case ATTRMISSING: return WGET_EXIT_PROTOCOL_ERROR; case WRONGCODE: case FTPPORTERR: case FTPSYSERR: case FTPNSFOD: case FTPUNKNOWNTYPE: case FTPSRVERR: case FTPRETRINT: case FTPRESTFAIL: case FTPNOPASV: case CONTNOTSUPPORTED: case RANGEERR: case RETRBADPATTERN: case PROXERR: case GATEWAYTIMEOUT: return WGET_EXIT_SERVER_ERROR; case URLERROR: case QUOTEXC: case SSLINITFAILED: case UNKNOWNATTR: default: return WGET_EXIT_UNKNOWN; } } /* inform_exit_status * * Ensure that Wget's exit status will reflect the problem indicated * by ERR, unless the exit status has already been set to reflect a more * important problem. */ void inform_exit_status (uerr_t err) { int new_status = get_status_for_err (err); if (new_status != WGET_EXIT_SUCCESS && (final_exit_status == WGET_EXIT_SUCCESS || new_status < final_exit_status)) { final_exit_status = new_status; } } int get_exit_status (void) { return (final_exit_status == WGET_EXIT_UNKNOWN) ? 1 : final_exit_status; } wget-1.21.2/src/ptimer.c0000644000000000000000000003057214115732710011675 00000000000000/* Portable timers. Copyright (C) 2005-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ /* This file implements "portable timers" (ptimers), objects that measure elapsed time using the primitives most appropriate for the underlying operating system. The entry points are: ptimer_new -- creates a timer. ptimer_reset -- resets the timer's elapsed time to zero. ptimer_measure -- measure and return the time elapsed since creation or last reset. ptimer_read -- reads the last measured elapsed value. ptimer_destroy -- destroy the timer. ptimer_granularity -- returns the approximate granularity of the timers. Timers measure time in seconds, returning the timings as floating point numbers, so they can carry as much precision as the underlying system timer supports. For example, to measure the time it takes to run a loop, you can use something like: ptimer *tmr = ptimer_new (); while (...) ... loop ... double secs = ptimer_measure (); printf ("The loop took %.2fs\n", secs); */ #include "wget.h" #include #include #include #include #include #include #include /* Cygwin currently (as of 2005-04-08, Cygwin 1.5.14) lacks clock_getres, but still defines _POSIX_TIMERS! Because of that we simply use the Windows timers under Cygwin. */ #ifdef __CYGWIN__ # include #endif #include "utils.h" #include "ptimer.h" /* Depending on the OS, one and only one of PTIMER_POSIX, PTIMER_GETTIMEOFDAY, or PTIMER_WINDOWS will be defined. */ #undef PTIMER_POSIX #undef PTIMER_GETTIMEOFDAY #undef PTIMER_WINDOWS #if defined(WINDOWS) || defined(__CYGWIN__) # define PTIMER_WINDOWS /* use Windows timers */ #elif _POSIX_TIMERS - 0 > 0 # define PTIMER_POSIX /* use POSIX timers (clock_gettime) */ #else # define PTIMER_GETTIMEOFDAY /* use gettimeofday */ #endif #ifdef PTIMER_POSIX /* Elapsed time measurement using POSIX timers: system time is held in struct timespec, time is retrieved using clock_gettime, and resolution using clock_getres. This method is used on Unix systems that implement POSIX timers. */ typedef struct timespec ptimer_system_time; #define IMPL_init posix_init #define IMPL_measure posix_measure #define IMPL_diff posix_diff #define IMPL_resolution posix_resolution /* clock_id to use for POSIX clocks. This tries to use CLOCK_MONOTONIC where available, CLOCK_REALTIME otherwise. */ static int posix_clock_id; /* Resolution of the clock, initialized in posix_init. */ static double posix_clock_resolution; /* Decide which clock_id to use. */ static void posix_init (void) { /* List of clocks we want to support: some systems support monotonic clocks, Solaris has "high resolution" clock (sometimes unavailable except to superuser), and all should support the real-time clock. */ #define NO_SYSCONF_CHECK -1 static const struct { int id; int sysconf_name; } clocks[] = { #if defined(_POSIX_MONOTONIC_CLOCK) && _POSIX_MONOTONIC_CLOCK - 0 >= 0 { CLOCK_MONOTONIC, _SC_MONOTONIC_CLOCK }, #endif #ifdef CLOCK_HIGHRES { CLOCK_HIGHRES, NO_SYSCONF_CHECK }, #endif { CLOCK_REALTIME, NO_SYSCONF_CHECK }, }; size_t i; /* Determine the clock we can use. For a clock to be usable, it must be confirmed with sysconf (where applicable) and with clock_getres. If no clock is found, CLOCK_REALTIME is used. */ for (i = 0; i < countof (clocks); i++) { struct timespec r; if (clocks[i].sysconf_name != NO_SYSCONF_CHECK) if (sysconf (clocks[i].sysconf_name) < 0) continue; /* sysconf claims this clock is unavailable */ if (clock_getres (clocks[i].id, &r) < 0) continue; /* clock_getres doesn't work for this clock */ posix_clock_id = clocks[i].id; posix_clock_resolution = (double) r.tv_sec + r.tv_nsec / 1e9; /* Guard against nonsense returned by a broken clock_getres. */ if (posix_clock_resolution == 0) posix_clock_resolution = 1e-3; break; } if (i == countof (clocks)) { /* If no clock was found, it means that clock_getres failed for the realtime clock. */ logprintf (LOG_NOTQUIET, _("Cannot get REALTIME clock frequency: %s\n"), strerror (errno)); /* Use CLOCK_REALTIME, but invent a plausible resolution. */ posix_clock_id = CLOCK_REALTIME; posix_clock_resolution = 1e-3; } } static inline void posix_measure (ptimer_system_time *pst) { clock_gettime (posix_clock_id, pst); } static inline double posix_diff (ptimer_system_time *pst1, ptimer_system_time *pst2) { return ((pst1->tv_sec - pst2->tv_sec) + (pst1->tv_nsec - pst2->tv_nsec) / 1e9); } static inline double posix_resolution (void) { return posix_clock_resolution; } #endif /* PTIMER_POSIX */ #ifdef PTIMER_GETTIMEOFDAY /* Elapsed time measurement using gettimeofday: system time is held in struct timeval, retrieved using gettimeofday, and resolution is unknown. This method is used Unix systems without POSIX timers. */ typedef struct timeval ptimer_system_time; #define IMPL_measure gettimeofday_measure #define IMPL_diff gettimeofday_diff #define IMPL_resolution gettimeofday_resolution static inline void gettimeofday_measure (ptimer_system_time *pst) { gettimeofday (pst, NULL); } static inline double gettimeofday_diff (ptimer_system_time *pst1, ptimer_system_time *pst2) { return ((pst1->tv_sec - pst2->tv_sec) + (pst1->tv_usec - pst2->tv_usec) / 1e6); } static inline double gettimeofday_resolution (void) { /* Granularity of gettimeofday varies wildly between architectures. However, it appears that on modern machines it tends to be better than 1ms. Assume 100 usecs. */ return 0.1; } #endif /* PTIMER_GETTIMEOFDAY */ #ifdef PTIMER_WINDOWS /* Elapsed time measurement on Windows: where high-resolution timers are available, time is stored in a LARGE_INTEGER and retrieved using QueryPerformanceCounter. Otherwise, it is stored in a DWORD and retrieved using GetTickCount. This method is used on Windows. */ typedef union { DWORD lores; /* In case GetTickCount is used */ LARGE_INTEGER hires; /* In case high-resolution timer is used */ } ptimer_system_time; #define IMPL_init windows_init #define IMPL_measure windows_measure #define IMPL_diff windows_diff #define IMPL_resolution windows_resolution /* Whether high-resolution timers are used. Set by ptimer_initialize_once the first time ptimer_new is called. */ static bool windows_hires_timers; /* Frequency of high-resolution timers -- number of updates per second. Calculated the first time ptimer_new is called provided that high-resolution timers are available. */ static double windows_hires_freq; static void windows_init (void) { LARGE_INTEGER freq; freq.QuadPart = 0; QueryPerformanceFrequency (&freq); if (freq.QuadPart != 0) { windows_hires_timers = true; windows_hires_freq = (double) freq.QuadPart; } } static inline void windows_measure (ptimer_system_time *pst) { if (windows_hires_timers) QueryPerformanceCounter (&pst->hires); else /* Where hires counters are not available, use GetTickCount rather GetSystemTime, because it is unaffected by clock skew and simpler to use. Note that overflows don't affect us because we never use absolute values of the ticker, only the differences. */ pst->lores = GetTickCount (); } static inline double windows_diff (ptimer_system_time *pst1, ptimer_system_time *pst2) { if (windows_hires_timers) return (pst1->hires.QuadPart - pst2->hires.QuadPart) / windows_hires_freq; else return pst1->lores - pst2->lores; } static double windows_resolution (void) { if (windows_hires_timers) return 1.0 / windows_hires_freq; else return 10; /* according to MSDN */ } #endif /* PTIMER_WINDOWS */ /* The code below this point is independent of timer implementation. */ struct ptimer { /* The starting point in time which, subtracted from the current time, yields elapsed time. */ ptimer_system_time start; /* The most recent elapsed time, calculated by ptimer_measure(). */ double elapsed_last; /* Approximately, the time elapsed between the true start of the measurement and the time represented by START. This is used for adjustment when clock skew is detected. */ double elapsed_pre_start; }; /* Allocate a new timer and reset it. Return the new timer. */ struct ptimer * ptimer_new (void) { struct ptimer *pt = xnew0 (struct ptimer); #ifdef IMPL_init static bool init_done; if (!init_done) { init_done = true; IMPL_init (); } #endif ptimer_reset (pt); return pt; } /* Free the resources associated with the timer. Its further use is prohibited. */ void ptimer_destroy (struct ptimer *pt) { xfree (pt); } /* Reset timer PT. This establishes the starting point from which ptimer_measure() will return the elapsed time in seconds. It is allowed to reset a previously used timer. */ void ptimer_reset (struct ptimer *pt) { /* Set the start time to the current time. */ IMPL_measure (&pt->start); pt->elapsed_last = 0; pt->elapsed_pre_start = 0; } /* Measure the elapsed time since timer creation/reset. This causes the timer to internally call clock_gettime (or gettimeofday, etc.) to update its idea of current time. The time is returned, but is also stored for later access through ptimer_read(). This function handles clock skew, i.e. time that moves backwards is ignored. */ double ptimer_measure (struct ptimer *pt) { ptimer_system_time now; double elapsed; IMPL_measure (&now); elapsed = pt->elapsed_pre_start + IMPL_diff (&now, &pt->start); /* Ideally we'd just return the difference between NOW and pt->start. However, the system timer can be set back, and we could return a value smaller than when we were last called, even a negative value. Both of these would confuse the callers, which expect us to return monotonically nondecreasing values. Therefore: if ELAPSED is smaller than its previous known value, we reset pt->start to the current time and effectively start measuring from this point. But since we don't want the elapsed value to start from zero, we set elapsed_pre_start to the last elapsed time and increment all future calculations by that amount. This cannot happen with Windows and POSIX monotonic/highres timers, but the check is not expensive. */ if (elapsed < pt->elapsed_last) { pt->start = now; pt->elapsed_pre_start = pt->elapsed_last; elapsed = pt->elapsed_last; } pt->elapsed_last = elapsed; return elapsed; } /* Return the most recent elapsed time measured with ptimer_measure. If ptimer_measure has not yet been called since the timer was created or reset, this returns 0. */ double ptimer_read (const struct ptimer *pt) { return pt->elapsed_last; } /* Return the assessed resolution of the timer implementation, in seconds. This is used by code that tries to substitute a better value for timers that have returned zero. */ double ptimer_resolution (void) { return IMPL_resolution (); } wget-1.21.2/src/wget.h0000644000000000000000000003151214115732710011343 00000000000000/* Miscellaneous declarations. Copyright (C) 1996-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ /* This file contains declarations that are universally useful and those that don't fit elsewhere. It also includes sysdep.h which includes some often-needed system includes, like the obnoxious inclusion. */ #ifndef WGET_H #define WGET_H #include "config.h" #if ((defined _WIN32 || defined __WIN32__) && !defined __CYGWIN__) # define WINDOWS #endif /* Include these, so random files need not include them. */ #include "sysdep.h" /* Disable assertions when debug support is not compiled in. */ #ifndef ENABLE_DEBUG #ifndef NDEBUG # define NDEBUG #endif #endif /* Is OpenSSL or GNUTLS available? */ #if defined HAVE_LIBSSL || defined HAVE_LIBSSL32 || defined HAVE_LIBGNUTLS # define HAVE_SSL # define HAVE_HSTS /* There's no sense in enabling HSTS without SSL */ #endif /* `gettext (FOO)' is long to write, so we use `_(FOO)'. If NLS is unavailable, _(STRING) simply returns STRING. */ #include "gettext.h" #define _(STRING) gettext(STRING) /* A pseudo function call that serves as a marker for the automated extraction of messages, but does not call gettext(). The run-time translation is done at a different place in the code. The purpose of the N_("...") call is to make the message snarfer aware that the "..." string needs to be translated. STRING should be a string literal. Concatenated strings and other string expressions won't work. The macro's expansion is not parenthesized, so that it is suitable as initializer for static 'char[]' or 'const char[]' variables. -- explanation partly taken from GNU make. */ #define N_(string) string #if HAVE_WCWIDTH && HAVE_MBTOWC # define USE_NLS_PROGRESS_BAR 1 #else /* Just to be a little paranoid about it. */ # undef USE_NLS_PROGRESS_BAR #endif /* I18N NOTE: You will notice that none of the DEBUGP messages are marked as translatable. This is intentional, for a few reasons: 1) The debug messages are not meant for the users to look at, but for the developers; as such, they should be considered more like source comments than real program output. 2) The messages are numerous, and yet they are random and frivolous ("double yuck!" and such). There would be a lot of work with no gain. 3) Finally, the debug messages are meant to be a clue for me to debug problems with Wget. If I get them in a language I don't understand, debugging will become a new challenge of its own! */ /* locale independent replacement for ctype.h */ #include "c-ctype.h" /* Conditionalize the use of GCC's __attribute__((format)) and __builtin_expect features using macros. */ #if defined(__GNUC__) && __GNUC__ >= 3 # define GCC_FORMAT_ATTR(a, b) __attribute__ ((format (printf, a, b))) # define LIKELY(exp) __builtin_expect (!!(exp), 1) # define UNLIKELY(exp) __builtin_expect ((exp), 0) #else # define GCC_FORMAT_ATTR(a, b) # define LIKELY(exp) (exp) # define UNLIKELY(exp) (exp) #endif /* Execute the following statement if debugging is both enabled at compile-time and requested at run-time; a no-op otherwise. */ #ifdef ENABLE_DEBUG # define IF_DEBUG if (UNLIKELY (opt.debug)) #else # define IF_DEBUG if (0) #endif /* Print ARGS if debugging is enabled and requested, otherwise do nothing. This must be called with an extra level of parentheses because it's not possible to pass a variable number of arguments to a macro (in portable C89). ARGS are like arguments to printf. */ #define DEBUGP(args) do { IF_DEBUG { debug_logprintf args; } } while (0) /* Pick an integer type large enough for file sizes, content lengths, and such. Because today's files can be very large, it should be a signed integer at least 64 bits wide. This can't be typedeffed to off_t because: a) off_t is always 32-bit on Windows, and b) we don't necessarily want to tie having a 64-bit type for internal calculations to having LFS support. */ /* Gnulib's stdint.h module essentially guarantees the existence of int64_t. * Thus we can simply assume it always exists and use it. */ #include typedef int64_t wgint; #define WGINT_MIN INT64_MIN #define WGINT_MAX INT64_MAX #define str_to_wgint strtoll #include "options.h" /* Everything uses this, so include them here directly. */ #ifdef __cplusplus # undef _Noreturn #endif #include "xalloc.h" /* Likewise for logging functions. */ #include "log.h" /* Likewise for quoting functions. */ #include "quote.h" #include "quotearg.h" /* Likewise for struct iri definition */ #include "iri.h" /* Useful macros used across the code: */ /* The number of elements in an array. For example: static char a[] = "foo"; -- countof(a) == 4 (note terminating \0) int a[5] = {1, 2}; -- countof(a) == 5 char *a[] = { -- countof(a) == 3 "foo", "bar", "baz" }; */ #define countof(array) (sizeof (array) / sizeof ((array)[0])) /* Zero out a value. */ #define xzero(x) memset (&(x), '\0', sizeof (x)) /* Convert an ASCII hex digit to the corresponding number between 0 and 15. c should be a hexadecimal digit that satisfies c_isxdigit; otherwise, the result is undefined. */ static inline unsigned char _unhex(unsigned char c) { return c <= '9' ? c - '0' : (c <= 'F' ? c - 'A' + 10 : c - 'a' + 10); } #define X2DIGITS_TO_NUM(h1, h2) ((_unhex (h1) << 4) + _unhex (h2)) /* The reverse of the above: convert a number in the [0, 16) range to the ASCII representation of the corresponding hexadecimal digit. `+ 0' is there so you can't accidentally use it as an lvalue. */ #define XNUM_TO_DIGIT(x) ("0123456789ABCDEF"[x] + 0) #define XNUM_TO_digit(x) ("0123456789abcdef"[x] + 0) /* Return non-zero if string bounded between BEG and END is equal to STRING_LITERAL. The comparison is case-sensitive. */ #define BOUNDED_EQUAL(beg, end, string_literal) \ ((end) - (beg) == sizeof (string_literal) - 1 \ && !memcmp (beg, string_literal, sizeof (string_literal) - 1)) /* The same as above, except the comparison is case-insensitive. */ #define BOUNDED_EQUAL_NO_CASE(beg, end, string_literal) \ ((end) - (beg) == sizeof (string_literal) - 1 \ && !c_strncasecmp (beg, string_literal, sizeof (string_literal) - 1)) /* Generally useful if you want to avoid arbitrary size limits but don't need a full dynamic array. Assumes that BASEVAR points to a malloced array of TYPE objects (or possibly a NULL pointer, if SIZEVAR is 0), with the total size stored in SIZEVAR. This macro will realloc BASEVAR as necessary so that it can hold at least NEEDED_SIZE objects. The reallocing is done by doubling, which ensures constant amortized time per element. */ #define DO_REALLOC(basevar, sizevar, needed_size, type) do { \ long DR_needed_size = (needed_size); \ long DR_newsize = 0; \ while ((sizevar) < (DR_needed_size)) { \ DR_newsize = sizevar << 1; \ if (DR_newsize < 16) \ DR_newsize = 16; \ (sizevar) = DR_newsize; \ } \ if (DR_newsize) \ basevar = xrealloc (basevar, DR_newsize * sizeof (type)); \ } while (0) /* Used to print pointers (usually for debugging). Print pointers using printf ("0x%0*lx", PTR_FORMAT (p)). (%p is too unpredictable; some implementations prepend 0x, while some don't, and most don't 0-pad the address.) */ #define PTR_FORMAT(p) (int) (2 * sizeof (void *)), (unsigned long) (p) /* Find the maximum buffer length needed to print an integer of type `x' in base 10. 24082 / 10000 = 8*log_{10}(2). */ #define MAX_INT_TO_STRING_LEN(x) ((sizeof(x) * 24082 / 10000) + 2) /* Find the minimum or maximum of two provided values */ # define MIN(i, j) ((i) <= (j) ? (i) : (j)) # define MAX(i, j) ((i) >= (j) ? (i) : (j)) extern const char *exec_name; extern const char *program_name; extern const char *program_argstring; /* Document type ("dt") flags */ enum { TEXTHTML = 0x0001, /* document is of type text/html or application/xhtml+xml */ RETROKF = 0x0002, /* retrieval was OK */ HEAD_ONLY = 0x0004, /* only send the HEAD request */ SEND_NOCACHE = 0x0008, /* send Cache-Control: no-cache and Pragma: no-cache directive */ ACCEPTRANGES = 0x0010, /* Accept-ranges header was found */ ADDED_HTML_EXTENSION = 0x0020, /* added ".html" extension due to -E */ TEXTCSS = 0x0040, /* document is of type text/css */ IF_MODIFIED_SINCE = 0x0080, /* use if-modified-since header */ METALINK_METADATA = 0x0100 /* use HTTP response for Metalink metadata */ }; /* Universal error type -- used almost everywhere. Error reporting of this detail is not generally used or needed and should be simplified. */ typedef enum { NOCONERROR, HOSTERR, CONSOCKERR, CONERROR, CONSSLERR, CONIMPOSSIBLE, NEWLOCATION, FTPOK, FTPLOGINC, FTPLOGREFUSED, FTPPORTERR, FTPSYSERR, FTPNSFOD, FTPUNKNOWNTYPE, FTPRERR, FTPSRVERR, FTPRETRINT, FTPRESTFAIL, URLERROR, FOPENERR, FOPEN_EXCL_ERR, FWRITEERR, HEOF, GATEWAYTIMEOUT, HERR, RETROK, RECLEVELEXC, WRONGCODE, FTPINVPASV, FTPNOPASV, FTPNOPBSZ, FTPNOPROT, FTPNOAUTH, CONTNOTSUPPORTED, RETRUNNEEDED, RETRFINISHED, READERR, TRYLIMEXC, FILEBADFILE, RANGEERR, RETRBADPATTERN, PROXERR, AUTHFAILED, QUOTEXC, WRITEFAILED, SSLINITFAILED, VERIFCERTERR, UNLINKERR, NEWLOCATION_KEEP_POST, CLOSEFAILED, ATTRMISSING, UNKNOWNATTR, WARC_ERR, WARC_TMP_FOPENERR, WARC_TMP_FWRITEERR, TIMECONV_ERR, METALINK_PARSE_ERROR, METALINK_RETR_ERROR, METALINK_CHKSUM_ERROR, METALINK_SIG_ERROR, METALINK_MISSING_RESOURCE, RETR_WITH_METALINK, METALINK_SIZE_ERROR } uerr_t; /* 2005-02-19 SMS. Select an appropriate "orig" suffix and a separator character for adding a unique suffix to a file name. A VMS ODS2 file system can't tolerate multiple dots. An ODS5 file system can, but even there not all dots are equal, and heroic effort would be needed to get ".html^.orig" rather than (the less desirable) "^.html.orig". It's more satisfactory always to use "_orig" on VMS (rather than including "vms.h", testing "ods5_dest", and acting accordingly). Note that code in various places assumes that this string is five characters long. */ # ifdef __VMS # define ORIG_SFX "_orig" # else /* def __VMS */ # define ORIG_SFX ".orig" # endif /* def __VMS [else] */ /* ".NNN" unique-ifying suffix separator character for unique_name() in url.c (and anywhere else). Note that on VMS, the file system's version numbers solve the problem that unique_name() is designed to handle, obviating this whole exercise. Other systems may specify a character different from "." here, if desired. */ # ifndef __VMS # define UNIQ_SEP '.' # endif /* ndef __VMS */ #if defined FUZZING && defined TESTING /* Rename fopen so we can have our own version in fuzz/main.c to not create random files. */ # define fopen(fp, mode) fopen_wget(fp, mode) # define exit(status) exit_wget(status) /* In run_wgetrc() we call fopen_wgetrc() instead of fopen, so we can catch the call in our fuzzers. */ FILE *fopen_wget(const char *pathname, const char *mode); FILE *fopen_wgetrc(const char *pathname, const char *mode); void exit_wget(int status); #else /* When not fuzzing, we want to call fopen() instead of fopen_wgetrc() */ # define fopen_wgetrc(fp, mode) fopen(fp, mode) #endif /* FUZZING && TESTING */ #endif /* WGET_H */ wget-1.21.2/src/url.h0000644000000000000000000001045414115732710011201 00000000000000/* Declarations for url.c. Copyright (C) 1996-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #ifndef URL_H #define URL_H /* Default port definitions */ #define DEFAULT_HTTP_PORT 80 #define DEFAULT_FTP_PORT 21 #define DEFAULT_HTTPS_PORT 443 #define DEFAULT_FTPS_IMPLICIT_PORT 990 /* This represents how many characters less than the OS max name length a file * should be. More precisely, a file name should be at most * (NAME_MAX - CHOMP_BUFFER) characters in length. This number was arrived at * by adding the lengths of all possible strings that could be appended to a * file name later in the code (e.g. ".orig", ".html", etc.). This is * hopefully plenty of extra characters, but I am not guaranteeing that a file * name will be of the proper length by the time the code wants to open a * file descriptor. */ #define CHOMP_BUFFER 19 /* The flags that allow clobbering the file (opening with "wb"). Defined here to avoid repetition later. #### This will require rework. */ #define ALLOW_CLOBBER (opt.noclobber || opt.always_rest || opt.timestamping \ || opt.dirstruct || opt.output_document || opt.backups > 0) /* Specifies how, or whether, user auth information should be included * in URLs regenerated from URL parse structures. */ enum url_auth_mode { URL_AUTH_SHOW, URL_AUTH_HIDE_PASSWD, URL_AUTH_HIDE }; /* Note: the ordering here is related to the order of elements in `supported_schemes' in url.c. */ enum url_scheme { SCHEME_HTTP, #ifdef HAVE_SSL SCHEME_HTTPS, #endif SCHEME_FTP, #ifdef HAVE_SSL SCHEME_FTPS, #endif SCHEME_INVALID }; /* Structure containing info on a URL. */ struct url { char *url; /* Original URL */ enum url_scheme scheme; /* URL scheme */ char *host; /* Extracted hostname */ int port; /* Port number */ /* URL components (URL-quoted). */ char *path; char *params; char *query; char *fragment; /* Extracted path info (unquoted). */ char *dir; char *file; /* Username and password (unquoted). */ char *user; char *passwd; }; /* Function declarations */ char *url_escape (const char *); char *url_escape_unsafe_and_reserved (const char *); void url_unescape (char *); void url_unescape_except_reserved (char *); struct url *url_parse (const char *, int *, struct iri *iri, bool percent_encode); char *url_error (const char *, int); char *url_full_path (const struct url *); void url_set_dir (struct url *, const char *); void url_set_file (struct url *, const char *); void url_free (struct url *); enum url_scheme url_scheme (const char *); bool url_has_scheme (const char *); bool url_valid_scheme (const char *); int scheme_default_port (enum url_scheme); void scheme_disable (enum url_scheme); const char *scheme_leading_string (enum url_scheme); char *url_string (const struct url *, enum url_auth_mode); char *url_file_name (const struct url *, char *); char *uri_merge (const char *, const char *); int mkalldirs (const char *); char *rewrite_shorthand_url (const char *); bool schemes_are_similar_p (enum url_scheme a, enum url_scheme b); bool are_urls_equal (const char *u1, const char *u2); #endif /* URL_H */ wget-1.21.2/src/res.h0000644000000000000000000000337414115732710011173 00000000000000/* Declarations for res.c. Copyright (C) 2001, 2007-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of Wget. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #ifndef RES_H #define RES_H struct robot_specs; struct robot_specs *res_parse (const char *, int); struct robot_specs *res_parse_from_file (const char *); bool res_match_path (const struct robot_specs *, const char *); void res_register_specs (const char *, int, struct robot_specs *); struct robot_specs *res_get_specs (const char *, int); bool res_retrieve_file (const char *, char **, struct iri *); bool is_robots_txt_url (const char *); void res_cleanup (void); #endif /* RES_H */ wget-1.21.2/src/log.h0000644000000000000000000000407714115732710011164 00000000000000/* Declarations for log.c. Copyright (C) 1998-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #ifndef LOG_H #define LOG_H /* The log file to which Wget writes to after HUP. */ #define DEFAULT_LOGFILE "wget-log" #include enum log_options { LOG_VERBOSE, LOG_NOTQUIET, LOG_NONVERBOSE, LOG_ALWAYS, LOG_PROGRESS }; void log_set_warc_log_fp (FILE *); void logprintf (enum log_options, const char *, ...) GCC_FORMAT_ATTR (2, 3); void debug_logprintf (const char *, ...) GCC_FORMAT_ATTR (1, 2); void logputs (enum log_options, const char *); void logflush (void); void log_set_flush (bool); bool log_set_save_context (bool); void log_init (const char *, bool); void log_close (void); void log_cleanup (void); void log_request_redirect_output (const char *); void redirect_output (bool, const char *); const char *escnonprint (const char *); const char *escnonprint_uri (const char *); #endif /* LOG_H */ wget-1.21.2/src/xattr.h0000644000000000000000000000316614115732710011543 00000000000000/* xattr.h -- POSIX Extended Attribute function mappings. Copyright (C) 2016, 2018-2021 Free Software Foundation, Inc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, see . */ #include #include #ifndef _XATTR_H #define _XATTR_H /* Store metadata name/value attributes against fp. */ int set_file_metadata (const struct url *origin_url, const struct url *referrer_url, FILE *fp); #if defined(__linux) /* libc on Linux has fsetxattr (5 arguments). */ # include # define USE_XATTR #elif defined(__APPLE__) /* libc on OS/X has fsetxattr (6 arguments). */ # include # define fsetxattr(file, name, buffer, size, flags) \ fsetxattr ((file), (name), (buffer), (size), 0, (flags)) # define USE_XATTR #elif defined(__FreeBSD_version) && (__FreeBSD_version > 500000) /* FreeBSD */ # include # include # define fsetxattr(file, name, buffer, size, flags) \ extattr_set_fd ((file), EXTATTR_NAMESPACE_USER, (name), (buffer), (size)) # define USE_XATTR #endif #endif /* _XATTR_H */ wget-1.21.2/src/hash.c0000644000000000000000000006334314115732710011322 00000000000000/* Hash tables. Copyright (C) 2000-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ /* With -DSTANDALONE, this file can be compiled outside Wget source tree. To test, also use -DTEST. */ #ifndef STANDALONE # include "wget.h" #endif #include #include #include #include #include #ifndef STANDALONE /* Get Wget's utility headers. */ # include "utils.h" #else /* Make do without them. */ # define xnew(type) (xmalloc (sizeof (type))) # define xnew0(type) (xcalloc (1, sizeof (type))) # define xnew_array(type, len) (xmalloc ((len) * sizeof (type))) # define xfree(p) do { free ((void *) (p)); p = NULL; } while (0) # ifndef countof # define countof(x) (sizeof (x) / sizeof ((x)[0])) # endif # include # define c_tolower(x) tolower ((unsigned char) (x)) # include #endif #include "hash.h" /* INTERFACE: Hash tables are a technique used to implement mapping between objects with near-constant-time access and storage. The table associates keys to values, and a value can be very quickly retrieved by providing the key. Fast lookup tables are typically implemented as hash tables. The entry points are hash_table_new -- creates the table. hash_table_destroy -- destroys the table. hash_table_put -- establishes or updates key->value mapping. hash_table_get -- retrieves value of key. hash_table_get_pair -- get key/value pair for key. hash_table_contains -- test whether the table contains key. hash_table_remove -- remove key->value mapping for given key. hash_table_for_each -- call function for each table entry. hash_table_iterate -- iterate over entries in hash table. hash_table_iter_next -- return next element during iteration. hash_table_clear -- clear hash table contents. hash_table_count -- return the number of entries in the table. The hash table grows internally as new entries are added and is not limited in size, except by available memory. The table doubles with each resize, which ensures that the amortized time per operation remains constant. If not instructed otherwise, tables created by hash_table_new consider the keys to be equal if their pointer values are the same. You can use make_string_hash_table to create tables whose keys are considered equal if their string contents are the same. In the general case, the criterion of equality used to compare keys is specified at table creation time with two callback functions, "hash" and "test". The hash function transforms the key into an arbitrary number that must be the same for two equal keys. The test function accepts two keys and returns non-zero if they are to be considered equal. Note that neither keys nor values are copied when inserted into the hash table, so they must exist for the lifetime of the table. This means that e.g. the use of static strings is OK, but objects with a shorter life-time probably need to be copied (with strdup() or the like in the case of strings) before being inserted. */ /* IMPLEMENTATION: The hash table is implemented as an open-addressed table with linear probing collision resolution. The above means that all the cells (each cell containing a key and a value pointer) are stored in a contiguous array. Array position of each cell is determined by the hash value of its key and the size of the table: location := hash(key) % size. If two different keys end up on the same position (collide), the one that came second is stored in the first unoccupied cell that follows it. This collision resolution technique is called "linear probing". There are more advanced collision resolution methods (quadratic probing, double hashing), but we don't use them because they incur more non-sequential access to the array, which results in worse CPU cache behavior. Linear probing works well as long as the count/size ratio (fullness) is kept below 75%. We make sure to grow and rehash the table whenever this threshold is exceeded. Collisions complicate deletion because simply clearing a cell followed by previously collided entries would cause those neighbors to not be picked up by find_cell later. One solution is to leave a "tombstone" marker instead of clearing the cell, and another is to recalculate the positions of adjacent cells. We take the latter approach because it results in less bookkeeping garbage and faster retrieval at the (slight) expense of deletion. */ /* Maximum allowed fullness: when hash table's fullness exceeds this value, the table is resized. */ #define HASH_MAX_FULLNESS 0.75 /* The hash table size is multiplied by this factor (and then rounded to the next prime) with each resize. This guarantees infrequent resizes. */ #define HASH_RESIZE_FACTOR 2 struct cell { void *key; void *value; }; typedef unsigned long (*hashfun_t) (const void *); typedef int (*testfun_t) (const void *, const void *); struct hash_table { hashfun_t hash_function; testfun_t test_function; struct cell *cells; /* contiguous array of cells. */ int size; /* size of the array. */ int count; /* number of occupied entries. */ int resize_threshold; /* after size exceeds this number of entries, resize the table. */ int prime_offset; /* the offset of the current prime in the prime table. */ }; /* We use the all-bits-set constant (INVALID_PTR) marker to mean that a cell is empty. It is unaligned and therefore illegal as a pointer. INVALID_PTR_CHAR (0xff) is the single-character constant used to initialize the entire cells array as empty. The all-bits-set value is a better choice than NULL because it allows the use of NULL/0 keys. Since the keys are either integers or pointers, the only key that cannot be used is the integer value -1. This is acceptable because it still allows the use of nonnegative integer keys. */ #define INVALID_PTR ((void *) ~(uintptr_t) 0) #ifndef UCHAR_MAX # define UCHAR_MAX 0xff #endif #define INVALID_PTR_CHAR UCHAR_MAX /* Whether the cell C is occupied (non-empty). */ #define CELL_OCCUPIED(c) ((c)->key != INVALID_PTR) /* Clear the cell C, i.e. mark it as empty (unoccupied). */ #define CLEAR_CELL(c) ((c)->key = INVALID_PTR) /* "Next" cell is the cell following C, but wrapping back to CELLS when C would reach CELLS+SIZE. */ #define NEXT_CELL(c, cells, size) (c != cells + (size - 1) ? c + 1 : cells) /* Loop over occupied cells starting at C, terminating the loop when an empty cell is encountered. */ #define FOREACH_OCCUPIED_ADJACENT(c, cells, size) \ for (; CELL_OCCUPIED (c); c = NEXT_CELL (c, cells, size)) /* Return the position of KEY in hash table SIZE large, hash function being HASHFUN. */ #define HASH_POSITION(key, hashfun, size) ((hashfun) (key) % size) /* Find a prime near, but greater than or equal to SIZE. The primes are looked up from a table with a selection of primes convenient for this purpose. PRIME_OFFSET is a minor optimization: it specifies start position for the search for the large enough prime. The final offset is stored in the same variable. That way the list of primes does not have to be scanned from the beginning each time around. */ static int prime_size (int size, int *prime_offset) { static const int primes[] = { 13, 19, 29, 41, 59, 79, 107, 149, 197, 263, 347, 457, 599, 787, 1031, 1361, 1777, 2333, 3037, 3967, 5167, 6719, 8737, 11369, 14783, 19219, 24989, 32491, 42257, 54941, 71429, 92861, 120721, 156941, 204047, 265271, 344857, 448321, 582821, 757693, 985003, 1280519, 1664681, 2164111, 2813353, 3657361, 4754591, 6180989, 8035301, 10445899, 13579681, 17653589, 22949669, 29834603, 38784989, 50420551, 65546729, 85210757, 110774011, 144006217, 187208107, 243370577, 316381771, 411296309, 534685237, 695090819, 903618083, 1174703521, 1527114613, 1837299131, 2147483647 }; size_t i; for (i = *prime_offset; i < countof (primes); i++) if (primes[i] >= size) { /* Set the offset to the next prime. That is safe because, next time we are called, it will be with a larger SIZE, which means we could never return the same prime anyway. (If that is not the case, the caller can simply reset *prime_offset.) */ *prime_offset = i + 1; return primes[i]; } abort (); } static int cmp_pointer (const void *, const void *); /* Create a hash table with hash function HASH_FUNCTION and test function TEST_FUNCTION. The table is empty (its count is 0), but pre-allocated to store at least ITEMS items. ITEMS is the number of items that the table can accept without needing to resize. It is useful when creating a table that is to be immediately filled with a known number of items. In that case, the regrows are a waste of time, and specifying ITEMS correctly will avoid them altogether. Note that hash tables grow dynamically regardless of ITEMS. The only use of ITEMS is to preallocate the table and avoid unnecessary dynamic regrows. Don't bother making ITEMS prime because it's not used as size unchanged. To start with a small table that grows as needed, simply specify zero ITEMS. If hash and test callbacks are not specified, identity mapping is assumed, i.e. pointer values are used for key comparison. (Common Lisp calls such tables EQ hash tables, and Java calls them IdentityHashMaps.) If your keys require different comparison, specify hash and test functions. For easy use of C strings as hash keys, you can use the convenience functions make_string_hash_table and make_nocase_string_hash_table. */ struct hash_table * hash_table_new (int items, unsigned long (*hash_function) (const void *), int (*test_function) (const void *, const void *)) { int size; struct hash_table *ht = xnew (struct hash_table); ht->hash_function = hash_function ? hash_function : hash_pointer; ht->test_function = test_function ? test_function : cmp_pointer; /* If the size of struct hash_table ever becomes a concern, this field can go. (Wget doesn't create many hashes.) */ ht->prime_offset = 0; /* Calculate the size that ensures that the table will store at least ITEMS keys without the need to resize. */ size = (int) (1 + items / HASH_MAX_FULLNESS); size = prime_size (size, &ht->prime_offset); ht->size = size; ht->resize_threshold = (int) (size * HASH_MAX_FULLNESS); /*assert (ht->resize_threshold >= items);*/ ht->cells = xnew_array (struct cell, ht->size); /* Mark cells as empty. We use 0xff rather than 0 to mark empty keys because it allows us to use NULL/0 as keys. */ memset (ht->cells, INVALID_PTR_CHAR, size * sizeof (struct cell)); ht->count = 0; return ht; } /* Free the data associated with hash table HT. */ void hash_table_destroy (struct hash_table *ht) { xfree (ht->cells); xfree (ht); } /* The heart of most functions in this file -- find the cell whose KEY is equal to key, using linear probing. Returns the cell that matches KEY, or the first empty cell if none matches. */ static inline struct cell * find_cell (const struct hash_table *ht, const void *key) { struct cell *cells = ht->cells; int size = ht->size; struct cell *c = cells + HASH_POSITION (key, ht->hash_function, size); testfun_t equals = ht->test_function; FOREACH_OCCUPIED_ADJACENT (c, cells, size) if (equals (key, c->key)) break; return c; } /* Get the value that corresponds to the key KEY in the hash table HT. If no value is found, return NULL. Note that NULL is a legal value for value; if you are storing NULLs in your hash table, you can use hash_table_contains to be sure that a (possibly NULL) value exists in the table. Or, you can use hash_table_get_pair instead of this function. */ void * hash_table_get (const struct hash_table *ht, const void *key) { struct cell *c = find_cell (ht, key); if (CELL_OCCUPIED (c)) return c->value; else return NULL; } /* Like hash_table_get, but writes out the pointers to both key and value. Returns non-zero on success. */ int hash_table_get_pair (const struct hash_table *ht, const void *lookup_key, void *orig_key, void *value) { struct cell *c = find_cell (ht, lookup_key); if (CELL_OCCUPIED (c)) { if (orig_key) *(void **)orig_key = c->key; if (value) *(void **)value = c->value; return 1; } else return 0; } /* Return 1 if HT contains KEY, 0 otherwise. */ int hash_table_contains (const struct hash_table *ht, const void *key) { struct cell *c = find_cell (ht, key); return CELL_OCCUPIED (c); } /* Grow hash table HT as necessary, and rehash all the key-value mappings. */ static void grow_hash_table (struct hash_table *ht) { hashfun_t hasher = ht->hash_function; struct cell *old_cells = ht->cells; struct cell *old_end = ht->cells + ht->size; struct cell *c, *cells; int newsize; newsize = prime_size (ht->size * HASH_RESIZE_FACTOR, &ht->prime_offset); #if 0 printf ("growing from %d to %d; fullness %.2f%% to %.2f%%\n", ht->size, newsize, 100.0 * ht->count / ht->size, 100.0 * ht->count / newsize); #endif ht->size = newsize; ht->resize_threshold = (int) (newsize * HASH_MAX_FULLNESS); cells = xnew_array (struct cell, newsize); memset (cells, INVALID_PTR_CHAR, newsize * sizeof (struct cell)); ht->cells = cells; for (c = old_cells; c < old_end; c++) if (CELL_OCCUPIED (c)) { struct cell *new_c; /* We don't need to test for uniqueness of keys because they come from the hash table and are therefore known to be unique. */ new_c = cells + HASH_POSITION (c->key, hasher, newsize); FOREACH_OCCUPIED_ADJACENT (new_c, cells, newsize) ; *new_c = *c; } xfree (old_cells); } /* Put VALUE in the hash table HT under the key KEY. This regrows the table if necessary. */ void hash_table_put (struct hash_table *ht, const void *key, const void *value) { struct cell *c = find_cell (ht, key); if (CELL_OCCUPIED (c)) { /* update existing item */ c->key = (void *)key; /* const? */ c->value = (void *)value; return; } /* If adding the item would make the table exceed max. fullness, grow the table first. */ if (ht->count >= ht->resize_threshold) { grow_hash_table (ht); c = find_cell (ht, key); } /* add new item */ ++ht->count; c->key = (void *)key; /* const? */ c->value = (void *)value; } /* Remove KEY->value mapping from HT. Return 0 if there was no such entry; return 1 if an entry was removed. */ int hash_table_remove (struct hash_table *ht, const void *key) { struct cell *c = find_cell (ht, key); if (!CELL_OCCUPIED (c)) return 0; else { int size = ht->size; struct cell *cells = ht->cells; hashfun_t hasher = ht->hash_function; CLEAR_CELL (c); --ht->count; /* Rehash all the entries following C. The alternative approach is to mark the entry as deleted, i.e. create a "tombstone". That speeds up removal, but leaves a lot of garbage and slows down hash_table_get and hash_table_put. */ c = NEXT_CELL (c, cells, size); FOREACH_OCCUPIED_ADJACENT (c, cells, size) { const void *key2 = c->key; struct cell *c_new; /* Find the new location for the key. */ c_new = cells + HASH_POSITION (key2, hasher, size); FOREACH_OCCUPIED_ADJACENT (c_new, cells, size) if (key2 == c_new->key) /* The cell C (key2) is already where we want it (in C_NEW's "chain" of keys.) */ goto next_rehash; *c_new = *c; CLEAR_CELL (c); next_rehash: ; } return 1; } } /* Clear HT of all entries. After calling this function, the count and the fullness of the hash table will be zero. The size will remain unchanged. */ void hash_table_clear (struct hash_table *ht) { memset (ht->cells, INVALID_PTR_CHAR, ht->size * sizeof (struct cell)); ht->count = 0; } /* Call FN for each entry in HT. FN is called with three arguments: the key, the value, and ARG. When FN returns a non-zero value, the mapping stops. It is undefined what happens if you add or remove entries in the hash table while hash_table_for_each is running. The exception is the entry you're currently mapping over; you may call hash_table_put or hash_table_remove on that entry's key. That is also the reason why this function cannot be implemented in terms of hash_table_iterate. */ void hash_table_for_each (struct hash_table *ht, int (*fn) (void *, void *, void *), void *arg) { struct cell *c = ht->cells; struct cell *end = ht->cells + ht->size; for (; c < end; c++) if (CELL_OCCUPIED (c)) { void *key; repeat: key = c->key; if (fn (key, c->value, arg)) return; /* hash_table_remove might have moved the adjacent cells. */ if (c->key != key && CELL_OCCUPIED (c)) goto repeat; } } /* Initiate iteration over HT. Entries are obtained with hash_table_iter_next, a typical iteration loop looking like this: hash_table_iterator iter; for (hash_table_iterate (ht, &iter); hash_table_iter_next (&iter); ) ... do something with iter.key and iter.value ... The iterator does not need to be deallocated after use. The hash table must not be modified while being iterated over. */ void hash_table_iterate (struct hash_table *ht, hash_table_iterator *iter) { iter->pos = ht->cells; iter->end = ht->cells + ht->size; } /* Get the next hash table entry. ITER is an iterator object initialized using hash_table_iterate. While there are more entries, the key and value pointers are stored to ITER->key and ITER->value respectively and 1 is returned. When there are no more entries, 0 is returned. If the hash table is modified between calls to this function, the result is undefined. */ int hash_table_iter_next (hash_table_iterator *iter) { struct cell *c = iter->pos; struct cell *end = iter->end; for (; c < end; c++) if (CELL_OCCUPIED (c)) { iter->key = c->key; iter->value = c->value; iter->pos = c + 1; return 1; } return 0; } /* Return the number of elements in the hash table. This is not the same as the physical size of the hash table, which is always greater than the number of elements. */ int hash_table_count (const struct hash_table *ht) { return ht->count; } /* Functions from this point onward are meant for convenience and don't strictly belong to this file. However, this is as good a place for them as any. */ /* Guidelines for creating custom hash and test functions: - The test function returns non-zero for keys that are considered "equal", zero otherwise. - The hash function returns a number that represents the "distinctness" of the object. In more precise terms, it means that for any two objects that test "equal" under the test function, the hash function MUST produce the same result. This does not mean that all different objects must produce different values (that would be "perfect" hashing), only that non-distinct objects must produce the same values! For instance, a hash function that returns 0 for any given object is a perfectly valid (albeit extremely bad) hash function. A hash function that hashes a string by adding up all its characters is another example of a valid (but still quite bad) hash function. It is not hard to make hash and test functions agree about equality. For example, if the test function compares strings case-insensitively, the hash function can lower-case the characters when calculating the hash value. That ensures that two strings differing only in case will hash the same. - To prevent performance degradation, choose a hash function with as good "spreading" as possible. A good hash function will use all the bits of the input when calculating the hash, and will react to even small changes in input with a completely different output. But don't make the hash function itself overly slow, because you'll be incurring a non-negligible overhead to all hash table operations. */ /* * Support for hash tables whose keys are strings. * */ /* Base 31 hash function. Taken from Gnome's glib, modified to use standard C types. We used to use the popular hash function from the Dragon Book, but this one seems to perform much better, both by being faster and by generating less collisions. */ #ifdef __clang__ __attribute__((no_sanitize("integer"))) #endif static unsigned long hash_string (const void *key) { const char *p = key; unsigned int h = *p; if (h) for (p += 1; *p != '\0'; p++) h = (h << 5) - h + *p; return h; } /* Frontend for strcmp usable for hash tables. */ static int cmp_string (const void *s1, const void *s2) { return !strcmp ((const char *)s1, (const char *)s2); } /* Return a hash table of preallocated to store at least ITEMS items suitable to use strings as keys. */ struct hash_table * make_string_hash_table (int items) { return hash_table_new (items, hash_string, cmp_string); } /* * Support for hash tables whose keys are strings, but which are * compared case-insensitively. * */ /* Like hash_string, but produce the same hash regardless of the case. */ #ifdef __clang__ __attribute__((no_sanitize("integer"))) #endif static unsigned long hash_string_nocase (const void *key) { const char *p = key; unsigned int h = c_tolower (*p); if (h) for (p += 1; *p != '\0'; p++) h = (h << 5) - h + c_tolower (*p); return h; } /* Like string_cmp, but doing case-insensitive comparison. */ static int string_cmp_nocase (const void *s1, const void *s2) { return !strcasecmp ((const char *)s1, (const char *)s2); } /* Like make_string_hash_table, but uses string_hash_nocase and string_cmp_nocase. */ struct hash_table * make_nocase_string_hash_table (int items) { return hash_table_new (items, hash_string_nocase, string_cmp_nocase); } /* Hashing of numeric values, such as pointers and integers. This implementation is the Robert Jenkins' 32 bit Mix Function, with a simple adaptation for 64-bit values. According to Jenkins it should offer excellent spreading of values. Unlike the popular Knuth's multiplication hash, this function doesn't need to know the hash table size to work. */ #ifdef __clang__ __attribute__((no_sanitize("integer"))) #endif unsigned long hash_pointer (const void *ptr) { uintptr_t key = (uintptr_t) ptr; key += (key << 12); key ^= (key >> 22); key += (key << 4); key ^= (key >> 9); key += (key << 10); key ^= (key >> 2); key += (key << 7); key ^= (key >> 12); #if SIZEOF_VOID_P > 4 key += (key << 44); key ^= (key >> 54); key += (key << 36); key ^= (key >> 41); key += (key << 42); key ^= (key >> 34); key += (key << 39); key ^= (key >> 44); #endif return (unsigned long) key; } static int cmp_pointer (const void *ptr1, const void *ptr2) { return ptr1 == ptr2; } #ifdef TEST #include #include void print_hash (struct hash_table *sht) { hash_table_iterator iter; int count = 0; for (hash_table_iterate (sht, &iter); hash_table_iter_next (&iter); ++count) printf ("%s: %s\n", iter.key, iter.value); assert (count == sht->count); } int main (void) { struct hash_table *ht = make_string_hash_table (0); char line[80]; #ifdef ENABLE_NLS /* Set the current locale. */ setlocale (LC_ALL, ""); /* Set the text message domain. */ bindtextdomain ("wget", LOCALEDIR); textdomain ("wget"); #endif /* ENABLE_NLS */ while ((fgets (line, sizeof (line), stdin))) { int len = strlen (line); if (len <= 1) continue; line[--len] = '\0'; if (!hash_table_contains (ht, line)) hash_table_put (ht, strdup (line), "here I am!"); #if 1 if (len % 5 == 0) { char *line_copy; if (hash_table_get_pair (ht, line, &line_copy, NULL)) { hash_table_remove (ht, line); xfree (line_copy); } } #endif } #if 0 print_hash (ht); #endif #if 1 printf ("%d %d\n", ht->count, ht->size); #endif return 0; } #endif /* TEST */ wget-1.21.2/src/retr.c0000644000000000000000000013153714115732710011354 00000000000000/* File retrieval. Copyright (C) 1996-2011, 2014-2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #include "wget.h" #include #include #include #include #include #include #ifdef VMS # include /* For delete(). */ #endif #ifdef HAVE_LIBZ # include #endif #include "exits.h" #include "utils.h" #include "retr.h" #include "progress.h" #include "url.h" #include "recur.h" #include "ftp.h" #include "http.h" #include "host.h" #include "connect.h" #include "hash.h" #include "convert.h" #include "ptimer.h" #include "html-url.h" #include "iri.h" #include "hsts.h" /* Total size of downloaded files. Used to enforce quota. */ wgint total_downloaded_bytes; /* Total download time in seconds. */ double total_download_time; /* If non-NULL, the stream to which output should be written. This stream is initialized when `-O' is used. */ FILE *output_stream; /* Whether output_document is a regular file we can manipulate, i.e. not `-' or a device file. */ bool output_stream_regular; static struct { wgint chunk_bytes; double chunk_start; double sleep_adjust; } limit_data; static void limit_bandwidth_reset (void) { xzero (limit_data); } #ifdef HAVE_LIBZ static voidpf zalloc (voidpf opaque, unsigned int items, unsigned int size) { (void) opaque; return (voidpf) xcalloc (items, size); } static void zfree (voidpf opaque, voidpf address) { (void) opaque; xfree (address); } #endif /* Limit the bandwidth by pausing the download for an amount of time. BYTES is the number of bytes received from the network, and TIMER is the timer that started at the beginning of download. */ static void limit_bandwidth (wgint bytes, struct ptimer *timer) { double delta_t = ptimer_read (timer) - limit_data.chunk_start; double expected; limit_data.chunk_bytes += bytes; /* Calculate the amount of time we expect downloading the chunk should take. If in reality it took less time, sleep to compensate for the difference. */ expected = (double) limit_data.chunk_bytes / opt.limit_rate; if (expected > delta_t) { double slp = expected - delta_t + limit_data.sleep_adjust; double t0, t1; if (slp < 0.2) { DEBUGP (("deferring a %.2f ms sleep (%s/%.2f).\n", slp * 1000, number_to_static_string (limit_data.chunk_bytes), delta_t)); return; } DEBUGP (("\nsleeping %.2f ms for %s bytes, adjust %.2f ms\n", slp * 1000, number_to_static_string (limit_data.chunk_bytes), limit_data.sleep_adjust)); t0 = ptimer_read (timer); xsleep (slp); t1 = ptimer_measure (timer); /* Due to scheduling, we probably slept slightly longer (or shorter) than desired. Calculate the difference between the desired and the actual sleep, and adjust the next sleep by that amount. */ limit_data.sleep_adjust = slp - (t1 - t0); /* If sleep_adjust is very large, it's likely due to suspension and not clock inaccuracy. Don't enforce those. */ if (limit_data.sleep_adjust > 0.5) limit_data.sleep_adjust = 0.5; else if (limit_data.sleep_adjust < -0.5) limit_data.sleep_adjust = -0.5; } limit_data.chunk_bytes = 0; limit_data.chunk_start = ptimer_read (timer); } /* Write data in BUF to OUT. However, if *SKIP is non-zero, skip that amount of data and decrease SKIP. Increment *TOTAL by the amount of data written. If OUT2 is not NULL, also write BUF to OUT2. In case of error writing to OUT, -2 is returned. In case of error writing to OUT2, -3 is returned. Return 1 if the whole BUF was skipped. */ static int write_data (FILE *out, FILE *out2, const char *buf, int bufsize, wgint *skip, wgint *written) { if (out == NULL && out2 == NULL) return 1; if (skip) { if (*skip > bufsize) { *skip -= bufsize; return 1; } if (*skip) { buf += *skip; bufsize -= *skip; *skip = 0; if (bufsize == 0) return 1; } } if (out) fwrite (buf, 1, bufsize, out); if (out2) fwrite (buf, 1, bufsize, out2); if (written) *written += bufsize; /* Immediately flush the downloaded data. This should not hinder performance: fast downloads will arrive in large 16K chunks (which stdio would write out immediately anyway), and slow downloads wouldn't be limited by disk speed. */ /* 2005-04-20 SMS. Perhaps it shouldn't hinder performance, but it sure does, at least on VMS (more than 2X). Rather than speculate on what it should or shouldn't do, it might make more sense to test it. Even better, it might be nice to explain what possible benefit it could offer, as it appears to be a clear invitation to poor performance with no actual justification. (Also, why 16K? Anyone test other values?) */ #ifndef __VMS if (out) fflush (out); if (out2) fflush (out2); #endif /* ndef __VMS */ if (out && ferror (out)) return -2; else if (out2 && ferror (out2)) return -3; return 0; } /* Read the contents of file descriptor FD until it the connection terminates or a read error occurs. The data is read in portions of up to 16K and written to OUT as it arrives. If opt.verbose is set, the progress is shown. TOREAD is the amount of data expected to arrive, normally only used by the progress gauge. STARTPOS is the position from which the download starts, used by the progress gauge. If QTYREAD is non-NULL, the value it points to is incremented by the amount of data read from the network. If QTYWRITTEN is non-NULL, the value it points to is incremented by the amount of data written to disk. The time it took to download the data is stored to ELAPSED. If OUT2 is non-NULL, the contents is also written to OUT2. OUT2 will get an exact copy of the response: if this is a chunked response, everything -- including the chunk headers -- is written to OUT2. (OUT will only get the unchunked response.) The function exits and returns the amount of data read. In case of error while reading data, -1 is returned. In case of error while writing data to OUT, -2 is returned. In case of error while writing data to OUT2, -3 is returned. */ int fd_read_body (const char *downloaded_filename, int fd, FILE *out, wgint toread, wgint startpos, wgint *qtyread, wgint *qtywritten, double *elapsed, int flags, FILE *out2) { int ret = 0; #undef max #define max(a,b) ((a) > (b) ? (a) : (b)) int dlbufsize = max (BUFSIZ, 8 * 1024); char *dlbuf = xmalloc (dlbufsize); struct ptimer *timer = NULL; double last_successful_read_tm = 0; /* The progress gauge, set according to the user preferences. */ void *progress = NULL; /* Non-zero if the progress gauge is interactive, i.e. if it can continually update the display. When true, smaller timeout values are used so that the gauge can update the display when data arrives slowly. */ bool progress_interactive = false; bool exact = !!(flags & rb_read_exactly); /* Used only by HTTP/HTTPS chunked transfer encoding. */ bool chunked = flags & rb_chunked_transfer_encoding; wgint skip = 0; /* How much data we've read/written. */ wgint sum_read = 0; wgint sum_written = 0; wgint remaining_chunk_size = 0; #ifdef HAVE_LIBZ /* try to minimize the number of calls to inflate() and write_data() per call to fd_read() */ unsigned int gzbufsize = dlbufsize * 4; char *gzbuf = NULL; z_stream gzstream; if (flags & rb_compressed_gzip) { gzbuf = xmalloc (gzbufsize); if (gzbuf != NULL) { gzstream.zalloc = zalloc; gzstream.zfree = zfree; gzstream.opaque = Z_NULL; gzstream.next_in = Z_NULL; gzstream.avail_in = 0; #define GZIP_DETECT 32 /* gzip format detection */ #define GZIP_WINDOW 15 /* logarithmic window size (default: 15) */ ret = inflateInit2 (&gzstream, GZIP_DETECT | GZIP_WINDOW); if (ret != Z_OK) { xfree (gzbuf); errno = (ret == Z_MEM_ERROR) ? ENOMEM : EINVAL; ret = -1; goto out; } } else { errno = ENOMEM; ret = -1; goto out; } } #endif if (flags & rb_skip_startpos) skip = startpos; if (opt.show_progress) { const char *filename_progress; /* If we're skipping STARTPOS bytes, pass 0 as the INITIAL argument to progress_create because the indicator doesn't (yet) know about "skipping" data. */ wgint start = skip ? 0 : startpos; if (opt.dir_prefix) filename_progress = downloaded_filename + strlen (opt.dir_prefix) + 1; else filename_progress = downloaded_filename; progress = progress_create (filename_progress, start, start + toread); progress_interactive = progress_interactive_p (progress); } if (opt.limit_rate) limit_bandwidth_reset (); /* A timer is needed for tracking progress, for throttling, and for tracking elapsed time. If either of these are requested, start the timer. */ if (progress || opt.limit_rate || elapsed) { timer = ptimer_new (); last_successful_read_tm = 0; } /* Use a smaller buffer for low requested bandwidths. For example, with --limit-rate=2k, it doesn't make sense to slurp in 16K of data and then sleep for 8s. With buffer size equal to the limit, we never have to sleep for more than one second. */ if (opt.limit_rate && opt.limit_rate < dlbufsize) dlbufsize = opt.limit_rate; /* Read from FD while there is data to read. Normally toread==0 means that it is unknown how much data is to arrive. However, if EXACT is set, then toread==0 means what it says: that no data should be read. */ while (!exact || (sum_read < toread)) { int rdsize; double tmout = opt.read_timeout; if (chunked) { if (remaining_chunk_size == 0) { char *line = fd_read_line (fd); char *endl; if (line == NULL) { ret = -1; break; } else if (out2 != NULL) fwrite (line, 1, strlen (line), out2); remaining_chunk_size = strtol (line, &endl, 16); xfree (line); if (remaining_chunk_size < 0) { ret = -1; break; } if (remaining_chunk_size == 0) { ret = 0; line = fd_read_line (fd); if (line == NULL) ret = -1; else { if (out2 != NULL) fwrite (line, 1, strlen (line), out2); xfree (line); } break; } } rdsize = MIN (remaining_chunk_size, dlbufsize); } else rdsize = exact ? MIN (toread - sum_read, dlbufsize) : dlbufsize; if (progress_interactive) { /* For interactive progress gauges, always specify a ~1s timeout, so that the gauge can be updated regularly even when the data arrives very slowly or stalls. */ tmout = 0.95; /* avoid wrong 'interactive timeout' */ errno = 0; if (opt.read_timeout) { double waittm; waittm = ptimer_read (timer) - last_successful_read_tm; if (waittm + tmout > opt.read_timeout) { /* Don't let total idle time exceed read timeout. */ tmout = opt.read_timeout - waittm; /* if 0 fd_read can be 'blocked read' */ if (tmout <= 0) { /* We've already exceeded the timeout. */ ret = -1, errno = ETIMEDOUT; break; } } } } ret = fd_read (fd, dlbuf, rdsize, tmout); if (progress_interactive && ret < 0 && errno == ETIMEDOUT) ret = 0; /* interactive timeout, handled above */ else if (ret <= 0) break; /* EOF or read error */ if (progress || opt.limit_rate || elapsed) { ptimer_measure (timer); if (ret > 0) last_successful_read_tm = ptimer_read (timer); } if (ret > 0) { int write_res; sum_read += ret; #ifdef HAVE_LIBZ if (gzbuf != NULL) { int err; int towrite; /* Write original data to WARC file */ write_res = write_data (NULL, out2, dlbuf, ret, NULL, NULL); if (write_res < 0) { ret = write_res; goto out; } gzstream.avail_in = ret; gzstream.next_in = (unsigned char *) dlbuf; do { gzstream.avail_out = gzbufsize; gzstream.next_out = (unsigned char *) gzbuf; err = inflate (&gzstream, Z_NO_FLUSH); switch (err) { case Z_MEM_ERROR: errno = ENOMEM; ret = -1; goto out; case Z_NEED_DICT: case Z_DATA_ERROR: errno = EINVAL; ret = -1; goto out; case Z_STREAM_END: if (exact && sum_read != toread) { DEBUGP(("zlib stream ended unexpectedly after %"PRId64"/%"PRId64 " bytes\n", sum_read, toread)); } } towrite = gzbufsize - gzstream.avail_out; write_res = write_data (out, NULL, gzbuf, towrite, &skip, &sum_written); if (write_res < 0) { ret = write_res; goto out; } } while (gzstream.avail_out == 0); } else #endif { write_res = write_data (out, out2, dlbuf, ret, &skip, &sum_written); if (write_res < 0) { ret = write_res; goto out; } } if (chunked) { remaining_chunk_size -= ret; if (remaining_chunk_size == 0) { char *line = fd_read_line (fd); if (line == NULL) { ret = -1; break; } else { if (out2 != NULL) fwrite (line, 1, strlen (line), out2); xfree (line); } } } } if (opt.limit_rate) limit_bandwidth (ret, timer); if (progress) progress_update (progress, ret, ptimer_read (timer)); #ifdef WINDOWS if (toread > 0 && opt.show_progress) ws_percenttitle (100.0 * (startpos + sum_read) / (startpos + toread)); #endif } if (ret < -1) ret = -1; out: if (progress) progress_finish (progress, ptimer_read (timer)); if (timer) { if (elapsed) *elapsed = ptimer_read (timer); ptimer_destroy (timer); } #ifdef HAVE_LIBZ if (gzbuf != NULL) { int err = inflateEnd (&gzstream); if (ret >= 0) { /* with compression enabled, ret must be 0 if successful */ if (err == Z_OK) ret = 0; else { errno = EINVAL; ret = -1; } } xfree (gzbuf); if (gzstream.total_in != (uLong) sum_read) { DEBUGP(("zlib read size differs from raw read size (%lu/%"PRId64")\n", gzstream.total_in, sum_read)); } } #endif if (qtyread) *qtyread += sum_read; if (qtywritten) *qtywritten += sum_written; xfree (dlbuf); return ret; } /* Read a hunk of data from FD, up until a terminator. The hunk is limited by whatever the TERMINATOR callback chooses as its terminator. For example, if terminator stops at newline, the hunk will consist of a line of data; if terminator stops at two newlines, it can be used to read the head of an HTTP response. Upon determining the boundary, the function returns the data (up to the terminator) in malloc-allocated storage. In case of read error, NULL is returned. In case of EOF and no data read, NULL is returned and errno set to 0. In case of having read some data, but encountering EOF before seeing the terminator, the data that has been read is returned, but it will (obviously) not contain the terminator. The TERMINATOR function is called with three arguments: the beginning of the data read so far, the beginning of the current block of peeked-at data, and the length of the current block. Depending on its needs, the function is free to choose whether to analyze all data or just the newly arrived data. If TERMINATOR returns NULL, it means that the terminator has not been seen. Otherwise it should return a pointer to the charactre immediately following the terminator. The idea is to be able to read a line of input, or otherwise a hunk of text, such as the head of an HTTP request, without crossing the boundary, so that the next call to fd_read etc. reads the data after the hunk. To achieve that, this function does the following: 1. Peek at incoming data. 2. Determine whether the peeked data, along with the previously read data, includes the terminator. 2a. If yes, read the data until the end of the terminator, and exit. 2b. If no, read the peeked data and goto 1. The function is careful to assume as little as possible about the implementation of peeking. For example, every peek is followed by a read. If the read returns a different amount of data, the process is retried until all data arrives safely. SIZEHINT is the buffer size sufficient to hold all the data in the typical case (it is used as the initial buffer size). MAXSIZE is the maximum amount of memory this function is allowed to allocate, or 0 if no upper limit is to be enforced. This function should be used as a building block for other functions -- see fd_read_line as a simple example. */ char * fd_read_hunk (int fd, hunk_terminator_t terminator, long sizehint, long maxsize) { long bufsize = sizehint; char *hunk = xmalloc (bufsize); int tail = 0; /* tail position in HUNK */ assert (!maxsize || maxsize >= bufsize); while (1) { const char *end; int pklen, rdlen, remain; /* First, peek at the available data. */ pklen = fd_peek (fd, hunk + tail, bufsize - 1 - tail, -1); if (pklen < 0) { xfree (hunk); return NULL; } end = terminator (hunk, hunk + tail, pklen); if (end) { /* The data contains the terminator: we'll drain the data up to the end of the terminator. */ remain = end - (hunk + tail); assert (remain >= 0); if (remain == 0) { /* No more data needs to be read. */ hunk[tail] = '\0'; return hunk; } if (bufsize - 1 < tail + remain) { bufsize = tail + remain + 1; hunk = xrealloc (hunk, bufsize); } } else /* No terminator: simply read the data we know is (or should be) available. */ remain = pklen; /* Now, read the data. Note that we make no assumptions about how much data we'll get. (Some TCP stacks are notorious for read returning less data than the previous MSG_PEEK.) */ rdlen = fd_read (fd, hunk + tail, remain, 0); if (rdlen < 0) { xfree (hunk); return NULL; } tail += rdlen; hunk[tail] = '\0'; if (rdlen == 0) { if (tail == 0) { /* EOF without anything having been read */ xfree (hunk); errno = 0; return NULL; } else /* EOF seen: return the data we've read. */ return hunk; } if (end && rdlen == remain) /* The terminator was seen and the remaining data drained -- we got what we came for. */ return hunk; /* Keep looping until all the data arrives. */ if (tail == bufsize - 1) { /* Double the buffer size, but refuse to allocate more than MAXSIZE bytes. */ if (maxsize && bufsize >= maxsize) { xfree (hunk); errno = ENOMEM; return NULL; } bufsize <<= 1; if (maxsize && bufsize > maxsize) bufsize = maxsize; hunk = xrealloc (hunk, bufsize); } } } static const char * line_terminator (const char *start _GL_UNUSED, const char *peeked, int peeklen) { const char *p = memchr (peeked, '\n', peeklen); if (p) /* p+1 because the line must include '\n' */ return p + 1; return NULL; } /* The maximum size of the single line we agree to accept. This is not meant to impose an arbitrary limit, but to protect the user from Wget slurping up available memory upon encountering malicious or buggy server output. Define it to 0 to remove the limit. */ #define FD_READ_LINE_MAX 4096 /* Read one line from FD and return it. The line is allocated using malloc, but is never larger than FD_READ_LINE_MAX. If an error occurs, or if no data can be read, NULL is returned. In the former case errno indicates the error condition, and in the latter case, errno is NULL. */ char * fd_read_line (int fd) { return fd_read_hunk (fd, line_terminator, 128, FD_READ_LINE_MAX); } /* Return a printed representation of the download rate, along with the units appropriate for the download speed. */ const char * retr_rate (wgint bytes, double secs) { static char res[20]; static const char *rate_names[] = {"B/s", "KB/s", "MB/s", "GB/s" }; static const char *rate_names_bits[] = {"b/s", "Kb/s", "Mb/s", "Gb/s" }; int units; double dlrate = calc_rate (bytes, secs, &units); /* Use more digits for smaller numbers (regardless of unit used), e.g. "1022", "247", "12.5", "2.38". */ snprintf (res, sizeof(res), "%.*f %s", dlrate >= 99.95 ? 0 : dlrate >= 9.995 ? 1 : 2, dlrate, !opt.report_bps ? rate_names[units]: rate_names_bits[units]); return res; } /* Calculate the download rate and trim it as appropriate for the speed. Appropriate means that if rate is greater than 1K/s, kilobytes are used, and if rate is greater than 1MB/s, megabytes are used. UNITS is zero for B/s, one for KB/s, two for MB/s, and three for GB/s. */ double calc_rate (wgint bytes, double secs, int *units) { double dlrate; double bibyte; if (!opt.report_bps) bibyte = 1024.0; else bibyte = 1000.0; if (secs == 0) /* If elapsed time is exactly zero, it means we're under the resolution of the timer. This can easily happen on systems that use time() for the timer. Since the interval lies between 0 and the timer's resolution, assume half the resolution. */ secs = ptimer_resolution () / 2.0; dlrate = secs ? convert_to_bits (bytes) / secs : 0; if (dlrate < bibyte) *units = 0; else if (dlrate < (bibyte * bibyte)) *units = 1, dlrate /= bibyte; else if (dlrate < (bibyte * bibyte * bibyte)) *units = 2, dlrate /= (bibyte * bibyte); else if (dlrate < (bibyte * bibyte * bibyte * bibyte)) *units = 3, dlrate /= (bibyte * bibyte * bibyte); else { *units = 4, dlrate /= (bibyte * bibyte * bibyte * bibyte); if (dlrate > 99.99) dlrate = 99.99; // upper limit 99.99TB/s } return dlrate; } #define SUSPEND_METHOD do { \ method_suspended = true; \ saved_body_data = opt.body_data; \ saved_body_file_name = opt.body_file; \ saved_method = opt.method; \ opt.body_data = NULL; \ opt.body_file = NULL; \ opt.method = NULL; \ } while (0) #define RESTORE_METHOD do { \ if (method_suspended) \ { \ opt.body_data = saved_body_data; \ opt.body_file = saved_body_file_name; \ opt.method = saved_method; \ method_suspended = false; \ } \ } while (0) static char *getproxy (struct url *); /* Retrieve the given URL. Decides which loop to call -- HTTP, FTP, FTP, proxy, etc. */ /* #### This function should be rewritten so it doesn't return from multiple points. */ uerr_t retrieve_url (struct url * orig_parsed, const char *origurl, char **file, char **newloc, const char *refurl, int *dt, bool recursive, struct iri *iri, bool register_status) { uerr_t result; char *url; bool location_changed; bool iri_fallbacked = 0; int dummy; char *mynewloc, *proxy; struct url *u = orig_parsed, *proxy_url; int up_error_code; /* url parse error code */ char *local_file = NULL; int redirection_count = 0; bool method_suspended = false; char *saved_body_data = NULL; char *saved_method = NULL; char *saved_body_file_name = NULL; /* If dt is NULL, use local storage. */ if (!dt) { dt = &dummy; dummy = 0; } url = xstrdup (origurl); if (newloc) *newloc = NULL; if (file) *file = NULL; if (!refurl) refurl = opt.referer; redirected: /* (also for IRI fallbacking) */ result = NOCONERROR; mynewloc = NULL; xfree(local_file); proxy_url = NULL; proxy = getproxy (u); if (proxy) { struct iri *pi = iri_new (); set_uri_encoding (pi, opt.locale, true); pi->utf8_encode = false; /* Parse the proxy URL. */ proxy_url = url_parse (proxy, &up_error_code, pi, true); if (!proxy_url) { char *error = url_error (proxy, up_error_code); logprintf (LOG_NOTQUIET, _("Error parsing proxy URL %s: %s.\n"), proxy, error); xfree (url); xfree (error); xfree (proxy); iri_free (pi); RESTORE_METHOD; result = PROXERR; if (orig_parsed != u) url_free (u); goto bail; } if (proxy_url->scheme != SCHEME_HTTP && proxy_url->scheme != u->scheme) { logprintf (LOG_NOTQUIET, _("Error in proxy URL %s: Must be HTTP.\n"), proxy); url_free (proxy_url); xfree (url); xfree (proxy); iri_free (pi); RESTORE_METHOD; result = PROXERR; if (orig_parsed != u) url_free (u); goto bail; } iri_free(pi); xfree (proxy); } if (u->scheme == SCHEME_HTTP #ifdef HAVE_SSL || u->scheme == SCHEME_HTTPS #endif || (proxy_url && proxy_url->scheme == SCHEME_HTTP)) { #ifdef HAVE_HSTS #ifdef TESTING /* we don't link against main.o when we're testing */ hsts_store_t hsts_store = NULL; #else extern hsts_store_t hsts_store; #endif if (opt.hsts && hsts_store) { if (hsts_match (hsts_store, u)) logprintf (LOG_VERBOSE, "URL transformed to HTTPS due to an HSTS policy\n"); } #endif result = http_loop (u, orig_parsed, &mynewloc, &local_file, refurl, dt, proxy_url, iri); } else if (u->scheme == SCHEME_FTP #ifdef HAVE_SSL || u->scheme == SCHEME_FTPS #endif ) { /* If this is a redirection, temporarily turn off opt.ftp_glob and opt.recursive, both being undesirable when following redirects. */ bool oldrec = recursive, glob = opt.ftp_glob; if (redirection_count) oldrec = glob = false; result = ftp_loop (u, orig_parsed, &local_file, dt, proxy_url, recursive, glob); recursive = oldrec; /* There is a possibility of having HTTP being redirected to FTP. In these cases we must decide whether the text is HTML according to the suffix. The HTML suffixes are `.html', `.htm' and a few others, case-insensitive. */ if (redirection_count && local_file && (u->scheme == SCHEME_FTP #ifdef HAVE_SSL || u->scheme == SCHEME_FTPS #endif )) { if (has_html_suffix_p (local_file)) *dt |= TEXTHTML; } } if (proxy_url) { url_free (proxy_url); proxy_url = NULL; } location_changed = (result == NEWLOCATION || result == NEWLOCATION_KEEP_POST); if (location_changed) { char *construced_newloc; struct url *newloc_parsed; assert (mynewloc != NULL); xfree (local_file); /* The HTTP specs only allow absolute URLs to appear in redirects, but a ton of boneheaded webservers and CGIs out there break the rules and use relative URLs, and popular browsers are lenient about this, so wget should be too. */ construced_newloc = uri_merge (url, mynewloc ? mynewloc : ""); xfree (mynewloc); mynewloc = construced_newloc; #ifdef ENABLE_IRI /* Reset UTF-8 encoding state, set the URI encoding and reset the content encoding. */ iri->utf8_encode = opt.enable_iri; if (opt.encoding_remote) set_uri_encoding (iri, opt.encoding_remote, true); set_content_encoding (iri, NULL); xfree (iri->orig_url); #endif /* Now, see if this new location makes sense. */ newloc_parsed = url_parse (mynewloc, &up_error_code, iri, true); if (!newloc_parsed) { char *error = url_error (mynewloc, up_error_code); logprintf (LOG_NOTQUIET, "%s: %s.\n", escnonprint_uri (mynewloc), error); if (orig_parsed != u) { url_free (u); } xfree (url); xfree (mynewloc); xfree (error); RESTORE_METHOD; goto bail; } /* Now mynewloc will become newloc_parsed->url, because if the Location contained relative paths like .././something, we don't want that propagating as url. */ xfree (mynewloc); mynewloc = xstrdup (newloc_parsed->url); /* Check for max. number of redirections. */ if (++redirection_count > opt.max_redirect) { logprintf (LOG_NOTQUIET, _("%d redirections exceeded.\n"), opt.max_redirect); url_free (newloc_parsed); if (orig_parsed != u) { url_free (u); } xfree (url); xfree (mynewloc); RESTORE_METHOD; result = WRONGCODE; goto bail; } xfree (url); url = mynewloc; if (orig_parsed != u) { url_free (u); } u = newloc_parsed; /* If we're being redirected from POST, and we received a redirect code different than 307, we don't want to POST again. Many requests answer POST with a redirection to an index page; that redirection is clearly a GET. We "suspend" POST data for the duration of the redirections, and restore it when we're done. RFC2616 HTTP/1.1 introduces code 307 Temporary Redirect specifically to preserve the method of the request. */ if (result != NEWLOCATION_KEEP_POST && !method_suspended) SUSPEND_METHOD; goto redirected; } else { xfree(mynewloc); } /* Try to not encode in UTF-8 if fetching failed */ if (!(*dt & RETROKF) && iri->utf8_encode) { iri->utf8_encode = false; if (orig_parsed != u) { url_free (u); } u = url_parse (origurl, NULL, iri, true); if (u) { if (strcmp(u->url, orig_parsed->url)) { DEBUGP (("[IRI fallbacking to non-utf8 for %s\n", quote (url))); xfree (url); url = xstrdup (u->url); iri_fallbacked = 1; goto redirected; } else DEBUGP (("[Needn't fallback to non-utf8 for %s\n", quote (url))); } else DEBUGP (("[Couldn't fallback to non-utf8 for %s\n", quote (url))); } if (local_file && u && (*dt & RETROKF || opt.content_on_error)) { register_download (u->url, local_file); if (!opt.spider && redirection_count && 0 != strcmp (origurl, u->url)) register_redirection (origurl, u->url); if (*dt & TEXTHTML) register_html (local_file); if (*dt & TEXTCSS) register_css (local_file); } if (file) *file = local_file ? local_file : NULL; else xfree (local_file); if (orig_parsed != u) url_free (u); if (redirection_count || iri_fallbacked) { if (newloc) *newloc = url; else xfree (url); } else { if (newloc) *newloc = NULL; xfree (url); } RESTORE_METHOD; bail: if (register_status) inform_exit_status (result); return result; } /* Find the URLs in the file and call retrieve_url() for each of them. If HTML is true, treat the file as HTML, and construct the URLs accordingly. If opt.recursive is set, call retrieve_tree() for each file. */ uerr_t retrieve_from_file (const char *file, bool html, int *count) { uerr_t status; struct urlpos *url_list, *cur_url; struct iri *iri = iri_new(); char *input_file, *url_file = NULL; const char *url = file; status = RETROK; /* Suppose everything is OK. */ *count = 0; /* Reset the URL count. */ /* sXXXav : Assume filename and links in the file are in the locale */ set_uri_encoding (iri, opt.locale, true); set_content_encoding (iri, opt.locale); if (url_valid_scheme (url)) { int dt,url_err; struct url *url_parsed = url_parse (url, &url_err, iri, true); if (!url_parsed) { char *error = url_error (url, url_err); logprintf (LOG_NOTQUIET, "%s: %s.\n", url, error); xfree (error); iri_free (iri); return URLERROR; } if (!opt.base_href) opt.base_href = xstrdup (url); status = retrieve_url (url_parsed, url, &url_file, NULL, NULL, &dt, false, iri, true); url_free (url_parsed); if (!url_file || (status != RETROK)) return status; if (dt & TEXTHTML) html = true; #ifdef ENABLE_IRI /* If we have a found a content encoding, use it. * ( == is okay, because we're checking for identical object) */ if (iri->content_encoding != opt.locale) set_uri_encoding (iri, iri->content_encoding, false); #endif /* Reset UTF-8 encode status */ iri->utf8_encode = opt.enable_iri; xfree (iri->orig_url); input_file = url_file; } else input_file = (char *) file; url_list = (html ? get_urls_html (input_file, NULL, NULL, iri) : get_urls_file (input_file)); xfree (url_file); for (cur_url = url_list; cur_url; cur_url = cur_url->next, ++*count) { char *filename = NULL, *new_file = NULL, *proxy; int dt = 0; struct iri *tmpiri = iri_dup (iri); struct url *parsed_url = NULL; if (cur_url->ignore_when_downloading) continue; if (opt.quota && total_downloaded_bytes > opt.quota) { status = QUOTEXC; break; } parsed_url = url_parse (cur_url->url->url, NULL, tmpiri, true); proxy = getproxy (cur_url->url); if ((opt.recursive || opt.page_requisites) && ((cur_url->url->scheme != SCHEME_FTP #ifdef HAVE_SSL && cur_url->url->scheme != SCHEME_FTPS #endif ) || proxy)) { int old_follow_ftp = opt.follow_ftp; /* Turn opt.follow_ftp on in case of recursive FTP retrieval */ if (cur_url->url->scheme == SCHEME_FTP #ifdef HAVE_SSL || cur_url->url->scheme == SCHEME_FTPS #endif ) opt.follow_ftp = 1; status = retrieve_tree (parsed_url ? parsed_url : cur_url->url, tmpiri); opt.follow_ftp = old_follow_ftp; } else status = retrieve_url (parsed_url ? parsed_url : cur_url->url, cur_url->url->url, &filename, &new_file, NULL, &dt, opt.recursive, tmpiri, true); xfree (proxy); if (parsed_url) url_free (parsed_url); if (filename && opt.delete_after && file_exists_p (filename, NULL)) { DEBUGP (("\ Removing file due to --delete-after in retrieve_from_file():\n")); logprintf (LOG_VERBOSE, _("Removing %s.\n"), filename); if (unlink (filename)) logprintf (LOG_NOTQUIET, "Failed to unlink %s: (%d) %s\n", filename, errno, strerror (errno)); dt &= ~RETROKF; } xfree (new_file); xfree (filename); iri_free (tmpiri); } /* Free the linked list of URL-s. */ free_urlpos (url_list); iri_free (iri); return status; } /* Print `giving up', or `retrying', depending on the impending action. N1 and N2 are the attempt number and the attempt limit. */ void printwhat (int n1, int n2) { logputs (LOG_VERBOSE, (n1 == n2) ? _("Giving up.\n\n") : _("Retrying.\n\n")); } /* If opt.wait or opt.waitretry are specified, and if certain conditions are met, sleep the appropriate number of seconds. See the documentation of --wait and --waitretry for more information. COUNT is the count of current retrieval, beginning with 1. */ void sleep_between_retrievals (int count) { static bool first_retrieval = true; if (first_retrieval) { /* Don't sleep before the very first retrieval. */ first_retrieval = false; return; } if (opt.waitretry && count > 1) { /* If opt.waitretry is specified and this is a retry, wait for COUNT-1 number of seconds, or for opt.waitretry seconds. */ if (count <= opt.waitretry) xsleep (count - 1); else xsleep (opt.waitretry); } else if (opt.wait) { if (!opt.random_wait || count > 1) /* If random-wait is not specified, or if we are sleeping between retries of the same download, sleep the fixed interval. */ xsleep (opt.wait); else { /* Sleep a random amount of time averaging in opt.wait seconds. The sleeping amount ranges from 0.5*opt.wait to 1.5*opt.wait. */ double waitsecs = (0.5 + random_float ()) * opt.wait; DEBUGP (("sleep_between_retrievals: avg=%f,sleep=%f\n", opt.wait, waitsecs)); xsleep (waitsecs); } } } /* Free the linked list of urlpos. */ void free_urlpos (struct urlpos *l) { while (l) { struct urlpos *next = l->next; if (l->url) url_free (l->url); xfree (l->local_name); xfree (l); l = next; } } /* Rotate FNAME opt.backups times */ void rotate_backups(const char *fname) { #ifdef __VMS # define SEP "_" # define AVS ";*" /* All-version suffix. */ # define AVSL (sizeof (AVS) - 1) #else # define SEP "." # define AVSL 0 #endif #define FILE_BUF_SIZE 1024 /* avoid alloca() here */ char from[FILE_BUF_SIZE], to[FILE_BUF_SIZE]; struct stat sb; bool overflow; int i; if (stat (fname, &sb) == 0) if (S_ISREG (sb.st_mode) == 0) return; for (i = opt.backups; i > 1; i--) { #ifdef VMS /* Delete (all versions of) any existing max-suffix file, to avoid * creating multiple versions of it. (On VMS, rename() will * create a new version of an existing destination file, not * destroy/overwrite it.) */ if (i == opt.backups) { if (((unsigned) snprintf (to, sizeof (to), "%s%s%d%s", fname, SEP, i, AVS)) >= sizeof (to)) logprintf (LOG_NOTQUIET, "Failed to delete %s: File name truncation\n", to); else delete (to); } #endif overflow = (unsigned) snprintf (to, FILE_BUF_SIZE, "%s%s%d", fname, SEP, i) >= FILE_BUF_SIZE; overflow |= (unsigned) snprintf (from, FILE_BUF_SIZE, "%s%s%d", fname, SEP, i - 1) >= FILE_BUF_SIZE; if (overflow) errno = ENAMETOOLONG; if (overflow || rename (from, to)) logprintf (LOG_NOTQUIET, "Failed to rename %s to %s: (%d) %s\n", from, to, errno, strerror (errno)); } overflow = (unsigned) snprintf (to, FILE_BUF_SIZE, "%s%s%d", fname, SEP, 1) >= FILE_BUF_SIZE; if (overflow) errno = ENAMETOOLONG; if (overflow || rename(fname, to)) logprintf (LOG_NOTQUIET, "Failed to rename %s to %s: (%d) %s\n", fname, to, errno, strerror (errno)); #undef FILE_BUF_SIZE } static bool no_proxy_match (const char *, const char **); /* Return the URL of the proxy appropriate for url U. */ static char * getproxy (struct url *u) { char *proxy = NULL; char *rewritten_url; if (!opt.use_proxy) return NULL; if (no_proxy_match (u->host, (const char **)opt.no_proxy)) return NULL; switch (u->scheme) { case SCHEME_HTTP: proxy = opt.http_proxy ? opt.http_proxy : getenv ("http_proxy"); break; #ifdef HAVE_SSL case SCHEME_HTTPS: proxy = opt.https_proxy ? opt.https_proxy : getenv ("https_proxy"); break; case SCHEME_FTPS: proxy = opt.ftp_proxy ? opt.ftp_proxy : getenv ("ftps_proxy"); break; #endif case SCHEME_FTP: proxy = opt.ftp_proxy ? opt.ftp_proxy : getenv ("ftp_proxy"); break; case SCHEME_INVALID: break; } if (!proxy || !*proxy) return NULL; /* Handle shorthands. `rewritten_storage' is a kludge to allow getproxy() to return static storage. */ rewritten_url = rewrite_shorthand_url (proxy); if (rewritten_url) return rewritten_url; return strdup(proxy); } /* Returns true if URL would be downloaded through a proxy. */ bool url_uses_proxy (struct url * u) { bool ret; char *proxy; if (!u) return false; proxy = getproxy (u); ret = proxy != NULL; xfree (proxy); return ret; } /* Should a host be accessed through proxy, concerning no_proxy? */ static bool no_proxy_match (const char *host, const char **no_proxy) { if (!no_proxy) return false; else return sufmatch (no_proxy, host); } /* Set the file parameter to point to the local file string. */ void set_local_file (const char **file, const char *default_file) { if (opt.output_document) { if (output_stream_regular) *file = opt.output_document; } else *file = default_file; } /* Return true for an input file's own URL, false otherwise. */ bool input_file_url (const char *input_file) { static bool first = true; if (input_file && url_has_scheme (input_file) && first) { first = false; return true; } else return false; } wget-1.21.2/src/metalink.c0000644000000000000000000015051614115732710012202 00000000000000/* Metalink module. Copyright (C) 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #include "wget.h" #ifdef HAVE_METALINK #include "metalink.h" #include "retr.h" #include "exits.h" #include "utils.h" #include "md2.h" #include "md4.h" #include "md5.h" #include "sha1.h" #include "sha256.h" #include "sha512.h" #include "filename.h" #include "xmemdup0.h" #include "xstrndup.h" #include "c-strcase.h" #include #include /* For unlink. */ #include #ifdef HAVE_GPGME #include #include /* For open and close. */ #endif #ifdef TESTING #include "../tests/unit-tests.h" #endif /* Loop through all files in metalink structure and retrieve them. Returns RETROK if all files were downloaded. Returns last retrieval error (from retrieve_url) if some files could not be downloaded. */ uerr_t retrieve_from_metalink (const metalink_t* metalink) { metalink_file_t **mfile_ptr; uerr_t last_retr_err = RETROK; /* Store last encountered retrieve error. */ FILE *_output_stream = output_stream; bool _output_stream_regular = output_stream_regular; char *_output_document = opt.output_document; /* metalink file counter */ unsigned mfc = 0; /* metalink retrieval type */ const char *metatpy = metalink->origin ? "Metalink/HTTP" : "Metalink/XML"; /* metalink mother source */ char *metasrc = metalink->origin ? metalink->origin : opt.input_metalink; DEBUGP (("Retrieving from Metalink %s\n", quote (metasrc))); /* No files to download. */ if (!metalink->files) return RETROK; if (opt.output_document) { /* We cannot support output_document as we need to compute checksum of downloaded file, and to remove it if the checksum is bad. */ logputs (LOG_NOTQUIET, _("-O not supported for metalink download. Ignoring.\n")); } for (mfile_ptr = metalink->files; *mfile_ptr; mfile_ptr++) { metalink_file_t *mfile = *mfile_ptr; metalink_resource_t **mres_ptr; char *planname = NULL; char *trsrname = NULL; char *filename; char *basename; char *safename = NULL; char *destname = NULL; bool size_ok = false; bool hash_ok = false; uerr_t retr_err = METALINK_MISSING_RESOURCE; /* -1 -> file should be rejected 0 -> could not verify 1 -> verified successfully */ char sig_status = 0; bool skip_mfile = false; output_stream = NULL; mfc++; /* The directory prefix for opt.metalink_over_http is handled by src/url.c (url_file_name), do not add it a second time. */ if (!metalink->origin && opt.dir_prefix && strlen (opt.dir_prefix)) planname = aprintf ("%s/%s", opt.dir_prefix, mfile->name); else planname = xstrdup (mfile->name); /* With Metalink/HTTP, trust the metalink file name (from cli). With --trust-server-names, trust the Metalink/XML file name, otherwise, use the basename of --input-metalink followed by the metalink file counter as suffix. */ if (metalink->origin || opt.trustservernames) { trsrname = xstrdup (mfile->name); } else { trsrname = xstrdup (get_metalink_basename (opt.input_metalink)); append_suffix_number (&trsrname, ".#", mfc); } /* Add the directory prefix for opt.input_metalink. */ if (!metalink->origin && opt.dir_prefix && strlen (opt.dir_prefix)) filename = aprintf ("%s/%s", opt.dir_prefix, trsrname); else filename = xstrdup (trsrname); /* Enforce libmetalink's metalink_check_safe_path(). */ basename = get_metalink_basename (filename); safename = metalink_check_safe_path (filename) ? filename : basename; DEBUGP (("Processing metalink file %s...\n", quote (mfile->name))); DEBUGP (("\n")); DEBUGP ((" %s\n", metatpy)); DEBUGP (("\n")); DEBUGP ((" --trust-server-names %s\n", opt.trustservernames ? "true" : "false")); DEBUGP ((" --directory-prefix %s\n", quote (opt.dir_prefix ? opt.dir_prefix : ""))); DEBUGP (("\n")); DEBUGP ((" Counted metalink file %u\n", mfc)); DEBUGP ((" Planned metalink file %s\n", quote (planname ? planname : ""))); DEBUGP ((" Trusted metalink file %s\n", quote (trsrname ? trsrname : ""))); DEBUGP ((" Current metalink file %s\n", quote (filename ? filename : ""))); DEBUGP ((" Cleaned metalink file %s\n", quote (basename ? basename : ""))); DEBUGP ((" Secured metalink file %s\n", quote (safename ? safename : ""))); DEBUGP (("\n")); /* Verify if the planned metalink file name is safe. */ if (!safename || strcmp (planname, safename)) { logprintf (LOG_NOTQUIET, _("[--trust-server-names %s, --directory-prefix=%s]\n"), (opt.trustservernames ? "true" : "false"), quote (opt.dir_prefix ? opt.dir_prefix : "")); logprintf (LOG_NOTQUIET, _("Planned metalink file: %s\n"), quote (planname ? planname : "")); logprintf (LOG_NOTQUIET, _("Secured metalink file: %s\n"), quote (safename ? safename : "")); if (!safename) { logprintf (LOG_NOTQUIET, _("Rejecting metalink file. Unsafe name.\n")); xfree (planname); xfree (trsrname); xfree (filename); continue; } } /* Process the chosen application/metalink4+xml metaurl. */ if (opt.metalink_index >= 0) { int _metalink_index = opt.metalink_index; metalink_metaurl_t **murl_ptr; int abs_count = 0, meta_count = 0; uerr_t x_retr_err = METALINK_MISSING_RESOURCE; opt.metalink_index = -1; DEBUGP (("Searching application/metalink4+xml ordinal number %d...\n", _metalink_index)); if (mfile->metaurls && mfile->metaurls[0]) for (murl_ptr = mfile->metaurls; *murl_ptr; murl_ptr++) { metalink_t* metaurl_xml; metalink_error_t meta_err; metalink_metaurl_t *murl = *murl_ptr; char *_dir_prefix = opt.dir_prefix; char *_input_metalink = opt.input_metalink; char *metafile = NULL; char *metadest = NULL; char *metadir = NULL; abs_count++; if (strcmp (murl->mediatype, "application/metalink4+xml")) continue; meta_count++; DEBUGP ((" Ordinal number %d: %s\n", meta_count, quote (murl->url))); if (_metalink_index > 0) { if (meta_count < _metalink_index) continue; else if (meta_count > _metalink_index) break; } logprintf (LOG_NOTQUIET, _("Processing metaurl %s...\n"), quote (murl->url)); /* Metalink/XML download file name. */ metafile = xstrdup (safename); if (opt.trustservernames) replace_metalink_basename (&metafile, murl->name ? murl->name : murl->url); else append_suffix_number (&metafile, ".meta#", meta_count); if (!metalink_check_safe_path (metafile)) { logprintf (LOG_NOTQUIET, _("Rejecting metaurl file %s. Unsafe name.\n"), quote (metafile)); xfree (metafile); if (_metalink_index > 0) break; continue; } /* For security reasons, always save metalink metaurl files as new unique files. Keep them on failure. */ x_retr_err = fetch_metalink_file (murl->url, false, false, metafile, &metadest); /* On failure, try the next metalink metaurl. */ if (x_retr_err != RETROK) { logprintf (LOG_VERBOSE, _("Failed to download %s. Skipping metaurl.\n"), quote (metadest ? metadest : metafile)); inform_exit_status (x_retr_err); xfree (metadest); xfree (metafile); if (_metalink_index > 0) break; continue; } /* Parse Metalink/XML. */ meta_err = metalink_parse_file (metadest, &metaurl_xml); /* On failure, try the next metalink metaurl. */ if (meta_err) { logprintf (LOG_NOTQUIET, _("Unable to parse metaurl file %s.\n"), quote (metadest)); x_retr_err = METALINK_PARSE_ERROR; inform_exit_status (x_retr_err); xfree (metadest); xfree (metafile); if (_metalink_index > 0) break; continue; } /* We need to sort the resources if preferred location was specified by the user. */ if (opt.preferred_location && opt.preferred_location[0]) { metalink_file_t **x_mfile_ptr; for (x_mfile_ptr = metaurl_xml->files; *x_mfile_ptr; x_mfile_ptr++) { metalink_resource_t **x_mres_ptr; metalink_file_t *x_mfile = *x_mfile_ptr; size_t mres_count = 0; for (x_mres_ptr = x_mfile->resources; *x_mres_ptr; x_mres_ptr++) mres_count++; stable_sort (x_mfile->resources, mres_count, sizeof (metalink_resource_t *), metalink_res_cmp); } } /* Insert the current "Directory Options". */ if (metalink->origin) { /* WARNING: Do not use lib/dirname.c (dir_name) to get the directory name, it may append a dot '.' character to the directory name. */ metadir = xstrdup (planname); replace_metalink_basename (&metadir, NULL); } else { metadir = xstrdup (opt.dir_prefix); } opt.dir_prefix = metadir; opt.input_metalink = metadest; x_retr_err = retrieve_from_metalink (metaurl_xml); if (x_retr_err != RETROK) logprintf (LOG_NOTQUIET, _("Could not download all resources from %s.\n"), quote (metadest)); metalink_delete (metaurl_xml); metaurl_xml = NULL; opt.input_metalink = _input_metalink; opt.dir_prefix = _dir_prefix; xfree (metadir); xfree (metadest); xfree (metafile); break; } if (x_retr_err != RETROK) logprintf (LOG_NOTQUIET, _("Metaurls processing returned with error.\n")); xfree (destname); xfree (filename); xfree (trsrname); xfree (planname); opt.output_document = _output_document; output_stream_regular = _output_stream_regular; output_stream = _output_stream; opt.metalink_index = _metalink_index; return x_retr_err; } /* Resources are sorted by priority. */ for (mres_ptr = mfile->resources; *mres_ptr && mfile->checksums && !skip_mfile; mres_ptr++) { metalink_resource_t *mres = *mres_ptr; metalink_checksum_t **mchksum_ptr, *mchksum; struct iri *iri; struct url *url; file_stats_t flstats; int url_err; clean_metalink_string (&mres->url); if (!RES_TYPE_SUPPORTED (mres->type)) { logprintf (LOG_VERBOSE, _("Resource type %s not supported, ignoring...\n"), quote (mres->type)); continue; } /* The file is fully downloaded, but some problems were encountered (checksum failure?). The loop had been continued to switch to the next url. */ if (output_stream && retr_err == RETROK) { /* Do not rename/remove a continued file. Skip it. */ if (opt.always_rest) { skip_mfile = true; continue; } fclose (output_stream); output_stream = NULL; badhash_or_remove (destname); xfree (destname); } else if (!output_stream && destname) { xfree (destname); } retr_err = METALINK_RETR_ERROR; /* Parse our resource URL. */ iri = iri_new (); set_uri_encoding (iri, opt.locale, true); url = url_parse (mres->url, &url_err, iri, false); if (!url) { char *error = url_error (mres->url, url_err); logprintf (LOG_NOTQUIET, "%s: %s.\n", mres->url, error); xfree (error); inform_exit_status (URLERROR); iri_free (iri); continue; } else { /* Avoid recursive Metalink from HTTP headers. */ bool _metalink_http = opt.metalink_over_http; /* If output_stream is not NULL, then we have failed on previous resource and are retrying. Thus, continue with the next resource. Do not close output_stream while iterating over the resources, or the download progress will be lost. */ if (output_stream) { DEBUGP (("Previous resource failed, continue with next resource.\n")); } else { /* Assure proper local file name regardless of the URL of particular Metalink resource. To do that we create the local file here and put it as output_stream. We restore the original configuration after we are finished with the file. */ if (opt.always_rest) /* continue previous download */ output_stream = fopen (safename, "ab"); else /* create a file with an unique name */ output_stream = unique_create (safename, true, &destname); } output_stream_regular = true; /* At this point, if output_stream is NULL, the file couldn't be created/opened. This happens when the metalink:file has a "path/file" name format and its directory tree cannot be created: * stdio.h (fopen) * src/utils.c (unique_create) RFC5854 requires a proper "path/file" format handling, this can be achieved setting opt.output_document while output_stream is left to NULL: * src/http.c (open_output_stream): If output_stream is NULL, create the opt.output_document "path/file" */ if (!destname) destname = xstrdup (safename); /* Store the real file name for displaying in messages, and for proper RFC5854 "path/file" handling. */ opt.output_document = destname; opt.metalink_over_http = false; DEBUGP (("Storing to %s\n", destname)); retr_err = retrieve_url (url, mres->url, NULL, NULL, NULL, NULL, opt.recursive, iri, false); opt.metalink_over_http = _metalink_http; /* Bug: output_stream is NULL, but retrieve_url() somehow created destname. Bugfix: point output_stream to destname if it exists. */ memset(&flstats, 0, sizeof(flstats)); if (!output_stream && file_exists_p (destname, &flstats)) output_stream = fopen_stat (destname, "ab", &flstats); } url_free (url); iri_free (iri); if (retr_err == RETROK) { FILE *local_file; /* Check the digest. */ local_file = fopen (destname, "rb"); if (!local_file) { logprintf (LOG_NOTQUIET, _("Could not open downloaded file.\n")); continue; } size_ok = false; logprintf (LOG_VERBOSE, _("Computing size for %s\n"), quote (destname)); if (!mfile->size) { size_ok = true; logprintf (LOG_VERBOSE, _("File size not declared. Skipping check.\n")); } else { wgint local_file_size = file_size (destname); if (local_file_size == -1) { logprintf (LOG_NOTQUIET, _("Could not get downloaded file's size.\n")); fclose (local_file); local_file = NULL; continue; } /* FIXME: what about int64? */ DEBUGP (("Declared size: %lld\n", mfile->size)); DEBUGP (("Computed size: %lld\n", (long long) local_file_size)); if (local_file_size != (wgint) mfile->size) { logprintf (LOG_NOTQUIET, _("Size mismatch for file %s.\n"), quote (destname)); fclose (local_file); local_file = NULL; continue; } else { size_ok = true; logputs (LOG_VERBOSE, _("Size matches.\n")); } } for (mchksum_ptr = mfile->checksums; *mchksum_ptr; mchksum_ptr++) { char md2[MD2_DIGEST_SIZE]; char md2_txt[2 * MD2_DIGEST_SIZE + 1]; char md4[MD4_DIGEST_SIZE]; char md4_txt[2 * MD4_DIGEST_SIZE + 1]; char md5[MD5_DIGEST_SIZE]; char md5_txt[2 * MD5_DIGEST_SIZE + 1]; char sha1[SHA1_DIGEST_SIZE]; char sha1_txt[2 * SHA1_DIGEST_SIZE + 1]; char sha224[SHA224_DIGEST_SIZE]; char sha224_txt[2 * SHA224_DIGEST_SIZE + 1]; char sha256[SHA256_DIGEST_SIZE]; char sha256_txt[2 * SHA256_DIGEST_SIZE + 1]; char sha384[SHA384_DIGEST_SIZE]; char sha384_txt[2 * SHA384_DIGEST_SIZE + 1]; char sha512[SHA512_DIGEST_SIZE]; char sha512_txt[2 * SHA512_DIGEST_SIZE + 1]; hash_ok = false; mchksum = *mchksum_ptr; /* I have seen both variants... */ if (c_strcasecmp (mchksum->type, "md2") && c_strcasecmp (mchksum->type, "md4") && c_strcasecmp (mchksum->type, "md5") && c_strcasecmp (mchksum->type, "sha1") && c_strcasecmp (mchksum->type, "sha-1") && c_strcasecmp (mchksum->type, "sha224") && c_strcasecmp (mchksum->type, "sha-224") && c_strcasecmp (mchksum->type, "sha256") && c_strcasecmp (mchksum->type, "sha-256") && c_strcasecmp (mchksum->type, "sha384") && c_strcasecmp (mchksum->type, "sha-384") && c_strcasecmp (mchksum->type, "sha512") && c_strcasecmp (mchksum->type, "sha-512")) { DEBUGP (("Ignoring unsupported checksum type %s.\n", quote (mchksum->type))); continue; } logprintf (LOG_VERBOSE, _("Computing checksum for %s\n"), quote (destname)); DEBUGP (("Declared hash: %s\n", mchksum->hash)); if (c_strcasecmp (mchksum->type, "md2") == 0) { md2_stream (local_file, md2); wg_hex_to_string (md2_txt, md2, MD2_DIGEST_SIZE); DEBUGP (("Computed hash: %s\n", md2_txt)); if (!strcmp (md2_txt, mchksum->hash)) hash_ok = true; } else if (c_strcasecmp (mchksum->type, "md4") == 0) { md4_stream (local_file, md4); wg_hex_to_string (md4_txt, md4, MD4_DIGEST_SIZE); DEBUGP (("Computed hash: %s\n", md4_txt)); if (!strcmp (md4_txt, mchksum->hash)) hash_ok = true; } else if (c_strcasecmp (mchksum->type, "md5") == 0) { md5_stream (local_file, md5); wg_hex_to_string (md5_txt, md5, MD5_DIGEST_SIZE); DEBUGP (("Computed hash: %s\n", md5_txt)); if (!strcmp (md5_txt, mchksum->hash)) hash_ok = true; } else if (c_strcasecmp (mchksum->type, "sha1") == 0 || c_strcasecmp (mchksum->type, "sha-1") == 0) { sha1_stream (local_file, sha1); wg_hex_to_string (sha1_txt, sha1, SHA1_DIGEST_SIZE); DEBUGP (("Computed hash: %s\n", sha1_txt)); if (!strcmp (sha1_txt, mchksum->hash)) hash_ok = true; } else if (c_strcasecmp (mchksum->type, "sha224") == 0 || c_strcasecmp (mchksum->type, "sha-224") == 0) { sha224_stream (local_file, sha224); wg_hex_to_string (sha224_txt, sha224, SHA224_DIGEST_SIZE); DEBUGP (("Computed hash: %s\n", sha224_txt)); if (!strcmp (sha224_txt, mchksum->hash)) hash_ok = true; } else if (c_strcasecmp (mchksum->type, "sha256") == 0 || c_strcasecmp (mchksum->type, "sha-256") == 0) { sha256_stream (local_file, sha256); wg_hex_to_string (sha256_txt, sha256, SHA256_DIGEST_SIZE); DEBUGP (("Computed hash: %s\n", sha256_txt)); if (!strcmp (sha256_txt, mchksum->hash)) hash_ok = true; } else if (c_strcasecmp (mchksum->type, "sha384") == 0 || c_strcasecmp (mchksum->type, "sha-384") == 0) { sha384_stream (local_file, sha384); wg_hex_to_string (sha384_txt, sha384, SHA384_DIGEST_SIZE); DEBUGP (("Computed hash: %s\n", sha384_txt)); if (!strcmp (sha384_txt, mchksum->hash)) hash_ok = true; } else if (c_strcasecmp (mchksum->type, "sha512") == 0 || c_strcasecmp (mchksum->type, "sha-512") == 0) { sha512_stream (local_file, sha512); wg_hex_to_string (sha512_txt, sha512, SHA512_DIGEST_SIZE); DEBUGP (("Computed hash: %s\n", sha512_txt)); if (!strcmp (sha512_txt, mchksum->hash)) hash_ok = true; } if (hash_ok) { logputs (LOG_VERBOSE, _("Checksum matches.\n")); } else { logprintf (LOG_NOTQUIET, _("Checksum mismatch for file %s.\n"), quote (destname)); } /* Stop as soon as we checked the supported checksum. */ break; } /* Iterate over available checksums. */ fclose (local_file); local_file = NULL; if (!hash_ok) continue; sig_status = 0; /* Not verified. */ #ifdef HAVE_GPGME /* Check the crypto signature. Note that the signatures from Metalink in XML will not be parsed when using libmetalink version older than 0.1.3. Metalink-over-HTTP is not affected by this problem. */ if (mfile->signature) { metalink_signature_t *msig = mfile->signature; gpgme_error_t gpgerr; gpgme_ctx_t gpgctx; gpgme_data_t gpgsigdata, gpgdata; gpgme_verify_result_t gpgres; gpgme_signature_t gpgsig; int fd; /* Initialize the library - as name suggests. */ gpgme_check_version (NULL); /* Open data file. */ fd = open (destname, O_RDONLY); if (fd == -1) { logputs (LOG_NOTQUIET, _("Could not open downloaded file for signature " "verification.\n")); goto gpg_skip_verification; } /* Assign file descriptor to GPG data structure. */ gpgerr = gpgme_data_new_from_fd (&gpgdata, fd); if (gpgerr != GPG_ERR_NO_ERROR) { logprintf (LOG_NOTQUIET, "GPGME data_new_from_fd: %s\n", gpgme_strerror (gpgerr)); goto gpg_skip_verification; } /* Prepare new GPGME context. */ gpgerr = gpgme_new (&gpgctx); if (gpgerr != GPG_ERR_NO_ERROR) { logprintf (LOG_NOTQUIET, "GPGME new: %s\n", gpgme_strerror (gpgerr)); gpgme_data_release (gpgdata); goto gpg_skip_verification; } DEBUGP (("Verifying signature %s:\n%s\n", quote (msig->mediatype), msig->signature)); /* Check signature type. */ if (strcmp (msig->mediatype, "application/pgp-signature")) { /* Unsupported signature type. */ gpgme_release (gpgctx); gpgme_data_release (gpgdata); goto gpg_skip_verification; } gpgerr = gpgme_set_protocol (gpgctx, GPGME_PROTOCOL_OpenPGP); if (gpgerr != GPG_ERR_NO_ERROR) { logprintf (LOG_NOTQUIET, "GPGME set_protocol: %s\n", gpgme_strerror (gpgerr)); gpgme_release (gpgctx); gpgme_data_release (gpgdata); goto gpg_skip_verification; } /* Load the signature. */ gpgerr = gpgme_data_new_from_mem (&gpgsigdata, msig->signature, strlen (msig->signature), 0); if (gpgerr != GPG_ERR_NO_ERROR) { logprintf (LOG_NOTQUIET, _("GPGME data_new_from_mem: %s\n"), gpgme_strerror (gpgerr)); gpgme_release (gpgctx); gpgme_data_release (gpgdata); goto gpg_skip_verification; } /* Verify the signature. */ gpgerr = gpgme_op_verify (gpgctx, gpgsigdata, gpgdata, NULL); if (gpgerr != GPG_ERR_NO_ERROR) { logprintf (LOG_NOTQUIET, _("GPGME op_verify: %s\n"), gpgme_strerror (gpgerr)); gpgme_data_release (gpgsigdata); gpgme_release (gpgctx); gpgme_data_release (gpgdata); goto gpg_skip_verification; } /* Check the results. */ gpgres = gpgme_op_verify_result (gpgctx); if (!gpgres) { logputs (LOG_NOTQUIET, _("GPGME op_verify_result: NULL\n")); gpgme_data_release (gpgsigdata); gpgme_release (gpgctx); gpgme_data_release (gpgdata); goto gpg_skip_verification; } /* The list is null-terminated. */ for (gpgsig = gpgres->signatures; gpgsig; gpgsig = gpgsig->next) { DEBUGP (("Checking signature %s\n", gpgsig->fpr)); if (gpgsig->summary & (GPGME_SIGSUM_VALID | GPGME_SIGSUM_GREEN)) { logputs (LOG_VERBOSE, _("Signature validation succeeded.\n")); sig_status = 1; break; } if (gpgsig->summary & GPGME_SIGSUM_RED) { logputs (LOG_NOTQUIET, _("Invalid signature. Rejecting resource.\n")); sig_status = -1; break; } if (gpgsig->summary == 0 && (gpgsig->status & 0xFFFF) == GPG_ERR_NO_ERROR) { logputs (LOG_VERBOSE, _("Data matches signature, but signature " "is not trusted.\n")); } if ((gpgsig->status & 0xFFFF) != GPG_ERR_NO_ERROR) { logprintf (LOG_NOTQUIET, "GPGME: %s\n", gpgme_strerror (gpgsig->status & 0xFFFF)); } } gpgme_data_release (gpgsigdata); gpgme_release (gpgctx); gpgme_data_release (gpgdata); gpg_skip_verification: if (fd != -1) close (fd); } /* endif (mfile->signature) */ #endif /* Stop if file was downloaded with success. */ if (sig_status >= 0) break; } /* endif RETR_OK. */ } /* Iterate over resources. */ if (!mfile->checksums) { logprintf (LOG_NOTQUIET, _("No checksums found.\n")); retr_err = METALINK_CHKSUM_ERROR; } if (retr_err != RETROK) { logprintf (LOG_VERBOSE, _("Failed to download %s. Skipping resource.\n"), quote (destname ? destname : safename)); } else if (!size_ok) { retr_err = METALINK_SIZE_ERROR; logprintf (LOG_NOTQUIET, _("File %s retrieved but size does not match. " "\n"), quote (destname)); } else if (!hash_ok) { retr_err = METALINK_CHKSUM_ERROR; logprintf (LOG_NOTQUIET, _("File %s retrieved but checksum does not match. " "\n"), quote (destname)); } #ifdef HAVE_GPGME /* Signature will be only validated if hash check was successful. */ else if (sig_status < 0) { retr_err = METALINK_SIG_ERROR; logprintf (LOG_NOTQUIET, _("File %s retrieved but signature does not match. " "\n"), quote (destname)); } #endif last_retr_err = retr_err == RETROK ? last_retr_err : retr_err; /* Rename the file if error encountered; remove if option specified. Note: the file has been downloaded using *_loop. Therefore, it is not necessary to keep the file for continuated download. */ if (((retr_err != RETROK && !opt.always_rest) || opt.delete_after) && destname != NULL && file_exists_p (destname, NULL)) { badhash_or_remove (destname); } if (output_stream) { fclose (output_stream); output_stream = NULL; } xfree (destname); xfree (filename); xfree (trsrname); xfree (planname); } /* Iterate over files. */ /* Restore original values. */ opt.output_document = _output_document; output_stream_regular = _output_stream_regular; output_stream = _output_stream; return last_retr_err; } /* Replace/remove the basename of a file name. The file name is permanently modified. Always set NAME to a string, even an empty one. Use REF's basename as replacement. If REF is NULL or if it doesn't provide a valid basename candidate, then remove NAME's basename. */ void replace_metalink_basename (char **name, char *ref) { int n; char *p, *new, *basename; if (!name) return; /* Strip old basename. */ if (*name) { basename = last_component (*name); if (basename == *name) xfree (*name); else *basename = '\0'; } /* New basename from file name reference. */ if (ref) ref = last_component (ref); /* Replace the old basename. */ new = aprintf ("%s%s", *name ? *name : "", ref ? ref : ""); xfree (*name); *name = new; /* Remove prefix drive letters if required, i.e. when in w32 environments. */ p = new; while (p[0] != '\0') { while ((n = FILE_SYSTEM_PREFIX_LEN (p)) > 0) p += n; if (p != new) { while (ISSLASH (p[0])) ++p; new = p; continue; } break; } if (*name != new) { new = xstrdup (new); xfree (*name); *name = new; } } /* Strip the directory components from the given name. Return a pointer to the end of the leading directory components. Return NULL if the resulting name is unsafe or invalid. Due to security issues posed by saving files with unsafe names, here the use of libmetalink's metalink_check_safe_path() is enforced. If this appears redundant because the given name was already verified, just remember to never underestimate unsafe file names. */ char * get_metalink_basename (char *name) { int n; char *basename; if (!name) return NULL; basename = last_component (name); while ((n = FILE_SYSTEM_PREFIX_LEN (basename)) > 0) basename += n; return metalink_check_safe_path (basename) ? basename : NULL; } /* Append a separator and a numeric suffix to a string. The string is permanently modified. */ void append_suffix_number (char **str, const char *sep, wgint num) { char *new, buf[24]; number_to_string (buf, num); new = aprintf ("%s%s%s", *str ? *str : "", sep ? sep : "", buf); xfree (*str); *str = new; } /* Remove the string's trailing/leading whitespaces and line breaks. The string is permanently modified. */ void clean_metalink_string (char **str) { int c; size_t len; char *new, *beg, *end; if (!str || !*str) return; beg = *str; while ((c = *beg) && (c == '\n' || c == '\r' || c == '\t' || c == ' ')) beg++; end = beg; /* To not truncate a string containing spaces, search the first '\r' or '\n' which ipotetically marks the end of the string. */ while ((c = *end) && (c != '\r') && (c != '\n')) end++; /* If we are at the end of the string, search the first legit character going backward. */ if (*end == '\0') while ((c = *(end - 1)) && (c == '\n' || c == '\r' || c == '\t' || c == ' ')) end--; len = end - beg; new = xmemdup0 (beg, len); xfree (*str); *str = new; } /* Remove the quotation surrounding a string. The string is permanently modified. */ void dequote_metalink_string (char **str) { char *new; size_t str_len; if (!str || !*str || ((*str)[0] != '\"' && (*str)[0] != '\'')) return; str_len = strlen (*str); /* current string length */ /* Verify if the current string is surrounded by quotes. */ if (str_len < 2 || (*str)[0] != (*str)[str_len - 1]) return; /* Dequoted string. */ new = xmemdup0 (*str + 1, str_len - 2); xfree (*str); *str = new; } /* Append the suffix ".badhash" to the file NAME, except without overwriting an existing file with that name and suffix. */ void badhash_suffix (char *name) { char *bhash, *uname; bhash = concat_strings (name, ".badhash", (char *)0); uname = unique_name (bhash); logprintf (LOG_VERBOSE, _("Renaming %s to %s.\n"), quote_n (0, name), quote_n (1, uname)); if (link (name, uname)) logprintf (LOG_NOTQUIET, "link: %s\n", strerror (errno)); else if (unlink (name)) logprintf (LOG_NOTQUIET, "unlink: %s\n", strerror (errno)); xfree (bhash); xfree (uname); } /* Append the suffix ".badhash" to the file NAME, except without overwriting an existing file with that name and suffix. Remove the file NAME if the option --delete-after is specified, or if the option --keep-badhash isn't set. */ void badhash_or_remove (char *name) { if (opt.delete_after || !opt.keep_badhash) { logprintf (LOG_VERBOSE, _("Removing %s.\n"), quote (name)); if (unlink (name)) logprintf (LOG_NOTQUIET, "unlink: %s\n", strerror (errno)); } else { badhash_suffix(name); } } /* Simple file fetch. Set DESTNAME to the name of the saved file. Resume previous download if RESUME is true. To disable Metalink/HTTP, set METALINK_HTTP to false. */ uerr_t fetch_metalink_file (const char *url_str, bool resume, bool metalink_http, const char *filename, char **destname) { FILE *_output_stream = output_stream; bool _output_stream_regular = output_stream_regular; char *_output_document = opt.output_document; bool _metalink_http = opt.metalink_over_http; char *local_file = NULL; uerr_t retr_err = URLERROR; struct iri *iri; struct url *url; int url_err; /* Parse the URL. */ iri = iri_new (); set_uri_encoding (iri, opt.locale, true); url = url_parse (url_str, &url_err, iri, false); if (!url) { char *error = url_error (url_str, url_err); logprintf (LOG_NOTQUIET, "%s: %s.\n", url_str, error); inform_exit_status (retr_err); iri_free (iri); xfree (error); return retr_err; } output_stream = NULL; if (resume) /* continue previous download */ output_stream = fopen (filename, "ab"); else /* create a file with an unique name */ output_stream = unique_create (filename, true, &local_file); output_stream_regular = true; /* If output_stream is NULL, the file couldn't be created/opened. This could be due to the impossibility to create a directory tree: * stdio.h (fopen) * src/utils.c (unique_create) A call to retrieve_url() can indirectly create a directory tree, when opt.output_document is set to the destination file name and output_stream is left to NULL: * src/http.c (open_output_stream): If output_stream is NULL, create the destination opt.output_document "path/file" */ if (!local_file) local_file = xstrdup (filename); /* Store the real file name for displaying in messages, and for proper "path/file" handling. */ opt.output_document = local_file; opt.metalink_over_http = metalink_http; DEBUGP (("Storing to %s\n", local_file)); retr_err = retrieve_url (url, url_str, NULL, NULL, NULL, NULL, opt.recursive, iri, false); if (retr_err == RETROK) { if (destname) *destname = local_file; else xfree (local_file); } if (output_stream) { fclose (output_stream); output_stream = NULL; } opt.metalink_over_http = _metalink_http; opt.output_document = _output_document; output_stream_regular = _output_stream_regular; output_stream = _output_stream; inform_exit_status (retr_err); iri_free (iri); url_free (url); return retr_err; } int metalink_res_cmp (const void* v1, const void* v2) { const metalink_resource_t *res1 = *(metalink_resource_t **) v1, *res2 = *(metalink_resource_t **) v2; if (res1->preference != res2->preference) return res2->preference - res1->preference; if (res1->priority != res2->priority) return res1->priority - res2->priority; if (opt.preferred_location) { int cmp = 0; if (res1->location && !c_strcasecmp (opt.preferred_location, res1->location)) cmp -= 1; if (res2->location && !c_strcasecmp (opt.preferred_location, res2->location)) cmp += 1; return cmp; } return 0; } int metalink_meta_cmp (const void* v1, const void* v2) { const metalink_metaurl_t *meta1 = *(metalink_metaurl_t **) v1, *meta2 = *(metalink_metaurl_t **) v2; if (meta1->priority != meta2->priority) return meta1->priority - meta2->priority; return 0; } /* Find value of given key. This is intended for Link header, but will work with any header that uses ';' as field separator and '=' as key-value separator. Link = "Link" ":" #link-value link-value = "<" URI-Reference ">" *( ";" link-param ) link-param = ( ( "rel" "=" relation-types ) | ( "anchor" "=" <"> URI-Reference <"> ) | ( "rev" "=" relation-types ) | ( "hreflang" "=" Language-Tag ) | ( "media" "=" ( MediaDesc | ( <"> MediaDesc <"> ) ) ) | ( "title" "=" quoted-string ) | ( "title*" "=" ext-value ) | ( "type" "=" ( media-type | quoted-mt ) ) | ( link-extension ) ) link-extension = ( parmname [ "=" ( ptoken | quoted-string ) ] ) | ( ext-name-star "=" ext-value ) ext-name-star = parmname "*" ; reserved for RFC2231-profiled ; extensions. Whitespace NOT ; allowed in between. ptoken = 1*ptokenchar ptokenchar = "!" | "#" | "$" | "%" | "&" | "'" | "(" | ")" | "*" | "+" | "-" | "." | "/" | DIGIT | ":" | "<" | "=" | ">" | "?" | "@" | ALPHA | "[" | "]" | "^" | "_" | "`" | "{" | "|" | "}" | "~" media-type = type-name "/" subtype-name quoted-mt = <"> media-type <"> relation-types = relation-type | <"> relation-type *( 1*SP relation-type ) <"> relation-type = reg-rel-type | ext-rel-type reg-rel-type = LOALPHA *( LOALPHA | DIGIT | "." | "-" ) ext-rel-type = URI See more: rfc5988 */ bool find_key_value (const char *start, const char *end, const char *key, char **value) { const char *eq; size_t key_len = strlen (key); const char *val_beg, *val_end; const char *key_beg; key_beg = start; while (key_beg + key_len + 1 < end) { /* Skip whitespaces. */ while (key_beg + key_len + 1 < end && c_isspace (*key_beg)) key_beg++; if (strncmp (key_beg, key, key_len)) { /* Find next token. */ while (key_beg + key_len + 1 < end && *key_beg != ';') key_beg++; key_beg++; continue; } else { /* Find equals sign. */ eq = key_beg + key_len; while (eq < end && c_isspace (*eq)) eq++; if (eq == end) return false; if (*eq != '=') { key_beg++; continue; } val_beg = eq + 1; while (val_beg < end && c_isspace (*val_beg)) val_beg++; if (val_beg == end) return false; val_end = val_beg + 1; while (val_end < end && *val_end != ';' && !c_isspace (*val_end)) val_end++; *value = xstrndup (val_beg, val_end - val_beg); dequote_metalink_string (value); return true; } } *value = NULL; return false; } /* This is to check if given token exists in HTTP header. Tokens are separated by ';'. */ bool has_key (const char *start, const char *end, const char *key) { const char *pos; /* Here would the token start. */ size_t key_len = strlen (key); pos = start; while (pos + key_len <= end) { /* Skip whitespaces at beginning. */ while (pos + key_len <= end && c_isspace (*pos)) pos++; /* Does the prefix of pos match our key? */ if (strncmp (key, pos, key_len)) { /* This was not a match. Skip all characters until beginning of next token. */ while (pos + key_len <= end && *pos != ';') pos++; pos++; continue; } /* key is prefix of pos. Is it the exact token or just a prefix? */ pos += key_len; while (pos < end && c_isspace (*pos)) pos++; if (pos == end || *pos == ';') return true; /* This was not a match (just a prefix). Skip all characters until beginning of next token. */ while (pos + key_len <= end && *pos != ';') pos++; pos++; } return false; } /* Find all key=value pairs delimited with ';' or ','. This is intended for Digest header parsing. The usage is: const char *pos; for (pos = header_beg; pos = find_key_values (pos, header_end, &key, &val); pos++) { ... } */ const char * find_key_values (const char *start, const char *end, char **key, char **value) { const char *key_start, *key_end; const char *eq; const char *val_start, *val_end; eq = start; while (eq < end && *eq != '=') { /* Skip tokens without =value part. */ if (*eq == ';' || *eq == ',') start = eq + 1; eq++; } if (eq >= end) return NULL; key_start = start; while (key_start < eq && c_isspace (*key_start)) key_start++; key_end = eq - 1; while (key_end > key_start && c_isspace (*key_end)) key_end--; key_end++; val_start = eq + 1; while (val_start < end && c_isspace (*val_start)) val_start++; val_end = val_start; while (val_end < end && *val_end != ';' && *val_end != ',' && !c_isspace (*val_end)) val_end++; *key = xstrndup (key_start, key_end - key_start); *value = xstrndup (val_start, val_end - val_start); dequote_metalink_string (value); /* Skip trailing whitespaces. */ while (val_end < end && c_isspace (*val_end)) val_end++; return val_end; } #ifdef TESTING const char * test_find_key_values (void) { static const char *header_data = "key1=val1;key2=\"val2\" ;key3=val3; key4=val4"\ " ; key5=val5;key6 ='val6';key7= val7; "\ "key8 = val8 ; key9 = \"val9\" "\ " ,key10= 'val10',key11,key12=val12"; static const struct { const char *key; const char *val; } test_array[] = { { "key1", "val1" }, { "key2", "val2" }, { "key3", "val3" }, { "key4", "val4" }, { "key5", "val5" }, { "key6", "val6" }, { "key7", "val7" }, { "key8", "val8" }, { "key9", "val9" }, { "key10", "val10" }, { "key12", "val12" } }; const char *pos; char *key, *value; size_t i = 0; for (pos = header_data; (pos = find_key_values (pos, header_data + strlen (header_data), &key, &value)); pos++) { mu_assert ("test_find_key_values: wrong result", !strcmp (test_array[i].val, value) && !strcmp (test_array[i].key, key)); xfree (key); xfree (value); i++; } return NULL; } const char * test_find_key_value (void) { static const char *header_data = "key1=val1;key2=val2 ;key3='val3'; key4=val4"\ " ; key5='val5';key6 =val6;key7= \"val7\"; "\ "key8 = \"val8\" ; key9 = val9 "; static const struct { const char *key; const char *val; bool result; } test_array[] = { { "key1", "val1", true }, { "key2", "val2", true }, { "key3", "val3", true }, { "key4", "val4", true }, { "key5", "val5", true }, { "key6", "val6", true }, { "key7", "val7", true }, { "key8", "val8", true }, { "key9", "val9", true }, { "key10", NULL, false }, { "ey1", NULL, false }, { "dey1", NULL, false } }; size_t i; for (i=0; i < countof (test_array); ++i) { bool result; char *value; result = find_key_value (header_data, header_data + strlen(header_data), test_array[i].key, &value); mu_assert ("test_find_key_value: wrong result", result == test_array[i].result && ((!test_array[i].result && !value) || !strcmp (value, test_array[i].val))); xfree (value); } return NULL; } const char * test_has_key (void) { static const char *header_data = "key1=val2;token1;xyz; token2;xyz;token3 ;"\ "xyz; token4 ;xyz; token5 "; struct { const char *token; bool result; } test_array[] = { { "key1=val2", true }, { "token1", true }, { "token2", true }, { "token3", true }, { "token4", true }, { "token5", true }, { "token6", false }, { "oken1", false }, { "poken1", false }, { "key1=val2", true } }; size_t i; for (i = 0; i < countof (test_array); ++i) mu_assert ("test_has_key: wrong result", has_key (header_data, header_data + strlen (header_data), test_array[i].token) == test_array[i].result); return NULL; } #endif #endif /* HAVE_METALINK */ wget-1.21.2/src/url.c0000644000000000000000000022243714115732710011202 00000000000000/* URL handling. Copyright (C) 1996-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #include "wget.h" #include #include #include #include #include #include #include "utils.h" #include "url.h" #include "host.h" /* for is_valid_ipv6_address */ #include "c-strcase.h" #ifdef HAVE_ICONV # include #endif #include #ifdef __VMS #include "vms.h" #endif /* def __VMS */ #ifdef TESTING #include "../tests/unit-tests.h" #endif enum { scm_disabled = 1, /* for https when OpenSSL fails to init. */ scm_has_params = 2, /* whether scheme has ;params */ scm_has_query = 4, /* whether scheme has ?query */ scm_has_fragment = 8 /* whether scheme has #fragment */ }; struct scheme_data { /* Short name of the scheme, such as "http" or "ftp". */ const char *name; /* Leading string that identifies the scheme, such as "https://". */ const char *leading_string; /* Default port of the scheme when none is specified. */ int default_port; /* Various flags. */ int flags; }; /* Supported schemes: */ static struct scheme_data supported_schemes[] = { { "http", "http://", DEFAULT_HTTP_PORT, scm_has_query|scm_has_fragment }, #ifdef HAVE_SSL { "https", "https://", DEFAULT_HTTPS_PORT, scm_has_query|scm_has_fragment }, #endif { "ftp", "ftp://", DEFAULT_FTP_PORT, scm_has_params|scm_has_fragment }, #ifdef HAVE_SSL /* * Explicit FTPS uses the same port as FTP. * Implicit FTPS has its own port (990), but it is disabled by default. */ { "ftps", "ftps://", DEFAULT_FTP_PORT, scm_has_params|scm_has_fragment }, #endif /* SCHEME_INVALID */ { NULL, NULL, -1, 0 } }; /* Forward declarations: */ static bool path_simplify (enum url_scheme, char *); /* Support for escaping and unescaping of URL strings. */ /* Table of "reserved" and "unsafe" characters. Those terms are rfc1738-speak, as such largely obsoleted by rfc2396 and later specs, but the general idea remains. A reserved character is the one that you can't decode without changing the meaning of the URL. For example, you can't decode "/foo/%2f/bar" into "/foo///bar" because the number and contents of path components is different. Non-reserved characters can be changed, so "/foo/%78/bar" is safe to change to "/foo/x/bar". The unsafe characters are loosely based on rfc1738, plus "$" and ",", as recommended by rfc2396, and minus "~", which is very frequently used (and sometimes unrecognized as %7E by broken servers). An unsafe character is the one that should be encoded when URLs are placed in foreign environments. E.g. space and newline are unsafe in HTTP contexts because HTTP uses them as separator and line terminator, so they must be encoded to %20 and %0A respectively. "*" is unsafe in shell context, etc. We determine whether a character is unsafe through static table lookup. This code assumes ASCII character set and 8-bit chars. */ enum { /* rfc1738 reserved chars + "$" and ",". */ urlchr_reserved = 1, /* rfc1738 unsafe chars, plus non-printables. */ urlchr_unsafe = 2 }; #define urlchr_test(c, mask) (urlchr_table[(unsigned char)(c)] & (mask)) #define URL_RESERVED_CHAR(c) urlchr_test(c, urlchr_reserved) #define URL_UNSAFE_CHAR(c) urlchr_test(c, urlchr_unsafe) /* Shorthands for the table: */ #define R urlchr_reserved #define U urlchr_unsafe #define RU R|U static const unsigned char urlchr_table[256] = { U, U, U, U, U, U, U, U, /* NUL SOH STX ETX EOT ENQ ACK BEL */ U, U, U, U, U, U, U, U, /* BS HT LF VT FF CR SO SI */ U, U, U, U, U, U, U, U, /* DLE DC1 DC2 DC3 DC4 NAK SYN ETB */ U, U, U, U, U, U, U, U, /* CAN EM SUB ESC FS GS RS US */ U, 0, U, RU, R, U, R, 0, /* SP ! " # $ % & ' */ 0, 0, 0, R, R, 0, 0, R, /* ( ) * + , - . / */ 0, 0, 0, 0, 0, 0, 0, 0, /* 0 1 2 3 4 5 6 7 */ 0, 0, RU, R, U, R, U, R, /* 8 9 : ; < = > ? */ RU, 0, 0, 0, 0, 0, 0, 0, /* @ A B C D E F G */ 0, 0, 0, 0, 0, 0, 0, 0, /* H I J K L M N O */ 0, 0, 0, 0, 0, 0, 0, 0, /* P Q R S T U V W */ 0, 0, 0, RU, U, RU, U, 0, /* X Y Z [ \ ] ^ _ */ U, 0, 0, 0, 0, 0, 0, 0, /* ` a b c d e f g */ 0, 0, 0, 0, 0, 0, 0, 0, /* h i j k l m n o */ 0, 0, 0, 0, 0, 0, 0, 0, /* p q r s t u v w */ 0, 0, 0, U, U, U, 0, U, /* x y z { | } ~ DEL */ U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, U, }; #undef R #undef U #undef RU static void url_unescape_1 (char *s, unsigned char mask) { unsigned char *t = (unsigned char *) s; /* t - tortoise */ unsigned char *h = (unsigned char *) s; /* h - hare */ for (; *h; h++, t++) { if (*h != '%') { copychar: *t = *h; } else { unsigned char c; /* Do nothing if '%' is not followed by two hex digits. */ if (!h[1] || !h[2] || !(c_isxdigit (h[1]) && c_isxdigit (h[2]))) goto copychar; c = X2DIGITS_TO_NUM (h[1], h[2]); if (urlchr_test(c, mask)) goto copychar; /* Don't unescape %00 because there is no way to insert it into a C string without effectively truncating it. */ if (c == '\0') goto copychar; *t = c; h += 2; } } *t = '\0'; } /* URL-unescape the string S. This is done by transforming the sequences "%HH" to the character represented by the hexadecimal digits HH. If % is not followed by two hexadecimal digits, it is inserted literally. The transformation is done in place. If you need the original string intact, make a copy before calling this function. */ void url_unescape (char *s) { url_unescape_1 (s, 0); } /* URL-unescape the string S. This functions behaves identically as url_unescape(), but does not convert characters from "reserved". In other words, it only converts "unsafe" characters. */ void url_unescape_except_reserved (char *s) { url_unescape_1 (s, urlchr_reserved); } /* The core of url_escape_* functions. Escapes the characters that match the provided mask in urlchr_table. If ALLOW_PASSTHROUGH is true, a string with no unsafe chars will be returned unchanged. If ALLOW_PASSTHROUGH is false, a freshly allocated string will be returned in all cases. */ static char * url_escape_1 (const char *s, unsigned char mask, bool allow_passthrough) { const char *p1; char *p2, *newstr; int newlen; int addition = 0; for (p1 = s; *p1; p1++) if (urlchr_test (*p1, mask)) addition += 2; /* Two more characters (hex digits) */ if (!addition) return allow_passthrough ? (char *)s : xstrdup (s); newlen = (p1 - s) + addition; newstr = xmalloc (newlen + 1); p1 = s; p2 = newstr; while (*p1) { /* Quote the characters that match the test mask. */ if (urlchr_test (*p1, mask)) { unsigned char c = *p1++; *p2++ = '%'; *p2++ = XNUM_TO_DIGIT (c >> 4); *p2++ = XNUM_TO_DIGIT (c & 0xf); } else *p2++ = *p1++; } assert (p2 - newstr == newlen); *p2 = '\0'; return newstr; } /* URL-escape the unsafe characters (see urlchr_table) in a given string, returning a freshly allocated string. */ char * url_escape (const char *s) { return url_escape_1 (s, urlchr_unsafe, false); } /* URL-escape the unsafe and reserved characters (see urlchr_table) in a given string, returning a freshly allocated string. */ char * url_escape_unsafe_and_reserved (const char *s) { return url_escape_1 (s, urlchr_unsafe|urlchr_reserved, false); } /* URL-escape the unsafe characters (see urlchr_table) in a given string. If no characters are unsafe, S is returned. */ static char * url_escape_allow_passthrough (const char *s) { return url_escape_1 (s, urlchr_unsafe, true); } /* Decide whether the char at position P needs to be encoded. (It is not enough to pass a single char *P because the function may need to inspect the surrounding context.) Return true if the char should be escaped as %XX, false otherwise. */ static inline bool char_needs_escaping (const char *p) { if (*p == '%') { if (c_isxdigit (*(p + 1)) && c_isxdigit (*(p + 2))) return false; else /* Garbled %.. sequence: encode `%'. */ return true; } else if (URL_UNSAFE_CHAR (*p) && !URL_RESERVED_CHAR (*p)) return true; else return false; } /* Translate a %-escaped (but possibly non-conformant) input string S into a %-escaped (and conformant) output string. If no characters are encoded or decoded, return the same string S; otherwise, return a freshly allocated string with the new contents. After a URL has been run through this function, the protocols that use `%' as the quote character can use the resulting string as-is, while those that don't can use url_unescape to get to the intended data. This function is stable: once the input is transformed, further transformations of the result yield the same output. Let's discuss why this function is needed. Imagine Wget is asked to retrieve `http://abc.xyz/abc def'. Since a raw space character would mess up the HTTP request, it needs to be quoted, like this: GET /abc%20def HTTP/1.0 It would appear that the unsafe chars need to be quoted, for example with url_escape. But what if we're requested to download `abc%20def'? url_escape transforms "%" to "%25", which would leave us with `abc%2520def'. This is incorrect -- since %-escapes are part of URL syntax, "%20" is the correct way to denote a literal space on the Wget command line. This leads to the conclusion that in that case Wget should not call url_escape, but leave the `%20' as is. This is clearly contradictory, but it only gets worse. What if the requested URI is `abc%20 def'? If we call url_escape, we end up with `/abc%2520%20def', which is almost certainly not intended. If we don't call url_escape, we are left with the embedded space and cannot complete the request. What the user meant was for Wget to request `/abc%20%20def', and this is where reencode_escapes kicks in. Wget used to solve this by first decoding %-quotes, and then encoding all the "unsafe" characters found in the resulting string. This was wrong because it didn't preserve certain URL special (reserved) characters. For instance, URI containing "a%2B+b" (0x2b == '+') would get translated to "a%2B%2Bb" or "a++b" depending on whether we considered `+' reserved (it is). One of these results is inevitable because by the second step we would lose information on whether the `+' was originally encoded or not. Both results were wrong because in CGI parameters + means space, while %2B means literal plus. reencode_escapes correctly translates the above to "a%2B+b", i.e. returns the original string. This function uses a modified version of the algorithm originally proposed by Anon Sricharoenchai: * Encode all "unsafe" characters, except those that are also "reserved", to %XX. See urlchr_table for which characters are unsafe and reserved. * Encode the "%" characters not followed by two hex digits to "%25". * Pass through all other characters and %XX escapes as-is. (Up to Wget 1.10 this decoded %XX escapes corresponding to "safe" characters, but that was obtrusive and broke some servers.) Anon's test case: "http://abc.xyz/%20%3F%%36%31%25aa% a?a=%61+a%2Ba&b=b%26c%3Dc" -> "http://abc.xyz/%20%3F%25%36%31%25aa%25%20a?a=%61+a%2Ba&b=b%26c%3Dc" Simpler test cases: "foo bar" -> "foo%20bar" "foo%20bar" -> "foo%20bar" "foo %20bar" -> "foo%20%20bar" "foo%%20bar" -> "foo%25%20bar" (0x25 == '%') "foo%25%20bar" -> "foo%25%20bar" "foo%2%20bar" -> "foo%252%20bar" "foo+bar" -> "foo+bar" (plus is reserved!) "foo%2b+bar" -> "foo%2b+bar" */ static char * reencode_escapes (const char *s) { const char *p1; char *newstr, *p2; int oldlen, newlen; int encode_count = 0; /* First pass: inspect the string to see if there's anything to do, and to calculate the new length. */ for (p1 = s; *p1; p1++) if (char_needs_escaping (p1)) ++encode_count; if (!encode_count) /* The string is good as it is. */ return (char *) s; /* C const model sucks. */ oldlen = p1 - s; /* Each encoding adds two characters (hex digits). */ newlen = oldlen + 2 * encode_count; newstr = xmalloc (newlen + 1); /* Second pass: copy the string to the destination address, encoding chars when needed. */ p1 = s; p2 = newstr; while (*p1) if (char_needs_escaping (p1)) { unsigned char c = *p1++; *p2++ = '%'; *p2++ = XNUM_TO_DIGIT (c >> 4); *p2++ = XNUM_TO_DIGIT (c & 0xf); } else *p2++ = *p1++; *p2 = '\0'; assert (p2 - newstr == newlen); return newstr; } /* Returns the scheme type if the scheme is supported, or SCHEME_INVALID if not. */ enum url_scheme url_scheme (const char *url) { int i; for (i = 0; supported_schemes[i].leading_string; i++) if (0 == c_strncasecmp (url, supported_schemes[i].leading_string, strlen (supported_schemes[i].leading_string))) { if (!(supported_schemes[i].flags & scm_disabled)) return (enum url_scheme) i; else return SCHEME_INVALID; } return SCHEME_INVALID; } #define SCHEME_CHAR(ch) (c_isalnum (ch) || (ch) == '-' || (ch) == '+') /* Return 1 if the URL begins with any "scheme", 0 otherwise. As currently implemented, it returns true if URL begins with [-+a-zA-Z0-9]+: . */ bool url_has_scheme (const char *url) { const char *p = url; /* The first char must be a scheme char. */ if (!*p || !SCHEME_CHAR (*p)) return false; ++p; /* Followed by 0 or more scheme chars. */ while (*p && SCHEME_CHAR (*p)) ++p; /* Terminated by ':'. */ return *p == ':'; } bool url_valid_scheme (const char *url) { enum url_scheme scheme = url_scheme (url); return scheme != SCHEME_INVALID; } int scheme_default_port (enum url_scheme scheme) { return supported_schemes[scheme].default_port; } void scheme_disable (enum url_scheme scheme) { supported_schemes[scheme].flags |= scm_disabled; } const char * scheme_leading_string (enum url_scheme scheme) { return supported_schemes[scheme].leading_string; } /* Skip the username and password, if present in the URL. The function should *not* be called with the complete URL, but with the portion after the scheme. If no username and password are found, return URL. */ static const char * url_skip_credentials (const char *url) { /* Look for '@' that comes before terminators, such as '/', '?', '#', or ';'. */ const char *p = (const char *)strpbrk (url, "@/?#;"); if (!p || *p != '@') return url; return p + 1; } /* Parse credentials contained in [BEG, END). The region is expected to have come from a URL and is unescaped. */ static bool parse_credentials (const char *beg, const char *end, char **user, char **passwd) { char *colon; const char *userend; if (beg == end) return false; /* empty user name */ colon = memchr (beg, ':', end - beg); if (colon == beg) return false; /* again empty user name */ if (colon) { *passwd = strdupdelim (colon + 1, end); userend = colon; url_unescape (*passwd); } else { *passwd = NULL; userend = end; } *user = strdupdelim (beg, userend); url_unescape (*user); return true; } /* Used by main.c: detect URLs written using the "shorthand" URL forms originally popularized by Netscape and NcFTP. HTTP shorthands look like this: www.foo.com[:port]/dir/file -> http://www.foo.com[:port]/dir/file www.foo.com[:port] -> http://www.foo.com[:port] FTP shorthands look like this: foo.bar.com:dir/file -> ftp://foo.bar.com/dir/file foo.bar.com:/absdir/file -> ftp://foo.bar.com//absdir/file If the URL needs not or cannot be rewritten, return NULL. */ char * rewrite_shorthand_url (const char *url) { const char *p; char *ret; if (url_scheme (url) != SCHEME_INVALID) return NULL; /* Look for a ':' or '/'. The former signifies NcFTP syntax, the latter Netscape. */ p = strpbrk (url, ":/"); if (p == url) return NULL; /* If we're looking at "://", it means the URL uses a scheme we don't support, which may include "https" when compiled without SSL support. Don't bogusly rewrite such URLs. */ if (p && p[0] == ':' && p[1] == '/' && p[2] == '/') return NULL; if (p && *p == ':') { /* Colon indicates ftp, as in foo.bar.com:path. Check for special case of http port number ("localhost:10000"). */ int digits = strspn (p + 1, "0123456789"); if (digits && (p[1 + digits] == '/' || p[1 + digits] == '\0')) goto http; /* Turn "foo.bar.com:path" to "ftp://foo.bar.com/path". */ if ((ret = aprintf ("ftp://%s", url)) != NULL) ret[6 + (p - url)] = '/'; } else { http: /* Just prepend "http://" to URL. */ ret = aprintf ("http://%s", url); } return ret; } static void split_path (const char *, char **, char **); /* Like strpbrk, with the exception that it returns the pointer to the terminating zero (end-of-string aka "eos") if no matching character is found. */ static inline char * strpbrk_or_eos (const char *s, const char *accept) { char *p = strpbrk (s, accept); if (!p) p = strchr (s, '\0'); return p; } /* Turn STR into lowercase; return true if a character was actually changed. */ static bool lowercase_str (char *str) { bool changed = false; for (; *str; str++) if (c_isupper (*str)) { changed = true; *str = c_tolower (*str); } return changed; } static const char * init_seps (enum url_scheme scheme) { static char seps[8] = ":/"; char *p = seps + 2; int flags = supported_schemes[scheme].flags; if (flags & scm_has_params) *p++ = ';'; if (flags & scm_has_query) *p++ = '?'; if (flags & scm_has_fragment) *p++ = '#'; *p = '\0'; return seps; } static const char *parse_errors[] = { #define PE_NO_ERROR 0 N_("No error"), #define PE_UNSUPPORTED_SCHEME 1 N_("Unsupported scheme %s"), /* support for format token only here */ #define PE_MISSING_SCHEME 2 N_("Scheme missing"), #define PE_INVALID_HOST_NAME 3 N_("Invalid host name"), #define PE_BAD_PORT_NUMBER 4 N_("Bad port number"), #define PE_INVALID_USER_NAME 5 N_("Invalid user name"), #define PE_UNTERMINATED_IPV6_ADDRESS 6 N_("Unterminated IPv6 numeric address"), #define PE_IPV6_NOT_SUPPORTED 7 N_("IPv6 addresses not supported"), #define PE_INVALID_IPV6_ADDRESS 8 N_("Invalid IPv6 numeric address") }; /* Parse a URL. Return a new struct url if successful, NULL on error. In case of error, and if ERROR is not NULL, also set *ERROR to the appropriate error code. */ struct url * url_parse (const char *url, int *error, struct iri *iri, bool percent_encode) { struct url *u; const char *p; bool path_modified, host_modified; enum url_scheme scheme; const char *seps; const char *uname_b, *uname_e; const char *host_b, *host_e; const char *path_b, *path_e; const char *params_b, *params_e; const char *query_b, *query_e; const char *fragment_b, *fragment_e; int port; char *user = NULL, *passwd = NULL; const char *url_encoded = NULL; int error_code; scheme = url_scheme (url); if (scheme == SCHEME_INVALID) { if (url_has_scheme (url)) error_code = PE_UNSUPPORTED_SCHEME; else error_code = PE_MISSING_SCHEME; goto error; } url_encoded = url; if (iri && iri->utf8_encode) { char *new_url = NULL; iri->utf8_encode = remote_to_utf8 (iri, iri->orig_url ? iri->orig_url : url, &new_url); if (!iri->utf8_encode) new_url = NULL; else { xfree (iri->orig_url); iri->orig_url = xstrdup (url); url_encoded = reencode_escapes (new_url); if (url_encoded != new_url) xfree (new_url); percent_encode = false; } } if (percent_encode) url_encoded = reencode_escapes (url); p = url_encoded; p += strlen (supported_schemes[scheme].leading_string); uname_b = p; p = url_skip_credentials (p); uname_e = p; /* scheme://user:pass@host[:port]... */ /* ^ */ /* We attempt to break down the URL into the components path, params, query, and fragment. They are ordered like this: scheme://host[:port][/path][;params][?query][#fragment] */ path_b = path_e = NULL; params_b = params_e = NULL; query_b = query_e = NULL; fragment_b = fragment_e = NULL; /* Initialize separators for optional parts of URL, depending on the scheme. For example, FTP has params, and HTTP and HTTPS have query string and fragment. */ seps = init_seps (scheme); host_b = p; if (*p == '[') { /* Handle IPv6 address inside square brackets. Ideally we'd just look for the terminating ']', but rfc2732 mandates rejecting invalid IPv6 addresses. */ /* The address begins after '['. */ host_b = p + 1; host_e = strchr (host_b, ']'); if (!host_e) { error_code = PE_UNTERMINATED_IPV6_ADDRESS; goto error; } #ifdef ENABLE_IPV6 /* Check if the IPv6 address is valid. */ if (!is_valid_ipv6_address(host_b, host_e)) { error_code = PE_INVALID_IPV6_ADDRESS; goto error; } /* Continue parsing after the closing ']'. */ p = host_e + 1; #else error_code = PE_IPV6_NOT_SUPPORTED; goto error; #endif /* The closing bracket must be followed by a separator or by the null char. */ /* http://[::1]... */ /* ^ */ if (!strchr (seps, *p)) { /* Trailing garbage after []-delimited IPv6 address. */ error_code = PE_INVALID_HOST_NAME; goto error; } } else { p = strpbrk_or_eos (p, seps); host_e = p; } ++seps; /* advance to '/' */ if (host_b == host_e) { error_code = PE_INVALID_HOST_NAME; goto error; } port = scheme_default_port (scheme); if (*p == ':') { const char *port_b, *port_e, *pp; /* scheme://host:port/tralala */ /* ^ */ ++p; port_b = p; p = strpbrk_or_eos (p, seps); port_e = p; /* Allow empty port, as per rfc2396. */ if (port_b != port_e) for (port = 0, pp = port_b; pp < port_e; pp++) { if (!c_isdigit (*pp)) { /* http://host:12randomgarbage/blah */ /* ^ */ error_code = PE_BAD_PORT_NUMBER; goto error; } port = 10 * port + (*pp - '0'); /* Check for too large port numbers here, before we have a chance to overflow on bogus port values. */ if (port > 0xffff) { error_code = PE_BAD_PORT_NUMBER; goto error; } } } /* Advance to the first separator *after* '/' (either ';' or '?', depending on the scheme). */ ++seps; /* Get the optional parts of URL, each part being delimited by current location and the position of the next separator. */ #define GET_URL_PART(sepchar, var) do { \ if (*p == sepchar) \ var##_b = ++p, var##_e = p = strpbrk_or_eos (p, seps); \ ++seps; \ } while (0) GET_URL_PART ('/', path); if (supported_schemes[scheme].flags & scm_has_params) GET_URL_PART (';', params); if (supported_schemes[scheme].flags & scm_has_query) GET_URL_PART ('?', query); if (supported_schemes[scheme].flags & scm_has_fragment) GET_URL_PART ('#', fragment); #undef GET_URL_PART assert (*p == 0); if (uname_b != uname_e) { /* http://user:pass@host */ /* ^ ^ */ /* uname_b uname_e */ if (!parse_credentials (uname_b, uname_e - 1, &user, &passwd)) { error_code = PE_INVALID_USER_NAME; goto error; } } u = xnew0 (struct url); u->scheme = scheme; u->host = strdupdelim (host_b, host_e); u->port = port; u->user = user; u->passwd = passwd; u->path = strdupdelim (path_b, path_e); path_modified = path_simplify (scheme, u->path); split_path (u->path, &u->dir, &u->file); host_modified = lowercase_str (u->host); /* Decode %HH sequences in host name. This is important not so much to support %HH sequences in host names (which other browser don't), but to support binary characters (which will have been converted to %HH by reencode_escapes). */ if (strchr (u->host, '%')) { url_unescape (u->host); host_modified = true; /* check for invalid control characters in host name */ for (p = u->host; *p; p++) { if (c_iscntrl(*p)) { url_free(u); error_code = PE_INVALID_HOST_NAME; goto error; } } /* Apply IDNA regardless of iri->utf8_encode status */ if (opt.enable_iri && iri) { char *new = idn_encode (iri, u->host); if (new) { xfree (u->host); u->host = new; host_modified = true; } } } if (params_b) u->params = strdupdelim (params_b, params_e); if (query_b) u->query = strdupdelim (query_b, query_e); if (fragment_b) u->fragment = strdupdelim (fragment_b, fragment_e); if (opt.enable_iri || path_modified || u->fragment || host_modified || path_b == path_e) { /* If we suspect that a transformation has rendered what url_string might return different from URL_ENCODED, rebuild u->url using url_string. */ u->url = url_string (u, URL_AUTH_SHOW); if (url_encoded != url) xfree (url_encoded); } else { if (url_encoded == url) u->url = xstrdup (url); else u->url = (char *) url_encoded; } return u; error: /* Cleanup in case of error: */ if (url_encoded && url_encoded != url) xfree (url_encoded); /* Transmit the error code to the caller, if the caller wants to know. */ if (error) *error = error_code; return NULL; } /* Return the error message string from ERROR_CODE, which should have been retrieved from url_parse. The error message is translated. */ char * url_error (const char *url, int error_code) { assert (error_code >= 0 && ((size_t) error_code) < countof (parse_errors)); if (error_code == PE_UNSUPPORTED_SCHEME) { char *error, *p; char *scheme = xstrdup (url); assert (url_has_scheme (url)); if ((p = strchr (scheme, ':'))) *p = '\0'; if (!c_strcasecmp (scheme, "https")) error = aprintf (_("HTTPS support not compiled in")); else error = aprintf (_(parse_errors[error_code]), quote (scheme)); xfree (scheme); return error; } else return xstrdup (_(parse_errors[error_code])); } /* Split PATH into DIR and FILE. PATH comes from the URL and is expected to be URL-escaped. The path is split into directory (the part up to the last slash) and file (the part after the last slash), which are subsequently unescaped. Examples: PATH DIR FILE "foo/bar/baz" "foo/bar" "baz" "foo/bar/" "foo/bar" "" "foo" "" "foo" "foo/bar/baz%2fqux" "foo/bar" "baz/qux" (!) DIR and FILE are freshly allocated. */ static void split_path (const char *path, char **dir, char **file) { char *last_slash = strrchr (path, '/'); if (!last_slash) { *dir = xstrdup (""); *file = xstrdup (path); } else { *dir = strdupdelim (path, last_slash); *file = xstrdup (last_slash + 1); } url_unescape (*dir); url_unescape (*file); } /* Note: URL's "full path" is the path with the query string and params appended. The "fragment" (#foo) is intentionally ignored, but that might be changed. For example, if the original URL was "http://host:port/foo/bar/baz;bullshit?querystring#uselessfragment", the full path will be "/foo/bar/baz;bullshit?querystring". */ /* Return the length of the full path, without the terminating zero. */ static int full_path_length (const struct url *url) { int len = 0; #define FROB(el) if (url->el) len += 1 + strlen (url->el) FROB (path); FROB (params); FROB (query); #undef FROB return len; } /* Write out the full path. */ static void full_path_write (const struct url *url, char *where) { #define FROB(el, chr) do { \ char *f_el = url->el; \ if (f_el) { \ int l = strlen (f_el); \ *where++ = chr; \ memcpy (where, f_el, l); \ where += l; \ } \ } while (0) FROB (path, '/'); FROB (params, ';'); FROB (query, '?'); #undef FROB } /* Public function for getting the "full path". E.g. if u->path is "foo/bar" and u->query is "param=value", full_path will be "/foo/bar?param=value". */ char * url_full_path (const struct url *url) { int length = full_path_length (url); char *full_path = xmalloc (length + 1); full_path_write (url, full_path); full_path[length] = '\0'; return full_path; } /* Unescape CHR in an otherwise escaped STR. Used to selectively escaping of certain characters, such as "/" and ":". Returns a count of unescaped chars. */ static void unescape_single_char (char *str, char chr) { const char c1 = XNUM_TO_DIGIT (chr >> 4); const char c2 = XNUM_TO_DIGIT (chr & 0xf); char *h = str; /* hare */ char *t = str; /* tortoise */ for (; *h; h++, t++) { if (h[0] == '%' && h[1] == c1 && h[2] == c2) { *t = chr; h += 2; } else *t = *h; } *t = '\0'; } /* Escape unsafe and reserved characters, except for the slash characters. */ static char * url_escape_dir (const char *dir) { char *newdir = url_escape_1 (dir, urlchr_unsafe | urlchr_reserved, 1); if (newdir == dir) return (char *)dir; unescape_single_char (newdir, '/'); return newdir; } /* Sync u->path and u->url with u->dir and u->file. Called after u->file or u->dir have been changed, typically by the FTP code. */ static void sync_path (struct url *u) { char *newpath, *efile, *edir; xfree (u->path); /* u->dir and u->file are not escaped. URL-escape them before reassembling them into u->path. That way, if they contain separators like '?' or even if u->file contains slashes, the path will be correctly assembled. (u->file can contain slashes if the URL specifies it with %2f, or if an FTP server returns it.) */ edir = url_escape_dir (u->dir); efile = url_escape_1 (u->file, urlchr_unsafe | urlchr_reserved, 1); if (!*edir) newpath = xstrdup (efile); else { int dirlen = strlen (edir); int filelen = strlen (efile); /* Copy "DIR/FILE" to newpath. */ char *p = newpath = xmalloc (dirlen + 1 + filelen + 1); memcpy (p, edir, dirlen); p += dirlen; *p++ = '/'; memcpy (p, efile, filelen); p += filelen; *p = '\0'; } u->path = newpath; if (edir != u->dir) xfree (edir); if (efile != u->file) xfree (efile); /* Regenerate u->url as well. */ xfree (u->url); u->url = url_string (u, URL_AUTH_SHOW); } /* Mutators. Code in ftp.c insists on changing u->dir and u->file. This way we can sync u->path and u->url when they get changed. */ void url_set_dir (struct url *url, const char *newdir) { xfree (url->dir); url->dir = xstrdup (newdir); sync_path (url); } void url_set_file (struct url *url, const char *newfile) { xfree (url->file); url->file = xstrdup (newfile); sync_path (url); } void url_free (struct url *url) { if (url) { xfree (url->host); xfree (url->path); xfree (url->url); xfree (url->params); xfree (url->query); xfree (url->fragment); xfree (url->user); xfree (url->passwd); xfree (url->dir); xfree (url->file); xfree (url); } } /* Create all the necessary directories for PATH (a file). Calls make_directory internally. */ int mkalldirs (const char *path) { const char *p; char *t; struct stat st; int res; p = strrchr(path, '/'); p = p == NULL ? path : p; /* Don't create if it's just a file. */ if ((p == path) && (*p != '/')) return 0; t = strdupdelim (path, p); /* Check whether the directory exists. */ if ((stat (t, &st) == 0)) { if (S_ISDIR (st.st_mode)) { xfree (t); return 0; } else { /* If the dir exists as a file name, remove it first. This is *only* for Wget to work with buggy old CERN http servers. Here is the scenario: When Wget tries to retrieve a directory without a slash, e.g. http://foo/bar (bar being a directory), CERN server will not redirect it too http://foo/bar/ -- it will generate a directory listing containing links to bar/file1, bar/file2, etc. Wget will lose because it saves this HTML listing to a file `bar', so it cannot create the directory. To work around this, if the file of the same name exists, we just remove it and create the directory anyway. */ DEBUGP (("Removing %s because of directory danger!\n", t)); if (unlink (t)) logprintf (LOG_NOTQUIET, "Failed to unlink %s (%d): %s\n", t, errno, strerror(errno)); } } res = make_directory (t); if (res != 0) logprintf (LOG_NOTQUIET, "%s: %s\n", t, strerror (errno)); xfree (t); return res; } /* Functions for constructing the file name out of URL components. */ /* A growable string structure, used by url_file_name and friends. This should perhaps be moved to utils.c. The idea is to have a convenient and efficient way to construct a string by having various functions append data to it. Instead of passing the obligatory BASEVAR, SIZEVAR and TAILPOS to all the functions in questions, we pass the pointer to this struct. Functions that write to the members in this struct must make sure that base remains null terminated by calling append_null(). */ struct growable { char *base; int size; /* memory allocated */ int tail; /* string length */ }; /* Ensure that the string can accept APPEND_COUNT more characters past the current TAIL position. If necessary, this will grow the string and update its allocated size. If the string is already large enough to take TAIL+APPEND_COUNT characters, this does nothing. */ #define GROW(g, append_size) do { \ struct growable *G_ = g; \ DO_REALLOC (G_->base, G_->size, G_->tail + append_size, char); \ } while (0) /* Return the tail position of the string. */ #define TAIL(r) ((r)->base + (r)->tail) /* Move the tail position by APPEND_COUNT characters. */ #define TAIL_INCR(r, append_count) ((r)->tail += append_count) /* Append NULL to DEST. */ static void append_null (struct growable *dest) { GROW (dest, 1); *TAIL (dest) = 0; } /* Append CH to DEST. */ static void append_char (char ch, struct growable *dest) { if (ch) { GROW (dest, 1); *TAIL (dest) = ch; TAIL_INCR (dest, 1); } append_null (dest); } /* Append the string STR to DEST. */ static void append_string (const char *str, struct growable *dest) { int l = strlen (str); if (l) { GROW (dest, l); memcpy (TAIL (dest), str, l); TAIL_INCR (dest, l); } append_null (dest); } enum { filechr_not_unix = 1, /* unusable on Unix, / and \0 */ filechr_not_vms = 2, /* unusable on VMS (ODS5), 0x00-0x1F * ? */ filechr_not_windows = 4, /* unusable on Windows, one of \|/<>?:*" */ filechr_control = 8 /* a control character, e.g. 0-31 */ }; #define FILE_CHAR_TEST(c, mask) \ ((opt.restrict_files_nonascii && !c_isascii ((unsigned char)(c))) || \ (filechr_table[(unsigned char)(c)] & (mask))) /* Shorthands for the table: */ #define U filechr_not_unix #define V filechr_not_vms #define W filechr_not_windows #define C filechr_control #define UVWC U|V|W|C #define UW U|W #define VC V|C #define VW V|W /* Table of characters unsafe under various conditions (see above). Arguably we could also claim `%' to be unsafe, since we use it as the escape character. If we ever want to be able to reliably translate file name back to URL, this would become important crucial. Right now, it's better to be minimal in escaping. */ static const unsigned char filechr_table[256] = { UVWC, VC, VC, VC, VC, VC, VC, VC, /* NUL SOH STX ETX EOT ENQ ACK BEL */ VC, VC, VC, VC, VC, VC, VC, VC, /* BS HT LF VT FF CR SO SI */ VC, VC, VC, VC, VC, VC, VC, VC, /* DLE DC1 DC2 DC3 DC4 NAK SYN ETB */ VC, VC, VC, VC, VC, VC, VC, VC, /* CAN EM SUB ESC FS GS RS US */ 0, 0, W, 0, 0, 0, 0, 0, /* SP ! " # $ % & ' */ 0, 0, VW, 0, 0, 0, 0, UW, /* ( ) * + , - . / */ 0, 0, 0, 0, 0, 0, 0, 0, /* 0 1 2 3 4 5 6 7 */ 0, 0, W, 0, W, 0, W, VW, /* 8 9 : ; < = > ? */ 0, 0, 0, 0, 0, 0, 0, 0, /* @ A B C D E F G */ 0, 0, 0, 0, 0, 0, 0, 0, /* H I J K L M N O */ 0, 0, 0, 0, 0, 0, 0, 0, /* P Q R S T U V W */ 0, 0, 0, 0, W, 0, 0, 0, /* X Y Z [ \ ] ^ _ */ 0, 0, 0, 0, 0, 0, 0, 0, /* ` a b c d e f g */ 0, 0, 0, 0, 0, 0, 0, 0, /* h i j k l m n o */ 0, 0, 0, 0, 0, 0, 0, 0, /* p q r s t u v w */ 0, 0, 0, 0, W, 0, 0, C, /* x y z { | } ~ DEL */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* 128-143 */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* 144-159 */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, }; #undef U #undef V #undef W #undef C #undef UW #undef UVWC #undef VC #undef VW /* FN_PORT_SEP is the separator between host and port in file names for non-standard port numbers. On Unix this is normally ':', as in "www.xemacs.org:4001/index.html". Under Windows, we set it to + because Windows can't handle ':' in file names. */ #define FN_PORT_SEP (opt.restrict_files_os != restrict_windows ? ':' : '+') /* FN_QUERY_SEP is the separator between the file name and the URL query, normally '?'. Because VMS and Windows cannot handle '?' in a file name, we use '@' instead there. */ #define FN_QUERY_SEP \ (((opt.restrict_files_os != restrict_vms) && \ (opt.restrict_files_os != restrict_windows)) ? '?' : '@') #define FN_QUERY_SEP_STR \ (((opt.restrict_files_os != restrict_vms) && \ (opt.restrict_files_os != restrict_windows)) ? "?" : "@") /* Quote path element, characters in [b, e), as file name, and append the quoted string to DEST. Each character is quoted as per file_unsafe_char and the corresponding table. If ESCAPED is true, the path element is considered to be URL-escaped and will be unescaped prior to inspection. */ static void append_uri_pathel (const char *b, const char *e, bool escaped, struct growable *dest) { const char *p; char buf[1024]; char *unescaped = NULL; int quoted, outlen; int mask; int max_length; if (!dest) return; if (opt.restrict_files_os == restrict_unix) mask = filechr_not_unix; else if (opt.restrict_files_os == restrict_vms) mask = filechr_not_vms; else mask = filechr_not_windows; if (opt.restrict_files_ctrl) mask |= filechr_control; /* Copy [b, e) to PATHEL and URL-unescape it. */ if (escaped) { size_t len = e - b; if (len < sizeof (buf)) unescaped = buf; else unescaped = xmalloc(len + 1); memcpy(unescaped, b, len); unescaped[len] = 0; url_unescape (unescaped); b = unescaped; e = unescaped + strlen (unescaped); } /* Defang ".." when found as component of path. Remember that path comes from the URL and might contain malicious input. */ if (e - b == 2 && b[0] == '.' && b[1] == '.') { b = "%2E%2E"; e = b + 6; } /* Walk the PATHEL string and check how many characters we'll need to quote. */ quoted = 0; for (p = b; p < e; p++) if (FILE_CHAR_TEST (*p, mask)) ++quoted; /* Calculate the length of the output string. e-b is the input string length. Each quoted char introduces two additional characters in the string, hence 2*quoted. */ outlen = (e - b) + (2 * quoted); # ifdef WINDOWS max_length = MAX_PATH; # else max_length = get_max_length(dest->base, dest->tail, _PC_NAME_MAX); # endif max_length -= CHOMP_BUFFER; if (max_length > 0 && outlen > max_length) { logprintf (LOG_NOTQUIET, "The destination name is too long (%d), reducing to %d\n", outlen, max_length); outlen = max_length; } GROW (dest, outlen); // This should not happen, but it's impossible to argue with static analysis that it can't happen // (in theory it can). So give static analyzers a hint. if (!dest->base) return; if (!quoted) { /* If there's nothing to quote, we can simply append the string without processing it again. */ memcpy (TAIL (dest), b, outlen); } else { char *q = TAIL (dest); int i; for (i = 0, p = b; p < e; p++) { if (!FILE_CHAR_TEST (*p, mask)) { if (i == outlen) break; *q++ = *p; i++; } else if (i + 3 > outlen) break; else { unsigned char ch = *p; *q++ = '%'; *q++ = XNUM_TO_DIGIT (ch >> 4); *q++ = XNUM_TO_DIGIT (ch & 0xf); i += 3; } } assert (q - TAIL (dest) <= outlen); } /* Perform inline case transformation if required. */ if (opt.restrict_files_case == restrict_lowercase || opt.restrict_files_case == restrict_uppercase) { char *q; for (q = TAIL (dest); q < TAIL (dest) + outlen; ++q) { if (opt.restrict_files_case == restrict_lowercase) *q = c_tolower (*q); else *q = c_toupper (*q); } } TAIL_INCR (dest, outlen); append_null (dest); if (unescaped && unescaped != buf) free (unescaped); } #ifdef HAVE_ICONV static char * convert_fname (char *fname) { char *converted_fname; const char *from_encoding = opt.encoding_remote; const char *to_encoding = opt.locale; iconv_t cd; size_t len, done, inlen, outlen; char *s; const char *orig_fname; /* Defaults for remote and local encodings. */ if (!from_encoding) from_encoding = "UTF-8"; if (!to_encoding) to_encoding = nl_langinfo (CODESET); cd = iconv_open (to_encoding, from_encoding); if (cd == (iconv_t) (-1)) { logprintf (LOG_VERBOSE, _ ("Conversion from %s to %s isn't supported\n"), quote_n (0, from_encoding), quote_n (1, to_encoding)); return fname; } orig_fname = fname; inlen = strlen (fname); len = outlen = inlen * 2; converted_fname = s = xmalloc (outlen + 1); done = 0; for (;;) { errno = 0; if (iconv (cd, (ICONV_CONST char **) &fname, &inlen, &s, &outlen) == 0 && iconv (cd, NULL, NULL, &s, &outlen) == 0) { *(converted_fname + len - outlen - done) = '\0'; iconv_close (cd); DEBUGP (("Converted file name '%s' (%s) -> '%s' (%s)\n", orig_fname, from_encoding, converted_fname, to_encoding)); xfree (orig_fname); return converted_fname; } /* Incomplete or invalid multibyte sequence */ if (errno == EINVAL || errno == EILSEQ || errno == 0) { if (errno) logprintf (LOG_VERBOSE, _ ("Incomplete or invalid multibyte sequence encountered\n")); else logprintf (LOG_VERBOSE, _ ("Unconvertable multibyte sequence encountered\n")); xfree (converted_fname); converted_fname = (char *) orig_fname; break; } else if (errno == E2BIG) /* Output buffer full */ { done = len; len = outlen = done + inlen * 2; converted_fname = xrealloc (converted_fname, outlen + 1); s = converted_fname + done; } else /* Weird, we got an unspecified error */ { logprintf (LOG_VERBOSE, _ ("Unhandled errno %d\n"), errno); xfree (converted_fname); converted_fname = (char *) orig_fname; break; } } DEBUGP (("Failed to convert file name '%s' (%s) -> '?' (%s)\n", orig_fname, from_encoding, to_encoding)); iconv_close (cd); return converted_fname; } #else static char * convert_fname (char *fname) { return fname; } #endif /* Append to DEST the directory structure that corresponds the directory part of URL's path. For example, if the URL is http://server/dir1/dir2/file, this appends "/dir1/dir2". Each path element ("dir1" and "dir2" in the above example) is examined, url-unescaped, and re-escaped as file name element. Additionally, it cuts as many directories from the path as specified by opt.cut_dirs. For example, if opt.cut_dirs is 1, it will produce "bar" for the above example. For 2 or more, it will produce "". Each component of the path is quoted for use as file name. */ static void append_dir_structure (const struct url *u, struct growable *dest) { char *pathel, *next; int cut = opt.cut_dirs; /* Go through the path components, de-URL-quote them, and quote them (if necessary) as file names. */ pathel = u->path; for (; (next = strchr (pathel, '/')) != NULL; pathel = next + 1) { if (cut-- > 0) continue; if (pathel == next) /* Ignore empty pathels. */ continue; if (dest->tail) append_char ('/', dest); append_uri_pathel (pathel, next, true, dest); } } /* Return a unique file name that matches the given URL as well as possible. Does not create directories on the file system. */ char * url_file_name (const struct url *u, char *replaced_filename) { struct growable fnres; /* stands for "file name result" */ struct growable temp_fnres; const char *u_file; char *fname, *unique, *fname_len_check; const char *index_filename = "index.html"; /* The default index file is index.html */ fnres.base = NULL; fnres.size = 0; fnres.tail = 0; temp_fnres.base = NULL; temp_fnres.size = 0; temp_fnres.tail = 0; /* If an alternative index file was defined, change index_filename */ if (opt.default_page) index_filename = opt.default_page; /* Start with the directory prefix, if specified. */ if (opt.dir_prefix) append_string (opt.dir_prefix, &fnres); /* If "dirstruct" is turned on (typically the case with -r), add the host and port (unless those have been turned off) and directory structure. */ /* All safe remote chars are unescaped and stored in temp_fnres, then converted to local and appended to fnres. Internationalized URL/IDN will produce punycode to lookup IP from DNS: https://en.wikipedia.org/wiki/URL https://en.wikipedia.org/wiki/Internationalized_domain_name Non-ASCII code chars in the path: https://en.wikipedia.org/wiki/List_of_Unicode_characters https://en.wikipedia.org/wiki/List_of_writing_systems */ if (opt.dirstruct) { if (opt.protocol_directories) { if (temp_fnres.tail) append_char ('/', &temp_fnres); append_string (supported_schemes[u->scheme].name, &temp_fnres); } if (opt.add_hostdir) { if (temp_fnres.tail) append_char ('/', &temp_fnres); if (0 != strcmp (u->host, "..")) append_string (u->host, &temp_fnres); else /* Host name can come from the network; malicious DNS may allow ".." to be resolved, causing us to write to "../". Defang such host names. */ append_string ("%2E%2E", &temp_fnres); if (u->port != scheme_default_port (u->scheme)) { char portstr[24]; number_to_string (portstr, u->port); append_char (FN_PORT_SEP, &temp_fnres); append_string (portstr, &temp_fnres); } } append_dir_structure (u, &temp_fnres); } if (!replaced_filename) { /* Create the filename. */ u_file = *u->file ? u->file : index_filename; /* Append "?query" to the file name, even if empty, * and create fname_len_check. */ if (u->query) fname_len_check = concat_strings (u_file, FN_QUERY_SEP_STR, u->query, NULL); else fname_len_check = strdupdelim (u_file, u_file + strlen (u_file)); } else { u_file = replaced_filename; fname_len_check = strdupdelim (u_file, u_file + strlen (u_file)); } if (temp_fnres.tail) append_char ('/', &temp_fnres); append_uri_pathel (fname_len_check, fname_len_check + strlen (fname_len_check), true, &temp_fnres); /* Zero-terminate the temporary file name. */ append_char ('\0', &temp_fnres); /* convert all remote chars before length check and appending to local path */ fname = convert_fname (temp_fnres.base); temp_fnres.base = NULL; temp_fnres.size = 0; temp_fnres.tail = 0; append_string (fname, &temp_fnres); xfree (fname); xfree (fname_len_check); /* The filename has already been 'cleaned' by append_uri_pathel() above. So, * just append it. */ if (fnres.tail) append_char ('/', &fnres); append_string (temp_fnres.base, &fnres); fname = fnres.base; /* Make a final check that the path length is acceptable? */ /* TODO: check fnres.base for path length problem */ xfree (temp_fnres.base); /* Check the cases in which the unique extensions are not used: 1) Clobbering is turned off (-nc). 2) Retrieval with regetting. 3) Timestamping is used. 4) Hierarchy is built. 5) Backups are specified. The exception is the case when file does exist and is a directory (see `mkalldirs' for explanation). */ if (ALLOW_CLOBBER && !(file_exists_p (fname, NULL) && !file_non_directory_p (fname))) { unique = fname; } else { unique = unique_name_passthrough (fname); if (unique != fname) xfree (fname); } /* On VMS, alter the name as required. */ #ifdef __VMS { char *unique2; unique2 = ods_conform( unique); if (unique2 != unique) { xfree (unique); unique = unique2; } } #endif /* def __VMS */ return unique; } /* Resolve "." and ".." elements of PATH by destructively modifying PATH and return true if PATH has been modified, false otherwise. The algorithm is in spirit similar to the one described in rfc1808, although implemented differently, in one pass. To recap, path elements containing only "." are removed, and ".." is taken to mean "back up one element". Single leading and trailing slashes are preserved. For example, "a/b/c/./../d/.." will yield "a/b/". More exhaustive test examples are provided below. If you change anything in this function, run test_path_simplify to make sure you haven't broken a test case. */ static bool path_simplify (enum url_scheme scheme, char *path) { char *h = path; /* hare */ char *t = path; /* tortoise */ char *beg = path; char *end = strchr (path, '\0'); while (h < end) { /* Hare should be at the beginning of a path element. */ if (h[0] == '.' && (h[1] == '/' || h[1] == '\0')) { /* Ignore "./". */ h += 2; } else if (h[0] == '.' && h[1] == '.' && (h[2] == '/' || h[2] == '\0')) { /* Handle "../" by retreating the tortoise by one path element -- but not past beginning. */ if (t > beg) { /* Move backwards until T hits the beginning of the previous path element or the beginning of path. */ for (--t; t > beg && t[-1] != '/'; t--) ; } else if (scheme == SCHEME_FTP #ifdef HAVE_SSL || scheme == SCHEME_FTPS #endif ) { /* If we're at the beginning, copy the "../" literally and move the beginning so a later ".." doesn't remove it. This violates RFC 3986; but we do it for FTP anyway because there is otherwise no way to get at a parent directory, when the FTP server drops us in a non-root directory (which is not uncommon). */ beg = t + 3; goto regular; } h += 3; } else { regular: /* A regular path element. If H hasn't advanced past T, simply skip to the next path element. Otherwise, copy the path element until the next slash. */ if (t == h) { /* Skip the path element, including the slash. */ while (h < end && *h != '/') t++, h++; if (h < end) t++, h++; } else { /* Copy the path element, including the final slash. */ while (h < end && *h != '/') *t++ = *h++; if (h < end) *t++ = *h++; } } } if (t != h) *t = '\0'; return t != h; } /* Return the length of URL's path. Path is considered to be terminated by one or more of the ?query or ;params or #fragment, depending on the scheme. */ static const char * path_end (const char *url) { enum url_scheme scheme = url_scheme (url); const char *seps; if (scheme == SCHEME_INVALID) scheme = SCHEME_HTTP; /* use http semantics for rel links */ /* +2 to ignore the first two separators ':' and '/' */ seps = init_seps (scheme) + 2; return strpbrk_or_eos (url, seps); } /* Find the last occurrence of character C in the range [b, e), or NULL, if none are present. */ #define find_last_char(b, e, c) memrchr ((b), (c), (e) - (b)) /* Merge BASE with LINK and return the resulting URI. Either of the URIs may be absolute or relative, complete with the host name, or path only. This tries to reasonably handle all foreseeable cases. It only employs minimal URL parsing, without knowledge of the specifics of schemes. I briefly considered making this function call path_simplify after the merging process, as rfc1738 seems to suggest. This is a bad idea for several reasons: 1) it complexifies the code, and 2) url_parse has to simplify path anyway, so it's wasteful to boot. */ char * uri_merge (const char *base, const char *link) { int linklength; const char *end; char *merge; if (url_has_scheme (link)) return xstrdup (link); /* We may not examine BASE past END. */ end = path_end (base); linklength = strlen (link); if (!*link) { /* Empty LINK points back to BASE, query string and all. */ return xstrdup (base); } else if (*link == '?') { /* LINK points to the same location, but changes the query string. Examples: */ /* uri_merge("path", "?new") -> "path?new" */ /* uri_merge("path?foo", "?new") -> "path?new" */ /* uri_merge("path?foo#bar", "?new") -> "path?new" */ /* uri_merge("path#foo", "?new") -> "path?new" */ int baselength = end - base; merge = xmalloc (baselength + linklength + 1); memcpy (merge, base, baselength); memcpy (merge + baselength, link, linklength); merge[baselength + linklength] = '\0'; } else if (*link == '#') { /* uri_merge("path", "#new") -> "path#new" */ /* uri_merge("path#foo", "#new") -> "path#new" */ /* uri_merge("path?foo", "#new") -> "path?foo#new" */ /* uri_merge("path?foo#bar", "#new") -> "path?foo#new" */ int baselength; const char *end1 = strchr (base, '#'); if (!end1) end1 = base + strlen (base); baselength = end1 - base; merge = xmalloc (baselength + linklength + 1); memcpy (merge, base, baselength); memcpy (merge + baselength, link, linklength); merge[baselength + linklength] = '\0'; } else if (*link == '/' && *(link + 1) == '/') { /* LINK begins with "//" and so is a net path: we need to replace everything after (and including) the double slash with LINK. */ /* uri_merge("foo", "//new/bar") -> "//new/bar" */ /* uri_merge("//old/foo", "//new/bar") -> "//new/bar" */ /* uri_merge("http://old/foo", "//new/bar") -> "http://new/bar" */ int span; const char *slash; const char *start_insert; /* Look for first slash. */ slash = memchr (base, '/', end - base); /* If found slash and it is a double slash, then replace from this point, else default to replacing from the beginning. */ if (slash && *(slash + 1) == '/') start_insert = slash; else start_insert = base; span = start_insert - base; merge = xmalloc (span + linklength + 1); if (span) memcpy (merge, base, span); memcpy (merge + span, link, linklength); merge[span + linklength] = '\0'; } else if (*link == '/') { /* LINK is an absolute path: we need to replace everything after (and including) the FIRST slash with LINK. So, if BASE is "http://host/whatever/foo/bar", and LINK is "/qux/xyzzy", our result should be "http://host/qux/xyzzy". */ int span; const char *slash; const char *start_insert = NULL; /* for gcc to shut up. */ const char *pos = base; bool seen_slash_slash = false; /* We're looking for the first slash, but want to ignore double slash. */ again: slash = memchr (pos, '/', end - pos); if (slash && !seen_slash_slash) if (*(slash + 1) == '/') { pos = slash + 2; seen_slash_slash = true; goto again; } /* At this point, SLASH is the location of the first / after "//", or the first slash altogether. START_INSERT is the pointer to the location where LINK will be inserted. When examining the last two examples, keep in mind that LINK begins with '/'. */ if (!slash && !seen_slash_slash) /* example: "foo" */ /* ^ */ start_insert = base; else if (!slash && seen_slash_slash) /* example: "http://foo" */ /* ^ */ start_insert = end; else if (slash && !seen_slash_slash) /* example: "foo/bar" */ /* ^ */ start_insert = base; else if (slash && seen_slash_slash) /* example: "http://something/" */ /* ^ */ start_insert = slash; span = start_insert - base; merge = xmalloc (span + linklength + 1); if (span) memcpy (merge, base, span); memcpy (merge + span, link, linklength); merge[span + linklength] = '\0'; } else { /* LINK is a relative URL: we need to replace everything after last slash (possibly empty) with LINK. So, if BASE is "whatever/foo/bar", and LINK is "qux/xyzzy", our result should be "whatever/foo/qux/xyzzy". */ bool need_explicit_slash = false; int span; const char *start_insert; const char *last_slash = find_last_char (base, end, '/'); if (!last_slash) { /* No slash found at all. Replace what we have with LINK. */ start_insert = base; } else if (last_slash && last_slash >= base + 2 && last_slash[-2] == ':' && last_slash[-1] == '/') { /* example: http://host" */ /* ^ */ start_insert = end + 1; need_explicit_slash = true; } else { /* example: "whatever/foo/bar" */ /* ^ */ start_insert = last_slash + 1; } span = start_insert - base; merge = xmalloc (span + linklength + 1); if (span) memcpy (merge, base, span); if (need_explicit_slash) merge[span - 1] = '/'; memcpy (merge + span, link, linklength); merge[span + linklength] = '\0'; } return merge; } #define APPEND(p, s) do { \ int len = strlen (s); \ memcpy (p, s, len); \ p += len; \ } while (0) /* Use this instead of password when the actual password is supposed to be hidden. We intentionally use a generic string without giving away the number of characters in the password, like previous versions did. */ #define HIDDEN_PASSWORD "*password*" /* Recreate the URL string from the data in URL. If HIDE is true (as it is when we're calling this on a URL we plan to print, but not when calling it to canonicalize a URL for use within the program), password will be hidden. Unsafe characters in the URL will be quoted. */ char * url_string (const struct url *url, enum url_auth_mode auth_mode) { int size; char *result, *p; char *quoted_host, *quoted_user = NULL, *quoted_passwd = NULL; int scheme_port = supported_schemes[url->scheme].default_port; const char *scheme_str = supported_schemes[url->scheme].leading_string; int fplen = full_path_length (url); bool brackets_around_host; assert (scheme_str != NULL); /* Make sure the user name and password are quoted. */ if (url->user) { if (auth_mode != URL_AUTH_HIDE) { quoted_user = url_escape_allow_passthrough (url->user); if (url->passwd) { if (auth_mode == URL_AUTH_HIDE_PASSWD) quoted_passwd = (char *) HIDDEN_PASSWORD; else quoted_passwd = url_escape_allow_passthrough (url->passwd); } } } /* In the unlikely event that the host name contains non-printable characters, quote it for displaying to the user. */ quoted_host = url_escape_allow_passthrough (url->host); /* Undo the quoting of colons that URL escaping performs. IPv6 addresses may legally contain colons, and in that case must be placed in square brackets. */ if (quoted_host != url->host) unescape_single_char (quoted_host, ':'); brackets_around_host = strchr (quoted_host, ':') != NULL; size = (strlen (scheme_str) + strlen (quoted_host) + (brackets_around_host ? 2 : 0) + fplen + 1); if (url->port != scheme_port) size += 1 + numdigit (url->port); if (quoted_user) { size += 1 + strlen (quoted_user); if (quoted_passwd) size += 1 + strlen (quoted_passwd); } p = result = xmalloc (size); APPEND (p, scheme_str); if (quoted_user) { APPEND (p, quoted_user); if (quoted_passwd) { *p++ = ':'; APPEND (p, quoted_passwd); } *p++ = '@'; } if (brackets_around_host) *p++ = '['; APPEND (p, quoted_host); if (brackets_around_host) *p++ = ']'; if (url->port != scheme_port) { *p++ = ':'; p = number_to_string (p, url->port); } full_path_write (url, p); p += fplen; *p++ = '\0'; assert (p - result == size); if (quoted_user && quoted_user != url->user) xfree (quoted_user); if (quoted_passwd && auth_mode == URL_AUTH_SHOW && quoted_passwd != url->passwd) xfree (quoted_passwd); if (quoted_host != url->host) xfree (quoted_host); return result; } /* Return true if scheme a is similar to scheme b. Schemes are similar if they are equal. If SSL is supported, schemes are also similar if one is http (SCHEME_HTTP) and the other is https (SCHEME_HTTPS). */ bool schemes_are_similar_p (enum url_scheme a, enum url_scheme b) { if (a == b) return true; #ifdef HAVE_SSL if ((a == SCHEME_HTTP && b == SCHEME_HTTPS) || (a == SCHEME_HTTPS && b == SCHEME_HTTP)) return true; #endif return false; } static int getchar_from_escaped_string (const char *str, char *c) { const char *p = str; assert (str && *str); assert (c); if (p[0] == '%') { if (!c_isxdigit(p[1]) || !c_isxdigit(p[2])) { *c = '%'; return 1; } else { if (p[2] == 0) return 0; /* error: invalid string */ *c = X2DIGITS_TO_NUM (p[1], p[2]); if (URL_RESERVED_CHAR(*c)) { *c = '%'; return 1; } else return 3; } } else { *c = p[0]; } return 1; } bool are_urls_equal (const char *u1, const char *u2) { const char *p, *q; int pp, qq; char ch1, ch2; assert(u1 && u2); p = u1; q = u2; while (*p && *q && (pp = getchar_from_escaped_string (p, &ch1)) && (qq = getchar_from_escaped_string (q, &ch2)) && (c_tolower(ch1) == c_tolower(ch2))) { p += pp; q += qq; } return (*p == 0 && *q == 0 ? true : false); } #ifdef TESTING /* Debugging and testing support for path_simplify. */ #if 0 /* Debug: run path_simplify on PATH and return the result in a new string. Useful for calling from the debugger. */ static char * ps (char *path) { char *copy = xstrdup (path); path_simplify (copy); return copy; } #endif static const char * run_test (const char *test, const char *expected_result, enum url_scheme scheme, bool expected_change) { char *test_copy = xstrdup (test); bool modified = path_simplify (scheme, test_copy); if (0 != strcmp (test_copy, expected_result)) { printf ("Failed path_simplify(\"%s\"): expected \"%s\", got \"%s\".\n", test, expected_result, test_copy); mu_assert ("", 0); } if (modified != expected_change) { if (expected_change) printf ("Expected modification with path_simplify(\"%s\").\n", test); else printf ("Expected no modification with path_simplify(\"%s\").\n", test); } xfree (test_copy); mu_assert ("", modified == expected_change); return NULL; } const char * test_path_simplify (void) { static const struct { const char *test, *result; enum url_scheme scheme; bool should_modify; } tests[] = { { "", "", SCHEME_HTTP, false }, { ".", "", SCHEME_HTTP, true }, { "./", "", SCHEME_HTTP, true }, { "..", "", SCHEME_HTTP, true }, { "../", "", SCHEME_HTTP, true }, { "..", "..", SCHEME_FTP, false }, { "../", "../", SCHEME_FTP, false }, { "foo", "foo", SCHEME_HTTP, false }, { "foo/bar", "foo/bar", SCHEME_HTTP, false }, { "foo///bar", "foo///bar", SCHEME_HTTP, false }, { "foo/.", "foo/", SCHEME_HTTP, true }, { "foo/./", "foo/", SCHEME_HTTP, true }, { "foo./", "foo./", SCHEME_HTTP, false }, { "foo/../bar", "bar", SCHEME_HTTP, true }, { "foo/../bar/", "bar/", SCHEME_HTTP, true }, { "foo/bar/..", "foo/", SCHEME_HTTP, true }, { "foo/bar/../x", "foo/x", SCHEME_HTTP, true }, { "foo/bar/../x/", "foo/x/", SCHEME_HTTP, true }, { "foo/..", "", SCHEME_HTTP, true }, { "foo/../..", "", SCHEME_HTTP, true }, { "foo/../../..", "", SCHEME_HTTP, true }, { "foo/../../bar/../../baz", "baz", SCHEME_HTTP, true }, { "foo/../..", "..", SCHEME_FTP, true }, { "foo/../../..", "../..", SCHEME_FTP, true }, { "foo/../../bar/../../baz", "../../baz", SCHEME_FTP, true }, { "a/b/../../c", "c", SCHEME_HTTP, true }, { "./a/../b", "b", SCHEME_HTTP, true } }; unsigned i; for (i = 0; i < countof (tests); i++) { const char *message; const char *test = tests[i].test; const char *expected_result = tests[i].result; enum url_scheme scheme = tests[i].scheme; bool expected_change = tests[i].should_modify; message = run_test (test, expected_result, scheme, expected_change); if (message) return message; } return NULL; } const char * test_append_uri_pathel(void) { unsigned i; static const struct { const char *original_url; const char *input; bool escaped; const char *expected_result; } test_array[] = { { "http://www.yoyodyne.com/path/", "somepage.html", false, "http://www.yoyodyne.com/path/somepage.html" }, }; for (i = 0; i < countof(test_array); ++i) { struct growable dest; const char *p = test_array[i].input; memset (&dest, 0, sizeof (dest)); append_string (test_array[i].original_url, &dest); append_uri_pathel (p, p + strlen(p), test_array[i].escaped, &dest); mu_assert ("test_append_uri_pathel: wrong result", strcmp (dest.base, test_array[i].expected_result) == 0); xfree (dest.base); } return NULL; } const char * test_are_urls_equal(void) { unsigned i; static const struct { const char *url1; const char *url2; bool expected_result; } test_array[] = { { "http://www.adomain.com/apath/", "http://www.adomain.com/apath/", true }, { "http://www.adomain.com/apath/", "http://www.adomain.com/anotherpath/", false }, { "http://www.adomain.com/apath/", "http://www.anotherdomain.com/path/", false }, { "http://www.adomain.com/~path/", "http://www.adomain.com/%7epath/", true }, { "http://www.adomain.com/longer-path/", "http://www.adomain.com/path/", false }, { "http://www.adomain.com/path%2f", "http://www.adomain.com/path/", false }, }; for (i = 0; i < countof(test_array); ++i) { mu_assert ("test_are_urls_equal: wrong result", are_urls_equal (test_array[i].url1, test_array[i].url2) == test_array[i].expected_result); } return NULL; } #endif /* TESTING */ /* * vim: et ts=2 sw=2 */ wget-1.21.2/src/connect.h0000644000000000000000000000531314115732710012026 00000000000000/* Declarations for connect. Copyright (C) 1996-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #ifndef CONNECT_H #define CONNECT_H #include "host.h" /* for definition of ip_address */ /* Function declarations */ /* Returned by connect_to_host when host name cannot be resolved. */ enum { E_HOST = -100 }; int connect_to_host (const char *, int); int connect_to_ip (const ip_address *, int, const char *); int bind_local (const ip_address *, int *); int accept_connection (int); enum { ENDPOINT_LOCAL, ENDPOINT_PEER }; bool socket_ip_address (int, ip_address *, int); int socket_family (int sock, int endpoint); bool retryable_socket_connect_error (int); /* Flags for select_fd's WAIT_FOR argument. */ enum { WAIT_FOR_READ = 1, WAIT_FOR_WRITE = 2 }; int select_fd (int, double, int); bool test_socket_open (int); struct transport_implementation { int (*reader) (int, char *, int, void *, double); int (*writer) (int, char *, int, void *); int (*poller) (int, double, int, void *); int (*peeker) (int, char *, int, void *, double); const char *(*errstr) (int, void *); void (*closer) (int, void *); }; void fd_register_transport (int, struct transport_implementation *, void *); void *fd_transport_context (int); int fd_read (int, char *, int, double); int fd_write (int, char *, int, double); int fd_peek (int, char *, int, double); const char *fd_errstr (int); void fd_close (int); void connect_cleanup (void); #ifdef WINDOWS int select_fd_nb (int, double, int); #else #define select_fd_nb select_fd #endif #endif /* CONNECT_H */ wget-1.21.2/src/res.c0000644000000000000000000004476514115732710011177 00000000000000/* Support for Robot Exclusion Standard (RES). Copyright (C) 2001, 2006-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of Wget. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ /* This file implements the Robot Exclusion Standard (RES). RES is a simple protocol that enables site admins to signalize to the web crawlers that certain parts of the site should not be accessed. All the admin needs to do is create a "robots.txt" file in the web server root, and use simple commands to allow or disallow access to certain parts of the site. The first specification was written by Martijn Koster in 1994, and is still available at . In 1996, Martijn wrote an Internet Draft specifying an improved RES specification; however, that work was apparently abandoned since the draft has expired in 1997 and hasn't been replaced since. The draft is available at . This file implements RES as specified by the draft. Note that this only handles the "robots.txt" support. The META tag that controls whether the links should be followed is handled in `html-url.c'. Known deviations: * The end-of-line comment recognition is more in the spirit of the Bourne Shell (as specified by RES-1994). That means that "foo#bar" is taken literally, whereas "foo #bar" is interpreted as "foo". The Draft apparently specifies that both should be interpreted as "foo". * We don't recognize sole CR as the line ending. * We don't implement expiry mechanism for /robots.txt specs. I consider it non-necessary for a relatively short-lived application such as Wget. Besides, it is highly questionable whether anyone deploys the recommended expiry scheme for robots.txt. Entry points are functions res_parse, res_parse_from_file, res_match_path, res_register_specs, res_get_specs, and res_retrieve_file. */ #include "wget.h" #include #include #include #include #include #include "utils.h" #include "hash.h" #include "url.h" #include "retr.h" #include "res.h" #include "c-strcase.h" #ifdef TESTING #include "../tests/unit-tests.h" #endif struct path_info { char *path; bool allowedp; bool user_agent_exact_p; }; struct robot_specs { int count; int size; struct path_info *paths; }; /* Parsing the robot spec. */ /* Check whether AGENT (a string of length LENGTH) equals "wget" or "*". If it is either of them, *matches is set to one. If it is "wget", *exact_match is set to one. */ static void match_user_agent (const char *agent, int length, bool *matches, bool *exact_match) { if (length == 1 && *agent == '*') { *matches = true; *exact_match = false; } else if (BOUNDED_EQUAL_NO_CASE (agent, agent + length, "wget")) { *matches = true; *exact_match = true; } else { *matches = false; *exact_match = false; } } /* Add a path specification between PATH_B and PATH_E as one of the paths in SPECS. */ static void add_path (struct robot_specs *specs, const char *path_b, const char *path_e, bool allowedp, bool exactp) { struct path_info pp; if (path_b < path_e && *path_b == '/') /* Our path representation doesn't use a leading slash, so remove one from theirs. */ ++path_b; pp.path = strdupdelim (path_b, path_e); pp.allowedp = allowedp; pp.user_agent_exact_p = exactp; ++specs->count; if (specs->count > specs->size) { if (specs->size == 0) specs->size = 1; else specs->size <<= 1; specs->paths = xrealloc (specs->paths, specs->size * sizeof (struct path_info)); } specs->paths[specs->count - 1] = pp; } /* Recreate SPECS->paths with only those paths that have user_agent_exact_p set to true. */ static void prune_non_exact (struct robot_specs *specs) { struct path_info *newpaths; int i, j, cnt; cnt = 0; for (i = 0; i < specs->count; i++) if (specs->paths[i].user_agent_exact_p) ++cnt; newpaths = xnew_array (struct path_info, cnt); for (i = 0, j = 0; i < specs->count; i++) if (specs->paths[i].user_agent_exact_p) newpaths[j++] = specs->paths[i]; else xfree (specs->paths[i].path); assert (j == cnt); xfree (specs->paths); specs->paths = newpaths; specs->count = cnt; specs->size = cnt; } #define EOL(p) ((p) >= lineend) #define SKIP_SPACE(p) do { \ while (!EOL (p) && c_isspace (*p)) \ ++p; \ } while (0) #define FIELD_IS(string_literal) \ BOUNDED_EQUAL_NO_CASE (field_b, field_e, string_literal) /* Parse textual RES specs beginning with SOURCE of length LENGTH. Return a specs objects ready to be fed to res_match_path. The parsing itself is trivial, but creating a correct SPECS object is trickier than it seems, because RES is surprisingly byzantine if you attempt to implement it correctly. A "record" is a block of one or more `User-Agent' lines followed by one or more `Allow' or `Disallow' lines. Record is accepted by Wget if one of the `User-Agent' lines was "wget", or if the user agent line was "*". After all the lines have been read, we examine whether an exact ("wget") user-agent field was specified. If so, we delete all the lines read under "User-Agent: *" blocks because we have our own Wget-specific blocks. This enables the admin to say: User-Agent: * Disallow: / User-Agent: google User-Agent: wget Disallow: /cgi-bin This means that to Wget and to Google, /cgi-bin is disallowed, whereas for all other crawlers, everything is disallowed. res_parse is implemented so that the order of records doesn't matter. In the case above, the "User-Agent: *" could have come after the other one. */ struct robot_specs * res_parse (const char *source, int length) { int line_count = 1; const char *p = source; const char *end = source + length; /* true if last applicable user-agent field matches Wget. */ bool user_agent_applies = false; /* true if last applicable user-agent field *exactly* matches Wget. */ bool user_agent_exact = false; /* whether we ever encountered exact user agent. */ bool found_exact = false; /* count of allow/disallow lines in the current "record", i.e. after the last `user-agent' instructions. */ int record_count = 0; struct robot_specs *specs = xnew0 (struct robot_specs); while (1) { const char *lineend, *lineend_real; const char *field_b, *field_e; const char *value_b, *value_e; if (p == end) break; lineend_real = memchr (p, '\n', end - p); if (lineend_real) ++lineend_real; else lineend_real = end; lineend = lineend_real; /* Before doing anything else, check whether the line is empty or comment-only. */ SKIP_SPACE (p); if (EOL (p) || *p == '#') goto next; /* Make sure the end-of-line comments are respected by setting lineend to a location preceding the first comment. Real line ending remains in lineend_real. */ for (lineend = p; lineend < lineend_real; lineend++) if ((lineend == p || c_isspace (*(lineend - 1))) && *lineend == '#') break; /* Ignore trailing whitespace in the same way. */ while (lineend > p && c_isspace (*(lineend - 1))) --lineend; assert (!EOL (p)); field_b = p; while (!EOL (p) && (c_isalnum (*p) || *p == '-')) ++p; field_e = p; SKIP_SPACE (p); if (field_b == field_e || EOL (p) || *p != ':') { DEBUGP (("Ignoring malformed line %d\n", line_count)); goto next; } ++p; /* skip ':' */ SKIP_SPACE (p); value_b = p; while (!EOL (p)) ++p; value_e = p; /* Finally, we have a syntactically valid line. */ if (FIELD_IS ("user-agent")) { /* We have to support several cases: --previous records-- User-Agent: foo User-Agent: Wget User-Agent: bar ... matching record ... User-Agent: baz User-Agent: qux ... non-matching record ... User-Agent: * ... matching record, but will be pruned later ... We have to respect `User-Agent' at the beginning of each new record simply because we don't know if we're going to encounter "Wget" among the agents or not. Hence, match_user_agent is called when record_count != 0. But if record_count is 0, we have to keep calling it until it matches, and if that happens, we must not call it any more, until the next record. Hence the other part of the condition. */ if (record_count != 0 || user_agent_applies == false) match_user_agent (value_b, value_e - value_b, &user_agent_applies, &user_agent_exact); if (user_agent_exact) found_exact = true; record_count = 0; } else if (FIELD_IS ("allow")) { if (user_agent_applies) { add_path (specs, value_b, value_e, true, user_agent_exact); } ++record_count; } else if (FIELD_IS ("disallow")) { if (user_agent_applies) { bool allowed = false; if (value_b == value_e) /* Empty "disallow" line means everything is *allowed*! */ allowed = true; add_path (specs, value_b, value_e, allowed, user_agent_exact); } ++record_count; } else { DEBUGP (("Ignoring unknown field at line %d\n", line_count)); goto next; } next: p = lineend_real; ++line_count; } if (found_exact) { /* We've encountered an exactly matching user-agent. Throw out all the stuff with user-agent: *. */ prune_non_exact (specs); } else if (specs->size > specs->count) { /* add_path normally over-allocates specs->paths. Reallocate it to the correct size in order to conserve some memory. */ specs->paths = xrealloc (specs->paths, specs->count * sizeof (struct path_info)); specs->size = specs->count; } return specs; } /* The same like res_parse, but first map the FILENAME into memory, and then parse it. */ struct robot_specs * res_parse_from_file (const char *filename) { struct robot_specs *specs; struct file_memory *fm = wget_read_file (filename); if (!fm) { logprintf (LOG_NOTQUIET, _("Cannot open %s: %s\n"), filename, strerror (errno)); return NULL; } specs = res_parse (fm->content, fm->length); wget_read_file_free (fm); return specs; } static void free_specs (struct robot_specs *specs) { int i; for (i = 0; i < specs->count; i++) xfree (specs->paths[i].path); xfree (specs->paths); xfree (specs); } /* Matching of a path according to the specs. */ /* If C is '%' and (ptr[1], ptr[2]) form a hexadecimal number, and if that number is not a numerical representation of '/', decode C and advance the pointer. */ #define DECODE_MAYBE(c, ptr) do { \ if (c == '%' && c_isxdigit (ptr[1]) && c_isxdigit (ptr[2])) \ { \ unsigned char decoded = X2DIGITS_TO_NUM (ptr[1], ptr[2]); \ if (decoded != '/') \ { \ c = decoded; \ ptr += 2; \ } \ } \ } while (0) /* The inner matching engine: return true if RECORD_PATH matches URL_PATH. The rules for matching are described at , section 3.2.2. */ static bool matches (const char *record_path, const char *url_path) { const char *rp = record_path; const char *up = url_path; for (; ; ++rp, ++up) { char rc = *rp; char uc = *up; if (!rc) return true; if (!uc) return false; DECODE_MAYBE(rc, rp); DECODE_MAYBE(uc, up); if (rc != uc) return false; } } /* Iterate through all paths in SPECS. For the first one that matches, return its allow/reject status. If none matches, retrieval is by default allowed. */ bool res_match_path (const struct robot_specs *specs, const char *path) { int i; if (!specs) return true; for (i = 0; i < specs->count; i++) if (matches (specs->paths[i].path, path)) { bool allowedp = specs->paths[i].allowedp; DEBUGP (("%s path %s because of rule %s.\n", allowedp ? "Allowing" : "Rejecting", path, quote (specs->paths[i].path))); return allowedp; } return true; } /* Registering the specs. */ static struct hash_table *registered_specs; /* Register RES specs that below to server on HOST:PORT. They will later be retrievable using res_get_specs. */ void res_register_specs (const char *host, int port, struct robot_specs *specs) { struct robot_specs *old; char buf[256], *hp, *hp_old; if (((unsigned) snprintf (buf, sizeof (buf), "%s:%d", host, port)) >= sizeof (buf)) hp = aprintf("%s:%d", host, port); else hp = buf; if (!registered_specs) registered_specs = make_nocase_string_hash_table (0); if (hash_table_get_pair (registered_specs, hp, &hp_old, &old)) { if (hp != buf) xfree (hp); if (old) free_specs (old); hash_table_put (registered_specs, hp_old, specs); } else { hash_table_put (registered_specs, hp == buf ? xstrdup (hp) : hp, specs); } } /* Get the specs that belong to HOST:PORT. */ struct robot_specs * res_get_specs (const char *host, int port) { char buf[256], *hp; if (!registered_specs) return NULL; if (((unsigned) snprintf (buf, sizeof (buf), "%s:%d", host, port)) >= sizeof (buf)) hp = aprintf("%s:%d", host, port); else hp = buf; return hash_table_get (registered_specs, hp); } /* Loading the robots file. */ #define RES_SPECS_LOCATION "/robots.txt" /* Retrieve the robots.txt from the server root of the server that serves URL. The file will be named according to the currently active rules, and the file name will be returned in *file. Return true if robots were retrieved OK, false otherwise. */ bool res_retrieve_file (const char *url, char **file, struct iri *iri) { struct iri *i = iri_new (); uerr_t err; char *robots_url = uri_merge (url, RES_SPECS_LOCATION); int saved_ts_val = opt.timestamping; int saved_sp_val = opt.spider, url_err; struct url * url_parsed; /* Copy server URI encoding for a possible IDNA transformation, no need to encode the full URI in UTF-8 because "robots.txt" is plain ASCII */ set_uri_encoding (i, iri->uri_encoding, false); i->utf8_encode = false; logputs (LOG_VERBOSE, _("Loading robots.txt; please ignore errors.\n")); *file = NULL; opt.timestamping = false; opt.spider = false; url_parsed = url_parse (robots_url, &url_err, i, true); if (!url_parsed) { char *error = url_error (robots_url, url_err); logprintf (LOG_NOTQUIET, "%s: %s.\n", robots_url, error); xfree (error); err = URLERROR; } else { err = retrieve_url (url_parsed, robots_url, file, NULL, NULL, NULL, false, i, false); url_free(url_parsed); } opt.timestamping = saved_ts_val; opt.spider = saved_sp_val; xfree (robots_url); iri_free (i); if (err != RETROK && *file != NULL) { /* If the file is not retrieved correctly, but retrieve_url allocated the file name, deallocate is here so that the caller doesn't have to worry about it. */ xfree (*file); } return err == RETROK; } bool is_robots_txt_url (const char *url) { char *robots_url = uri_merge (url, RES_SPECS_LOCATION); bool ret = are_urls_equal (url, robots_url); xfree (robots_url); return ret; } #if defined DEBUG_MALLOC || defined TESTING void res_cleanup (void) { if (registered_specs) { hash_table_iterator iter; for (hash_table_iterate (registered_specs, &iter); hash_table_iter_next (&iter); ) { xfree (iter.key); free_specs (iter.value); } hash_table_destroy (registered_specs); registered_specs = NULL; } } #endif #ifdef TESTING const char * test_is_robots_txt_url(void) { unsigned i; static const struct { const char *url; bool expected_result; } test_array[] = { { "http://www.yoyodyne.com/robots.txt", true }, { "http://www.yoyodyne.com/somepath/", false }, { "http://www.yoyodyne.com/somepath/robots.txt", false }, }; for (i = 0; i < countof(test_array); ++i) { mu_assert ("test_is_robots_txt_url: wrong result", is_robots_txt_url (test_array[i].url) == test_array[i].expected_result); } return NULL; } #endif /* TESTING */ /* * vim: et ts=2 sw=2 */ wget-1.21.2/src/convert.h0000644000000000000000000001056414115732710012061 00000000000000/* Declarations for convert.c Copyright (C) 2003-2006, 2009-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #ifndef CONVERT_H #define CONVERT_H struct hash_table; /* forward decl */ extern struct hash_table *dl_url_file_map; extern struct hash_table *downloaded_html_set; extern struct hash_table *downloaded_css_set; enum convert_options { CO_NOCONVERT = 0, /* don't convert this URL */ CO_CONVERT_TO_RELATIVE, /* convert to relative, e.g. to "../../otherdir/foo.gif" */ CO_CONVERT_BASENAME_ONLY, /* convert the file portion only (basename) leaving the rest of the URL unchanged */ CO_CONVERT_TO_COMPLETE, /* convert to absolute, e.g. to "http://orighost/somedir/bar.jpg". */ CO_NULLIFY_BASE /* change to empty string. */ }; struct url; /* A structure that defines the whereabouts of a URL, i.e. its position in an HTML document, etc. */ struct urlpos { struct url *url; /* the URL of the link, after it has been merged with the base */ char *local_name; /* local file to which it was saved (used by convert_links) */ /* reserved for special links such as which are used when converting links, but ignored when downloading. */ unsigned int ignore_when_downloading :1; /* Information about the original link: */ unsigned int link_relative_p :1; /* the link was relative */ unsigned int link_complete_p :1; /* the link was complete (had host name) */ unsigned int link_base_p :1; /* the url came from */ unsigned int link_inline_p :1; /* needed to render the page */ unsigned int link_css_p :1; /* the url came from CSS */ unsigned int link_noquote_html_p :1; /* from HTML, but doesn't need " */ unsigned int link_expect_html :1; /* expected to contain HTML */ unsigned int link_expect_css :1; /* expected to contain CSS */ unsigned int link_refresh_p :1; /* link was received from */ int refresh_timeout; /* for reconstructing the refresh. */ /* Conversion requirements: */ enum convert_options convert; /* is conversion required? */ /* URL's position in the buffer. */ int pos, size; struct urlpos *next; /* next list element */ }; /* downloaded_file() takes a parameter of this type and returns this type. */ typedef enum { /* Return enumerators: */ FILE_NOT_ALREADY_DOWNLOADED = 0, /* Return / parameter enumerators: */ FILE_DOWNLOADED_NORMALLY, FILE_DOWNLOADED_AND_HTML_EXTENSION_ADDED, /* Parameter enumerators: */ CHECK_FOR_FILE } downloaded_file_t; downloaded_file_t downloaded_file (downloaded_file_t, const char *); void register_download (const char *, const char *); void register_redirection (const char *, const char *); void register_html (const char *); void register_css (const char *); void register_delete_file (const char *); void convert_all_links (void); void convert_cleanup (void); char *html_quote_string (const char *); #endif /* CONVERT_H */ wget-1.21.2/src/options.h0000644000000000000000000003624714115732710012102 00000000000000/* struct options. Copyright (C) 1996-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ enum CHECK_CERT_MODES { CHECK_CERT_OFF, CHECK_CERT_ON, CHECK_CERT_QUIET }; struct options { int verbose; /* Are we verbose? (First set to -1, hence not boolean.) */ bool quiet; /* Are we quiet? */ int ntry; /* Number of tries per URL */ bool retry_connrefused; /* Treat CONNREFUSED as non-fatal. */ bool retry_on_host_error; /* Treat host errors as non-fatal. */ char *retry_on_http_error; /* Treat given HTTP errors as non-fatal. */ bool background; /* Whether we should work in background. */ bool ignore_length; /* Do we heed content-length at all? */ bool recursive; /* Are we recursive? */ bool spanhost; /* Do we span across hosts in recursion? */ int max_redirect; /* Maximum number of times we'll allow a page to redirect. */ bool relative_only; /* Follow only relative links. */ bool no_parent; /* Restrict access to the parent directory. */ int reclevel; /* Maximum level of recursion */ bool dirstruct; /* Do we build the directory structure as we go along? */ bool no_dirstruct; /* Do we hate dirstruct? */ int cut_dirs; /* Number of directory components to cut. */ bool add_hostdir; /* Do we add hostname directory? */ bool protocol_directories; /* Whether to prepend "http"/"ftp" to dirs. */ bool noclobber; /* Disables clobbering of existing data. */ bool unlink_requested; /* remove file before clobbering */ char *dir_prefix; /* The top of directory tree */ char *lfilename; /* Log filename */ char *input_filename; /* Input filename */ #ifdef HAVE_METALINK char *input_metalink; /* Input metalink file */ int metalink_index; /* Metalink application/metalink4+xml metaurl ordinal number. */ bool metalink_over_http; /* Use Metalink if present in HTTP response */ char *preferred_location; /* Preferred location for Metalink resources */ #endif char *choose_config; /* Specified config file */ bool noconfig; /* Ignore all config files? */ bool force_html; /* Is the input file an HTML file? */ char *default_page; /* Alternative default page (index file) */ bool spider; /* Is Wget in spider mode? */ char **accepts; /* List of patterns to accept. */ char **rejects; /* List of patterns to reject. */ const char **excludes; /* List of excluded FTP directories. */ const char **includes; /* List of FTP directories to follow. */ bool ignore_case; /* Whether to ignore case when matching dirs and files */ char *acceptregex_s; /* Patterns to accept (a regex string). */ char *rejectregex_s; /* Patterns to reject (a regex string). */ void *acceptregex; /* Patterns to accept (a regex struct). */ void *rejectregex; /* Patterns to reject (a regex struct). */ enum { #if defined HAVE_LIBPCRE || HAVE_LIBPCRE2 regex_type_pcre, #endif regex_type_posix } regex_type; /* The regex library. */ void *(*regex_compile_fun)(const char *); /* Function to compile a regex. */ bool (*regex_match_fun)(const void *, const char *); /* Function to match a string to a regex. */ #ifdef HAVE_LIBCARES char *bind_dns_address; char *dns_servers; #endif char **domains; /* See host.c */ char **exclude_domains; bool dns_cache; /* whether we cache DNS lookups. */ char **follow_tags; /* List of HTML tags to recursively follow. */ char **ignore_tags; /* List of HTML tags to ignore if recursing. */ bool follow_ftp; /* Are FTP URL-s followed in recursive retrieving? */ bool retr_symlinks; /* Whether we retrieve symlinks in FTP. */ char *output_document; /* The output file to which the documents will be printed. */ char *warc_filename; /* WARC output filename */ char *warc_tempdir; /* WARC temp dir */ char *warc_cdx_dedup_filename;/* CDX file to be used for deduplication. */ wgint warc_maxsize; /* WARC max archive size */ bool warc_compression_enabled;/* For GZIP compression. */ bool warc_digests_enabled; /* For SHA1 digests. */ bool warc_cdx_enabled; /* Create CDX files? */ bool warc_keep_log; /* Store the log file in a WARC record. */ char **warc_user_headers; /* User-defined WARC header(s). */ bool enable_xattr; /* Store metadata in POSIX extended attributes. */ char *user; /* Generic username */ char *passwd; /* Generic password */ bool ask_passwd; /* Ask for password? */ char *use_askpass; /* value to use for use-askpass if WGET_ASKPASS is not set */ bool always_rest; /* Always use REST. */ wgint start_pos; /* Start position of a download. */ char *ftp_user; /* FTP username */ char *ftp_passwd; /* FTP password */ bool netrc; /* Whether to read .netrc. */ bool ftp_glob; /* FTP globbing */ bool ftp_pasv; /* Passive FTP. */ char *http_user; /* HTTP username. */ char *http_passwd; /* HTTP password. */ char **user_headers; /* User-defined header(s). */ bool http_keep_alive; /* whether we use keep-alive */ bool use_proxy; /* Do we use proxy? */ bool allow_cache; /* Do we allow server-side caching? */ char *http_proxy, *ftp_proxy, *https_proxy; char **no_proxy; char *base_href; char *progress_type; /* progress indicator type. */ int show_progress; /* Show only the progress bar */ bool noscroll; /* Don't scroll the filename in the progressbar */ char *proxy_user; /*oli*/ char *proxy_passwd; double read_timeout; /* The read/write timeout. */ double dns_timeout; /* The DNS timeout. */ double connect_timeout; /* The connect timeout. */ bool random_wait; /* vary from 0 .. wait secs by random()? */ double wait; /* The wait period between retrievals. */ double waitretry; /* The wait period between retries. - HEH */ bool use_robots; /* Do we heed robots.txt? */ wgint limit_rate; /* Limit the download rate to this many bps. */ wgint quota; /* Maximum file size to download and store. */ bool server_response; /* Do we print server response? */ bool save_headers; /* Do we save headers together with file? */ bool content_on_error; /* Do we output the content when the HTTP status code indicates a server error */ bool debug; /* Debugging on/off */ #ifdef USE_WATT32 bool wdebug; /* Watt-32 tcp/ip debugging on/off */ #endif bool timestamping; /* Whether to use time-stamping. */ bool if_modified_since; /* Whether to use conditional get requests. */ bool backup_converted; /* Do we save pre-converted files as *.orig? */ int backups; /* Are numeric backups made? */ char *useragent; /* User-Agent string, which can be set to something other than Wget. */ char *referer; /* Naughty Referer, which can be set to something other than NULL. */ bool convert_links; /* Will the links be converted locally? */ bool convert_file_only; /* Convert only the file portion of the URI (i.e. basename). Leave everything else untouched. */ bool remove_listing; /* Do we remove .listing files generated by FTP? */ bool htmlify; /* Do we HTML-ify the OS-dependent listings? */ char *dot_style; wgint dot_bytes; /* How many bytes in a printing dot. */ int dots_in_line; /* How many dots in one line. */ int dot_spacing; /* How many dots between spacings. */ bool delete_after; /* Whether the files will be deleted after download. */ bool adjust_extension; /* Use ".html" extension on all text/html? */ bool page_requisites; /* Whether we need to download all files necessary to display a page properly. */ char *bind_address; /* What local IP address to bind to. */ #ifdef HAVE_SSL enum { secure_protocol_auto, secure_protocol_sslv2, secure_protocol_sslv3, secure_protocol_tlsv1, secure_protocol_tlsv1_1, secure_protocol_tlsv1_2, secure_protocol_tlsv1_3, secure_protocol_pfs } secure_protocol; /* type of secure protocol to use. */ int check_cert; /* whether to validate the server's cert */ char *cert_file; /* external client certificate to use. */ char *private_key; /* private key file (if not internal). */ enum keyfile_type { keyfile_pem, keyfile_asn1 } cert_type; /* type of client certificate file */ enum keyfile_type private_key_type; /* type of private key file */ char *ca_directory; /* CA directory (hash files) */ char *ca_cert; /* CA certificate file to use */ char *crl_file; /* file with CRLs */ char *pinnedpubkey; /* Public key (PEM/DER) file, or any number of base64 encoded sha256 hashes preceded by \'sha256//\' and separated by \';\', to verify peer against */ char *random_file; /* file with random data to seed the PRNG */ char *egd_file; /* file name of the egd daemon socket */ bool https_only; /* whether to follow HTTPS only */ bool ftps_resume_ssl; bool ftps_fallback_to_ftp; bool ftps_implicit; bool ftps_clear_data_connection; char *tls_ciphers_string; #endif /* HAVE_SSL */ bool cookies; /* whether cookies are used. */ char *cookies_input; /* file we're loading the cookies from. */ char *cookies_output; /* file we're saving the cookies to. */ bool keep_badhash; /* Keep files with checksum mismatch. */ bool keep_session_cookies; /* whether session cookies should be saved and loaded. */ char *post_data; /* POST query string */ char *post_file_name; /* File to post */ char *method; /* HTTP Method to use in Header */ char *body_data; /* HTTP Method Data String */ char *body_file; /* HTTP Method File */ enum { restrict_unix, restrict_vms, restrict_windows } restrict_files_os; /* file name restriction ruleset. */ bool restrict_files_ctrl; /* non-zero if control chars in URLs are restricted from appearing in generated file names. */ bool restrict_files_nonascii; /* non-zero if bytes with values greater than 127 are restricted. */ enum { restrict_no_case_restriction, restrict_lowercase, restrict_uppercase } restrict_files_case; /* file name case restriction. */ bool strict_comments; /* whether strict SGML comments are enforced. */ bool preserve_perm; /* whether remote permissions are used or that what is set by umask. */ #ifdef ENABLE_IPV6 bool ipv4_only; /* IPv4 connections have been requested. */ bool ipv6_only; /* IPv4 connections have been requested. */ #endif enum { prefer_ipv4, prefer_ipv6, prefer_none } prefer_family; /* preferred address family when more than one type is available */ bool content_disposition; /* Honor HTTP Content-Disposition header. */ bool auth_without_challenge; /* Issue Basic authentication creds without waiting for a challenge. */ bool enable_iri; char *encoding_remote; const char *locale; bool trustservernames; #ifdef __VMS int ftp_stmlf; /* Force Stream_LF format for binary FTP. */ #endif /* def __VMS */ bool useservertimestamps; /* Update downloaded files' timestamps to match those on server? */ bool show_all_dns_entries; /* Show all the DNS entries when resolving a name. */ bool report_bps; /*Output bandwidth in bits format*/ #ifdef HAVE_LIBZ enum compression_options { compression_auto, compression_gzip, compression_none } compression; /* type of HTTP compression to use */ #endif char *rejected_log; /* The file to log rejected URLS to. */ #ifdef HAVE_HSTS bool hsts; char *hsts_file; #endif const char *homedir; /* the homedir of the running process */ const char *wgetrcfile; /* the wgetrc file to be loaded */ }; extern struct options opt; wget-1.21.2/src/progress.h0000644000000000000000000000324414115732710012242 00000000000000/* Download progress. Copyright (C) 2001-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or \(at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #ifndef PROGRESS_H #define PROGRESS_H bool valid_progress_implementation_p (const char *); void set_progress_implementation (const char *); void progress_schedule_redirect (void); void *progress_create (const char *, wgint, wgint); bool progress_interactive_p (void *); void progress_update (void *, wgint, double); void progress_finish (void *, double); void progress_handle_sigwinch (int); #endif /* PROGRESS_H */ wget-1.21.2/src/init.h0000644000000000000000000000332614115732710011342 00000000000000/* Declarations for init.c. Copyright (C) 1996-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #ifndef INIT_H #define INIT_H char *ajoin_dir_file (const char *, const char *); char *wgetrc_env_file_name (void); char *wgetrc_user_file_name (void); char *wgetrc_file_name (void); int initialize (void); void run_command (const char *); void setoptval (const char *, const char *, const char *); char *home_dir (void); void cleanup (void); void defaults (void); bool run_wgetrc (const char *file, file_stats_t *); #define MAX_LONGOPTION 26 #endif /* INIT_H */ wget-1.21.2/src/css_.c0000644000000000000000000050126414115733377011337 00000000000000#include "wget.h" #line 1 "css.c" /* config.h must precede flex's inclusion of in order for its _GNU_SOURCE definition to take effect. */ #include #line 6 "css.c" #define YY_INT_ALIGNED short int /* A lexical scanner generated by flex */ #define FLEX_SCANNER #define YY_FLEX_MAJOR_VERSION 2 #define YY_FLEX_MINOR_VERSION 6 #define YY_FLEX_SUBMINOR_VERSION 4 #if YY_FLEX_SUBMINOR_VERSION > 0 #define FLEX_BETA #endif /* First, we deal with platform-specific or compiler-specific issues. */ /* begin standard C headers. */ #include #include #include #include /* end standard C headers. */ /* flex integer type definitions */ #ifndef FLEXINT_H #define FLEXINT_H /* C99 systems have . Non-C99 systems may or may not. */ #if defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L /* C99 says to define __STDC_LIMIT_MACROS before including stdint.h, * if you want the limit (max/min) macros for int types. */ #ifndef __STDC_LIMIT_MACROS #define __STDC_LIMIT_MACROS 1 #endif #include typedef int8_t flex_int8_t; typedef uint8_t flex_uint8_t; typedef int16_t flex_int16_t; typedef uint16_t flex_uint16_t; typedef int32_t flex_int32_t; typedef uint32_t flex_uint32_t; #else typedef signed char flex_int8_t; typedef short int flex_int16_t; typedef int flex_int32_t; typedef unsigned char flex_uint8_t; typedef unsigned short int flex_uint16_t; typedef unsigned int flex_uint32_t; /* Limits of integral types. */ #ifndef INT8_MIN #define INT8_MIN (-128) #endif #ifndef INT16_MIN #define INT16_MIN (-32767-1) #endif #ifndef INT32_MIN #define INT32_MIN (-2147483647-1) #endif #ifndef INT8_MAX #define INT8_MAX (127) #endif #ifndef INT16_MAX #define INT16_MAX (32767) #endif #ifndef INT32_MAX #define INT32_MAX (2147483647) #endif #ifndef UINT8_MAX #define UINT8_MAX (255U) #endif #ifndef UINT16_MAX #define UINT16_MAX (65535U) #endif #ifndef UINT32_MAX #define UINT32_MAX (4294967295U) #endif #ifndef SIZE_MAX #define SIZE_MAX (~(size_t)0) #endif #endif /* ! C99 */ #endif /* ! FLEXINT_H */ /* begin standard C++ headers. */ /* TODO: this is always defined, so inline it */ #define yyconst const #if defined(__GNUC__) && __GNUC__ >= 3 #define yynoreturn __attribute__((__noreturn__)) #else #define yynoreturn #endif /* Returned upon end-of-file. */ #define YY_NULL 0 /* Promotes a possibly negative, possibly signed char to an * integer in range [0..255] for use as an array index. */ #define YY_SC_TO_UI(c) ((YY_CHAR) (c)) /* Enter a start condition. This macro really ought to take a parameter, * but we do it the disgusting crufty way forced on us by the ()-less * definition of BEGIN. */ #define BEGIN (yy_start) = 1 + 2 * /* Translate the current start state into a value that can be later handed * to BEGIN to return to the state. The YYSTATE alias is for lex * compatibility. */ #define YY_START (((yy_start) - 1) / 2) #define YYSTATE YY_START /* Action number for EOF rule of a given start state. */ #define YY_STATE_EOF(state) (YY_END_OF_BUFFER + state + 1) /* Special action meaning "start processing a new file". */ #define YY_NEW_FILE yyrestart( yyin ) #define YY_END_OF_BUFFER_CHAR 0 /* Size of default input buffer. */ #ifndef YY_BUF_SIZE #ifdef __ia64__ /* On IA-64, the buffer size is 16k, not 8k. * Moreover, YY_BUF_SIZE is 2*YY_READ_BUF_SIZE in the general case. * Ditto for the __ia64__ case accordingly. */ #define YY_BUF_SIZE 32768 #else #define YY_BUF_SIZE 16384 #endif /* __ia64__ */ #endif /* The state buf must be large enough to hold one state per character in the main buffer. */ #define YY_STATE_BUF_SIZE ((YY_BUF_SIZE + 2) * sizeof(yy_state_type)) #ifndef YY_TYPEDEF_YY_BUFFER_STATE #define YY_TYPEDEF_YY_BUFFER_STATE typedef struct yy_buffer_state *YY_BUFFER_STATE; #endif #ifndef YY_TYPEDEF_YY_SIZE_T #define YY_TYPEDEF_YY_SIZE_T typedef size_t yy_size_t; #endif extern int yyleng; extern FILE *yyin, *yyout; #define EOB_ACT_CONTINUE_SCAN 0 #define EOB_ACT_END_OF_FILE 1 #define EOB_ACT_LAST_MATCH 2 #define YY_LESS_LINENO(n) #define YY_LINENO_REWIND_TO(ptr) /* Return all but the first "n" matched characters back to the input stream. */ #define yyless(n) \ do \ { \ /* Undo effects of setting up yytext. */ \ int yyless_macro_arg = (n); \ YY_LESS_LINENO(yyless_macro_arg);\ *yy_cp = (yy_hold_char); \ YY_RESTORE_YY_MORE_OFFSET \ (yy_c_buf_p) = yy_cp = yy_bp + yyless_macro_arg - YY_MORE_ADJ; \ YY_DO_BEFORE_ACTION; /* set up yytext again */ \ } \ while ( 0 ) #define unput(c) yyunput( c, (yytext_ptr) ) #ifndef YY_STRUCT_YY_BUFFER_STATE #define YY_STRUCT_YY_BUFFER_STATE struct yy_buffer_state { FILE *yy_input_file; char *yy_ch_buf; /* input buffer */ char *yy_buf_pos; /* current position in input buffer */ /* Size of input buffer in bytes, not including room for EOB * characters. */ int yy_buf_size; /* Number of characters read into yy_ch_buf, not including EOB * characters. */ int yy_n_chars; /* Whether we "own" the buffer - i.e., we know we created it, * and can realloc() it to grow it, and should free() it to * delete it. */ int yy_is_our_buffer; /* Whether this is an "interactive" input source; if so, and * if we're using stdio for input, then we want to use getc() * instead of fread(), to make sure we stop fetching input after * each newline. */ int yy_is_interactive; /* Whether we're considered to be at the beginning of a line. * If so, '^' rules will be active on the next match, otherwise * not. */ int yy_at_bol; int yy_bs_lineno; /**< The line count. */ int yy_bs_column; /**< The column count. */ /* Whether to try to fill the input buffer when we reach the * end of it. */ int yy_fill_buffer; int yy_buffer_status; #define YY_BUFFER_NEW 0 #define YY_BUFFER_NORMAL 1 /* When an EOF's been seen but there's still some text to process * then we mark the buffer as YY_EOF_PENDING, to indicate that we * shouldn't try reading from the input source any more. We might * still have a bunch of tokens to match, though, because of * possible backing-up. * * When we actually see the EOF, we change the status to "new" * (via yyrestart()), so that the user can continue scanning by * just pointing yyin at a new input file. */ #define YY_BUFFER_EOF_PENDING 2 }; #endif /* !YY_STRUCT_YY_BUFFER_STATE */ /* Stack of input buffers. */ static size_t yy_buffer_stack_top = 0; /**< index of top of stack. */ static size_t yy_buffer_stack_max = 0; /**< capacity of stack. */ static YY_BUFFER_STATE * yy_buffer_stack = NULL; /**< Stack as an array. */ /* We provide macros for accessing buffer states in case in the * future we want to put the buffer states in a more general * "scanner state". * * Returns the top of the stack, or NULL. */ #define YY_CURRENT_BUFFER ( (yy_buffer_stack) \ ? (yy_buffer_stack)[(yy_buffer_stack_top)] \ : NULL) /* Same as previous macro, but useful when we know that the buffer stack is not * NULL or when we need an lvalue. For internal use only. */ #define YY_CURRENT_BUFFER_LVALUE (yy_buffer_stack)[(yy_buffer_stack_top)] /* yy_hold_char holds the character lost when yytext is formed. */ static char yy_hold_char; static int yy_n_chars; /* number of characters read into yy_ch_buf */ int yyleng; /* Points to current character in buffer. */ static char *yy_c_buf_p = NULL; static int yy_init = 0; /* whether we need to initialize */ static int yy_start = 0; /* start state number */ /* Flag which is used to allow yywrap()'s to do buffer switches * instead of setting up a fresh yyin. A bit of a hack ... */ static int yy_did_buffer_switch_on_eof; void yyrestart ( FILE *input_file ); void yy_switch_to_buffer ( YY_BUFFER_STATE new_buffer ); YY_BUFFER_STATE yy_create_buffer ( FILE *file, int size ); void yy_delete_buffer ( YY_BUFFER_STATE b ); void yy_flush_buffer ( YY_BUFFER_STATE b ); void yypush_buffer_state ( YY_BUFFER_STATE new_buffer ); void yypop_buffer_state ( void ); static void yyensure_buffer_stack ( void ); static void yy_load_buffer_state ( void ); static void yy_init_buffer ( YY_BUFFER_STATE b, FILE *file ); #define YY_FLUSH_BUFFER yy_flush_buffer( YY_CURRENT_BUFFER ) YY_BUFFER_STATE yy_scan_buffer ( char *base, yy_size_t size ); YY_BUFFER_STATE yy_scan_string ( const char *yy_str ); YY_BUFFER_STATE yy_scan_bytes ( const char *bytes, int len ); void *yyalloc ( yy_size_t ); void *yyrealloc ( void *, yy_size_t ); void yyfree ( void * ); #define yy_new_buffer yy_create_buffer #define yy_set_interactive(is_interactive) \ { \ if ( ! YY_CURRENT_BUFFER ){ \ yyensure_buffer_stack (); \ YY_CURRENT_BUFFER_LVALUE = \ yy_create_buffer( yyin, YY_BUF_SIZE ); \ } \ YY_CURRENT_BUFFER_LVALUE->yy_is_interactive = is_interactive; \ } #define yy_set_bol(at_bol) \ { \ if ( ! YY_CURRENT_BUFFER ){\ yyensure_buffer_stack (); \ YY_CURRENT_BUFFER_LVALUE = \ yy_create_buffer( yyin, YY_BUF_SIZE ); \ } \ YY_CURRENT_BUFFER_LVALUE->yy_at_bol = at_bol; \ } #define YY_AT_BOL() (YY_CURRENT_BUFFER_LVALUE->yy_at_bol) /* Begin user sect3 */ #define yywrap() (/*CONSTCOND*/1) #define YY_SKIP_YYWRAP typedef flex_uint8_t YY_CHAR; FILE *yyin = NULL, *yyout = NULL; typedef int yy_state_type; extern int yylineno; int yylineno = 1; extern char *yytext; #ifdef yytext_ptr #undef yytext_ptr #endif #define yytext_ptr yytext static yy_state_type yy_get_previous_state ( void ); static yy_state_type yy_try_NUL_trans ( yy_state_type current_state ); static int yy_get_next_buffer ( void ); static void yynoreturn yy_fatal_error ( const char* msg ); /* Done after the current pattern has been matched and before the * corresponding action - sets up yytext. */ #define YY_DO_BEFORE_ACTION \ (yytext_ptr) = yy_bp; \ yyleng = (int) (yy_cp - yy_bp); \ (yy_hold_char) = *yy_cp; \ *yy_cp = '\0'; \ (yy_c_buf_p) = yy_cp; #define YY_NUM_RULES 41 #define YY_END_OF_BUFFER 42 /* This struct is not used in this scanner, but its presence is necessary. */ struct yy_trans_info { flex_int32_t yy_verify; flex_int32_t yy_nxt; }; static const flex_int16_t yy_accept[1103] = { 0, 0, 0, 42, 40, 1, 1, 40, 10, 40, 10, 40, 40, 40, 35, 40, 40, 11, 11, 40, 40, 40, 1, 0, 0, 0, 0, 10, 9, 10, 12, 0, 0, 10, 10, 0, 11, 0, 35, 4, 34, 0, 0, 35, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 30, 0, 0, 0, 0, 0, 0, 0, 39, 11, 0, 11, 11, 11, 8, 7, 0, 0, 0, 0, 0, 0, 0, 10, 10, 10, 0, 12, 12, 10, 10, 10, 6, 4, 4, 0, 33, 0, 21, 0, 33, 0, 18, 19, 0, 33, 0, 31, 0, 23, 0, 33, 0, 22, 29, 0, 25, 24, 20, 0, 33, 0, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 30, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 11, 11, 11, 11, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 10, 10, 0, 0, 12, 12, 10, 10, 10, 4, 2, 33, 33, 33, 33, 33, 21, 26, 0, 33, 33, 33, 33, 33, 33, 33, 33, 18, 19, 33, 0, 33, 33, 33, 33, 33, 33, 33, 31, 33, 33, 33, 23, 32, 0, 33, 33, 33, 33, 33, 33, 33, 33, 33, 22, 29, 33, 33, 33, 33, 33, 24, 20, 27, 0, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 30, 33, 33, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 11, 11, 38, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 3, 12, 10, 4, 4, 33, 33, 33, 33, 33, 21, 21, 33, 33, 33, 26, 33, 33, 33, 33, 33, 33, 33, 33, 33, 18, 19, 18, 28, 0, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 31, 31, 33, 33, 33, 23, 23, 33, 33, 33, 32, 33, 33, 33, 33, 33, 33, 33, 33, 33, 22, 29, 22, 33, 33, 33, 33, 33, 25, 24, 20, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 30, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 25, 33, 33, 33, 30, 30, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 14, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 11, 38, 38, 38, 38, 37, 0, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 12, 10, 33, 33, 33, 33, 21, 21, 21, 21, 33, 33, 33, 26, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 18, 19, 18, 18, 18, 19, 19, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 31, 31, 31, 31, 33, 33, 33, 23, 23, 23, 23, 33, 33, 33, 32, 32, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 22, 29, 22, 22, 22, 29, 29, 33, 33, 33, 33, 33, 25, 24, 20, 25, 25, 24, 24, 20, 20, 33, 33, 33, 27, 33, 33, 33, 33, 33, 33, 27, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 30, 33, 33, 33, 25, 33, 27, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 15, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 11, 38, 38, 38, 38, 38, 38, 38, 38, 0, 38, 37, 38, 38, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 12, 10, 33, 33, 33, 33, 21, 21, 33, 33, 33, 26, 26, 26, 33, 33, 33, 33, 33, 33, 33, 33, 33, 18, 19, 18, 33, 33, 33, 28, 33, 33, 33, 33, 33, 33, 28, 33, 33, 33, 33, 33, 28, 33, 33, 33, 31, 31, 33, 33, 33, 23, 23, 33, 33, 33, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33, 33, 33, 22, 29, 22, 33, 33, 33, 33, 33, 25, 24, 20, 33, 33, 33, 27, 27, 27, 33, 33, 33, 33, 27, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 30, 33, 33, 33, 25, 33, 27, 0, 13, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 14, 0, 0, 0, 0, 11, 38, 36, 38, 38, 38, 38, 38, 38, 38, 38, 38, 0, 38, 38, 37, 38, 0, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 12, 10, 33, 33, 33, 21, 21, 33, 33, 33, 26, 33, 33, 33, 33, 33, 33, 33, 18, 19, 18, 33, 33, 33, 28, 28, 28, 33, 33, 33, 33, 33, 33, 33, 33, 28, 33, 33, 31, 31, 33, 33, 23, 23, 33, 33, 33, 32, 32, 33, 33, 33, 33, 33, 33, 33, 22, 29, 22, 33, 33, 33, 33, 25, 24, 20, 33, 33, 33, 27, 33, 33, 33, 27, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 30, 33, 33, 33, 25, 33, 27, 0, 0, 0, 0, 13, 0, 0, 0, 0, 0, 0, 0, 15, 0, 0, 0, 0, 0, 14, 14, 0, 11, 38, 38, 38, 38, 38, 38, 38, 0, 0, 38, 38, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 33, 21, 21, 33, 33, 26, 33, 18, 19, 18, 33, 33, 33, 28, 33, 33, 33, 33, 33, 28, 31, 31, 23, 23, 33, 33, 32, 32, 33, 22, 29, 22, 25, 24, 20, 33, 33, 27, 33, 27, 16, 0, 13, 0, 0, 0, 0, 0, 15, 15, 0, 0, 38, 38, 38, 0, 0, 0, 0, 38, 17, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 26, 33, 33, 28, 33, 32, 32, 27, 0, 13, 13, 0, 0, 38, 38, 38, 0, 0, 38, 0, 0, 0, 17, 0, 0, 0, 0, 0, 0, 0, 28, 0, 38, 38, 38, 0, 38, 0, 17, 0, 0, 0, 0, 38, 38, 0, 38, 0, 17, 17, 0, 0, 0, 0, 0 } ; static const YY_CHAR yy_ec[256] = { 0, 1, 1, 1, 1, 1, 1, 1, 1, 2, 3, 1, 4, 5, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 6, 7, 8, 9, 10, 11, 10, 12, 13, 14, 15, 10, 10, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 10, 10, 29, 30, 31, 10, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 42, 49, 50, 51, 52, 42, 42, 53, 42, 54, 10, 55, 10, 10, 42, 10, 56, 57, 58, 59, 60, 61, 62, 63, 64, 42, 65, 66, 67, 68, 69, 70, 42, 71, 72, 73, 74, 42, 42, 75, 42, 76, 10, 77, 10, 78, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79 } ; static const YY_CHAR yy_meta[80] = { 0, 1, 2, 3, 3, 3, 2, 2, 4, 2, 2, 2, 4, 5, 2, 2, 6, 2, 7, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 2, 2, 2, 2, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 2, 2, 9 } ; static const flex_int16_t yy_base[1137] = { 0, 0, 0, 4195, 7110, 78, 83, 88, 87, 78, 85, 82, 88, 4161, 142, 4151, 90, 86, 206, 259, 4118, 4083, 98, 234, 4083, 72, 109, 116, 7110, 318, 100, 4082, 361, 208, 420, 4017, 92, 463, 205, 4024, 7110, 3977, 222, 0, 3974, 209, 89, 257, 202, 180, 245, 248, 259, 355, 272, 3955, 524, 3987, 83, 280, 311, 274, 585, 7110, 117, 637, 348, 210, 697, 7110, 7110, 3963, 302, 382, 243, 3940, 3933, 371, 251, 356, 757, 3945, 101, 817, 246, 357, 877, 7110, 3938, 252, 920, 3894, 974, 3886, 1017, 397, 447, 3861, 3860, 661, 376, 904, 3850, 721, 3841, 448, 451, 1040, 3840, 3825, 1045, 3813, 3796, 3787, 1063, 569, 600, 3784, 1137, 484, 1180, 613, 1221, 748, 380, 566, 347, 591, 907, 654, 462, 3767, 3789, 118, 559, 779, 633, 593, 588, 628, 782, 654, 3776, 775, 3775, 802, 772, 682, 336, 1279, 345, 447, 1322, 3754, 254, 695, 837, 658, 581, 632, 656, 806, 1012, 456, 862, 1365, 3744, 256, 903, 1408, 278, 964, 1451, 3730, 7110, 3689, 1511, 839, 751, 790, 3655, 3633, 1086, 865, 3648, 3645, 868, 812, 3627, 850, 3599, 3559, 3554, 895, 897, 344, 3587, 3580, 885, 900, 892, 922, 3526, 1001, 929, 943, 3484, 3483, 1165, 1002, 3500, 3499, 1037, 1074, 945, 3485, 969, 3476, 3439, 3424, 1096, 3450, 410, 3442, 438, 3403, 3397, 3396, 1109, 1104, 3416, 3396, 745, 1554, 1126, 1597, 1199, 1638, 1237, 1696, 1591, 1451, 1224, 1636, 1495, 1673, 1680, 1762, 1836, 1318, 1740, 1703, 7110, 573, 1049, 1161, 1158, 970, 987, 1131, 1017, 1192, 1245, 3385, 3371, 1046, 1269, 1252, 3369, 3358, 1342, 1600, 1403, 1552, 1788, 1263, 1891, 1796, 1934, 3362, 1868, 1238, 1307, 1248, 3350, 3304, 1055, 1352, 1779, 1931, 1391, 1392, 1393, 1983, 3284, 7110, 2026, 2069, 3253, 347, 892, 2112, 1392, 1108, 1110, 1248, 1488, 1500, 3239, 3204, 3174, 1541, 3194, 3177, 2112, 1624, 1119, 3145, 1176, 3142, 1861, 1873, 1878, 3104, 1704, 1607, 3071, 2999, 1394, 2951, 2939, 1979, 1503, 1180, 1209, 1896, 1901, 1669, 1259, 1273, 1936, 1971, 1707, 1278, 1296, 2904, 1691, 2927, 2900, 2022, 1731, 1336, 2867, 1403, 2856, 1994, 2027, 2063, 1753, 2851, 716, 2848, 950, 2068, 2106, 2117, 1786, 2836, 2831, 1793, 2833, 2795, 2150, 2191, 1807, 2234, 2138, 2275, 2156, 2333, 2228, 2237, 2273, 2344, 2358, 2367, 2378, 2449, 2523, 2189, 2433, 2413, 1498, 2121, 2020, 2269, 2451, 1769, 2278, 1797, 2271, 1426, 1684, 1625, 2322, 1465, 2311, 2342, 2464, 2148, 2457, 2261, 1528, 2388, 2423, 2759, 1179, 1347, 2149, 2296, 2245, 2683, 2676, 1875, 1831, 2535, 2548, 1910, 2480, 2177, 2645, 2637, 2383, 2555, 7110, 2446, 2468, 2600, 2595, 2508, 2546, 2570, 2561, 2456, 2552, 2533, 2540, 1981, 2561, 2614, 2651, 2674, 741, 457, 7110, 2729, 2807, 2597, 661, 2134, 2596, 2579, 1548, 1591, 2330, 2405, 2814, 2604, 1589, 2641, 97, 2882, 2584, 771, 2942, 3003, 3046, 2605, 1637, 1675, 2688, 2695, 2537, 1235, 2639, 2556, 2555, 2710, 2662, 2543, 2538, 2819, 1925, 2402, 2684, 1849, 2508, 1979, 2507, 2715, 2720, 2844, 2463, 1448, 2462, 1598, 2832, 2475, 2463, 2837, 2420, 2412, 2894, 2582, 2374, 2365, 3042, 2023, 2628, 2816, 2853, 2025, 2109, 2889, 2919, 2312, 1614, 2857, 2229, 2270, 2929, 3047, 2262, 1615, 2931, 2423, 2439, 3052, 3083, 3071, 2289, 2269, 3106, 2300, 2407, 3094, 2461, 2233, 2530, 2232, 3119, 3124, 3129, 2191, 1714, 2183, 1734, 3117, 2186, 1033, 2175, 1110, 3142, 3147, 3152, 2135, 1803, 2132, 2181, 2103, 2338, 3076, 2090, 2077, 3160, 3148, 2035, 2003, 3172, 3132, 3134, 0, 3213, 1864, 3230, 3173, 3254, 3181, 3312, 3378, 3437, 3511, 3581, 3656, 3723, 3785, 3859, 3933, 3219, 3988, 4050, 3236, 2813, 3269, 2888, 2606, 2901, 3277, 3160, 2608, 2666, 3251, 3162, 3338, 3190, 2881, 3264, 7110, 3191, 3285, 1994, 1985, 2987, 3302, 3350, 3326, 3303, 3358, 3340, 1965, 1964, 3341, 3369, 3367, 3038, 3373, 3009, 4107, 3394, 1039, 3416, 4166, 696, 4225, 3442, 3463, 3481, 3497, 3517, 4285, 4346, 4407, 3059, 3408, 3193, 1957, 1942, 3295, 3430, 3539, 3544, 3283, 2258, 3485, 311, 4467, 4510, 4553, 4596, 590, 2669, 3080, 3327, 3553, 3451, 1920, 1919, 3559, 1876, 2372, 805, 1897, 1896, 3572, 3260, 3261, 1801, 3311, 1794, 3565, 3614, 3619, 3505, 1764, 1751, 3626, 3572, 1712, 1704, 3635, 3537, 3597, 0, 842, 1683, 1668, 3640, 3543, 0, 962, 3354, 3355, 3645, 3661, 1143, 3420, 3436, 3672, 3681, 3673, 3477, 3547, 3701, 3710, 1613, 2478, 1302, 1608, 1607, 3716, 3707, 3553, 1605, 3554, 1537, 3732, 3737, 3748, 3732, 1529, 1796, 1502, 1830, 3762, 3770, 3777, 3774, 1498, 1459, 3798, 1394, 3135, 1339, 1384, 1374, 3811, 0, 4639, 3784, 1740, 3822, 2122, 3822, 3869, 3895, 3916, 3941, 3970, 3978, 3995, 4699, 4002, 3880, 4083, 4105, 3722, 0, 3634, 0, 1410, 7110, 3904, 3810, 1349, 1342, 3782, 3868, 4066, 4635, 3883, 3654, 3856, 1427, 3908, 1333, 1308, 3944, 4176, 3996, 3768, 3982, 1575, 4001, 4071, 4011, 3853, 4024, 1763, 4756, 4088, 7110, 1487, 3423, 4816, 896, 3208, 4876, 4262, 4444, 4919, 4504, 4547, 4591, 4677, 4979, 5040, 5101, 3896, 4040, 4023, 1274, 1266, 3798, 4080, 4509, 4041, 4065, 4639, 1821, 4709, 5144, 5187, 5230, 3632, 3726, 4208, 4213, 2076, 1260, 1207, 4231, 1176, 1123, 4793, 3760, 1108, 3762, 1099, 4236, 4322, 4327, 4093, 1043, 1038, 4332, 994, 3434, 2212, 1028, 1002, 4913, 992, 951, 4552, 3833, 0, 3946, 3953, 4474, 4479, 4041, 4059, 4645, 4716, 2284, 4091, 4106, 4721, 4733, 929, 840, 4919, 4115, 838, 4116, 837, 4738, 4743, 4756, 811, 2240, 777, 3157, 4763, 4798, 4803, 2371, 745, 718, 4823, 658, 646, 5138, 0, 4956, 5225, 5181, 5269, 5279, 5291, 5296, 5303, 5315, 5386, 5143, 5350, 5369, 5375, 5394, 5408, 5448, 5413, 642, 4130, 618, 583, 7110, 4253, 5467, 4134, 4243, 5276, 2587, 4250, 4854, 4277, 4128, 4675, 2903, 4432, 7110, 526, 2948, 4859, 1544, 4963, 5522, 1357, 3705, 5565, 5608, 5472, 5669, 5477, 5730, 4466, 5171, 4581, 508, 493, 4435, 5496, 4865, 4634, 5275, 3053, 5506, 5511, 5565, 478, 449, 4864, 5767, 5602, 5607, 5772, 3203, 451, 445, 4924, 443, 442, 5777, 5790, 5795, 5805, 5812, 5835, 5853, 5859, 4140, 4164, 4984, 5017, 5866, 5871, 5876, 5881, 5889, 5894, 5899, 411, 381, 5022, 5907, 5913, 7110, 4866, 5027, 5214, 4678, 5521, 3400, 4944, 7110, 370, 3468, 3483, 5950, 5993, 6036, 5987, 6030, 6096, 0, 6139, 7110, 5502, 5131, 4161, 4218, 4832, 5304, 6035, 5313, 4441, 5307, 3527, 5992, 344, 285, 5490, 6133, 6073, 6138, 6176, 5382, 7110, 296, 3629, 3675, 6213, 6256, 6299, 6199, 6342, 6385, 5403, 237, 230, 7110, 5415, 6212, 6255, 5439, 3976, 5380, 3852, 6293, 3928, 6428, 6471, 6514, 6557, 6600, 5441, 5531, 5500, 5100, 5886, 4057, 6643, 6686, 6729, 5612, 5628, 7110, 133, 4058, 6772, 4181, 6336, 7110, 6833, 6837, 6846, 6850, 6855, 6864, 6873, 6882, 6891, 6900, 112, 6904, 6913, 6922, 6931, 6940, 6949, 6958, 6967, 6976, 6984, 6993, 7002, 7011, 7020, 7029, 7038, 7047, 7056, 7065, 7074, 7083, 7092, 7100 } ; static const flex_int16_t yy_def[1137] = { 0, 1102, 1, 1102, 1102, 1102, 1102, 1102, 1103, 1104, 1105, 1106, 1102, 1102, 1102, 1102, 1102, 1107, 1107, 1108, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1103, 1102, 1109, 1104, 1102, 1110, 1105, 1111, 1102, 1107, 1108, 14, 1112, 1102, 1113, 1102, 14, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1107, 1115, 1107, 1107, 1107, 1102, 1102, 1116, 1102, 1102, 1102, 1102, 1102, 1102, 1103, 1103, 1103, 1117, 1104, 1104, 1105, 1105, 1105, 1102, 1112, 1118, 56, 1114, 1119, 1114, 1119, 1114, 94, 1114, 1114, 94, 1114, 94, 1114, 94, 1114, 94, 1114, 94, 1114, 1114, 94, 1114, 1114, 1114, 94, 1114, 94, 1114, 1114, 118, 118, 118, 118, 118, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1107, 68, 1107, 1107, 68, 1116, 1120, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1103, 1103, 80, 1117, 1121, 1104, 83, 1105, 1105, 86, 1122, 1102, 1114, 118, 176, 176, 176, 1114, 1114, 94, 176, 176, 176, 176, 176, 176, 176, 176, 1114, 1114, 1114, 94, 176, 176, 176, 1114, 176, 176, 176, 1114, 176, 176, 176, 1114, 1114, 94, 176, 176, 176, 1114, 176, 176, 176, 176, 176, 1114, 1114, 176, 176, 176, 176, 176, 1114, 1114, 1114, 94, 176, 176, 176, 1114, 118, 233, 233, 233, 233, 233, 233, 239, 239, 239, 239, 239, 239, 239, 239, 233, 248, 248, 239, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1107, 68, 1123, 68, 1124, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 80, 1125, 1102, 83, 86, 1122, 1126, 1114, 176, 301, 301, 301, 301, 301, 176, 176, 176, 1114, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 1114, 94, 176, 176, 176, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 176, 176, 176, 1114, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 176, 176, 176, 301, 301, 301, 301, 233, 373, 373, 373, 373, 373, 373, 379, 379, 379, 379, 379, 379, 379, 379, 373, 388, 388, 379, 1114, 1114, 1114, 1114, 373, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 373, 1114, 1114, 373, 1114, 1114, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 276, 1123, 1123, 1127, 1128, 1102, 1102, 276, 1129, 1130, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1131, 1132, 1133, 1102, 86, 301, 476, 476, 476, 476, 476, 1114, 1114, 301, 301, 301, 301, 476, 476, 476, 476, 1114, 1114, 476, 476, 476, 476, 476, 476, 476, 476, 1114, 1114, 1114, 1114, 176, 176, 176, 301, 301, 301, 301, 476, 476, 476, 476, 1114, 1114, 476, 476, 476, 476, 476, 476, 1114, 1114, 476, 476, 476, 476, 476, 1114, 1114, 301, 301, 301, 301, 301, 476, 476, 476, 476, 1114, 1114, 476, 476, 476, 476, 476, 476, 476, 476, 1114, 1114, 1114, 1114, 476, 476, 476, 476, 476, 476, 476, 476, 1114, 1114, 1114, 1114, 1114, 1114, 301, 301, 301, 301, 476, 476, 476, 476, 1114, 1114, 476, 373, 582, 582, 582, 582, 582, 582, 582, 582, 582, 582, 582, 582, 590, 590, 582, 582, 582, 590, 582, 582, 582, 582, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 276, 1102, 1127, 1102, 1134, 1128, 1135, 1123, 1123, 1102, 1123, 1123, 1123, 1102, 456, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1131, 474, 475, 476, 668, 668, 668, 668, 668, 476, 476, 476, 476, 1114, 1114, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 301, 301, 301, 301, 476, 476, 476, 476, 1114, 1114, 476, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 476, 476, 476, 476, 476, 1114, 1114, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 476, 476, 476, 476, 1114, 1114, 668, 668, 668, 668, 668, 582, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 276, 1102, 1102, 1127, 1127, 1127, 1128, 1128, 1128, 1123, 1123, 649, 1136, 1136, 1123, 1136, 649, 1102, 651, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1131, 474, 475, 668, 842, 842, 842, 842, 668, 668, 668, 668, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 476, 476, 476, 476, 1114, 1114, 668, 668, 668, 668, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 668, 668, 668, 668, 668, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 668, 668, 668, 668, 842, 842, 842, 842, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 826, 1127, 1127, 813, 1128, 1128, 816, 649, 1136, 1102, 1136, 824, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1114, 1114, 1114, 842, 842, 842, 1114, 1114, 1114, 1114, 668, 668, 668, 668, 842, 842, 842, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 842, 842, 842, 842, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 842, 842, 842, 1114, 1114, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 813, 816, 649, 1136, 1136, 1136, 962, 824, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1114, 842, 842, 842, 1114, 1114, 1114, 1114, 1102, 1102, 1102, 1102, 1102, 813, 816, 649, 1136, 1033, 824, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1114, 1102, 813, 816, 649, 1033, 824, 1102, 1102, 1102, 1102, 1102, 1102, 813, 816, 1033, 1082, 1102, 1102, 1102, 1102, 1033, 1102, 1136, 0, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102 } ; static const flex_int16_t yy_nxt[7190] = { 0, 4, 5, 6, 5, 5, 5, 7, 8, 9, 4, 4, 10, 4, 4, 4, 11, 12, 13, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 15, 4, 4, 16, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 18, 17, 17, 19, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 18, 17, 17, 20, 21, 17, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 23, 23, 23, 23, 23, 28, 31, 28, 35, 63, 22, 22, 22, 22, 22, 63, 24, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 72, 1102, 1102, 75, 44, 76, 133, 28, 58, 95, 73, 74, 25, 63, 59, 75, 32, 76, 60, 1096, 37, 61, 72, 34, 65, 29, 26, 96, 62, 133, 65, 58, 95, 77, 253, 25, 40, 59, 32, 32, 60, 41, 42, 61, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 29, 65, 77, 253, 44, 44, 45, 46, 47, 44, 48, 49, 50, 44, 51, 44, 52, 44, 44, 53, 54, 55, 44, 44, 44, 44, 56, 44, 44, 45, 46, 47, 44, 48, 49, 50, 51, 44, 52, 44, 44, 53, 54, 55, 44, 44, 44, 44, 63, 28, 44, 1102, 63, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 102, 103, 23, 23, 23, 23, 23, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 100, 24, 1086, 93, 66, 102, 101, 28, 28, 1086, 65, 161, 34, 94, 65, 75, 89, 76, 154, 174, 167, 280, 100, 295, 25, 93, 66, 68, 68, 68, 68, 68, 68, 68, 68, 68, 68, 106, 26, 28, 104, 68, 68, 68, 68, 68, 68, 25, 1057, 105, 34, 97, 107, 108, 115, 29, 138, 1078, 109, 98, 106, 99, 104, 110, 68, 68, 68, 68, 68, 68, 27, 27, 79, 97, 134, 108, 116, 115, 139, 138, 109, 98, 34, 158, 135, 159, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 134, 136, 63, 155, 80, 80, 80, 80, 80, 80, 156, 277, 27, 33, 63, 299, 328, 28, 174, 137, 1078, 329, 28, 330, 136, 155, 1024, 80, 80, 80, 80, 80, 80, 83, 83, 83, 83, 83, 83, 83, 83, 83, 83, 111, 65, 150, 104, 83, 83, 83, 83, 83, 83, 65, 157, 105, 65, 1055, 158, 112, 159, 113, 193, 114, 29, 34, 111, 150, 104, 72, 83, 83, 83, 83, 83, 83, 33, 33, 85, 73, 160, 112, 100, 113, 194, 193, 364, 1055, 101, 181, 365, 72, 86, 86, 86, 86, 86, 86, 86, 86, 86, 86, 160, 67, 100, 182, 86, 86, 86, 86, 86, 86, 181, 63, 364, 1052, 1052, 28, 365, 183, 203, 1051, 640, 184, 204, 185, 205, 1051, 1048, 86, 86, 86, 86, 86, 86, 68, 68, 68, 68, 68, 68, 68, 68, 68, 68, 175, 175, 206, 115, 68, 68, 68, 68, 68, 68, 65, 234, 1048, 207, 208, 234, 234, 234, 234, 29, 643, 1043, 175, 175, 206, 116, 115, 68, 68, 68, 68, 68, 68, 117, 117, 207, 1043, 950, 117, 117, 117, 117, 117, 117, 117, 117, 117, 117, 117, 117, 117, 118, 119, 119, 119, 120, 121, 122, 123, 119, 119, 117, 117, 117, 117, 119, 119, 119, 119, 119, 119, 124, 125, 126, 117, 127, 117, 128, 117, 117, 129, 130, 131, 117, 117, 117, 117, 117, 119, 119, 119, 119, 119, 119, 124, 125, 126, 127, 117, 128, 117, 117, 129, 130, 131, 117, 117, 117, 117, 117, 117, 117, 140, 227, 1018, 254, 141, 142, 143, 144, 262, 843, 255, 844, 263, 288, 264, 229, 102, 103, 417, 230, 228, 231, 145, 265, 227, 254, 146, 106, 247, 147, 248, 249, 234, 234, 234, 234, 288, 1018, 102, 266, 417, 175, 107, 267, 1016, 145, 265, 268, 146, 269, 106, 147, 149, 149, 149, 149, 149, 149, 149, 149, 149, 149, 1014, 175, 289, 260, 149, 149, 149, 149, 149, 149, 458, 287, 1014, 280, 186, 158, 271, 159, 187, 188, 189, 190, 261, 111, 272, 289, 260, 149, 149, 149, 149, 149, 149, 67, 67, 67, 151, 67, 155, 112, 191, 113, 640, 114, 63, 156, 111, 272, 192, 138, 152, 152, 152, 152, 152, 152, 152, 152, 152, 152, 155, 112, 191, 113, 152, 152, 152, 152, 152, 152, 192, 139, 138, 563, 199, 1013, 281, 564, 176, 200, 176, 201, 117, 640, 282, 643, 65, 152, 152, 152, 152, 152, 152, 163, 78, 78, 164, 163, 281, 28, 175, 247, 1013, 248, 249, 234, 234, 234, 234, 202, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 473, 305, 175, 295, 165, 165, 165, 165, 165, 165, 641, 202, 256, 1008, 92, 270, 257, 271, 258, 141, 142, 143, 144, 136, 305, 274, 29, 165, 165, 165, 165, 165, 165, 82, 82, 82, 168, 82, 259, 290, 306, 137, 851, 75, 852, 76, 136, 1008, 274, 1102, 169, 169, 169, 169, 169, 169, 169, 169, 169, 169, 259, 134, 320, 306, 169, 169, 169, 169, 169, 169, 283, 135, 302, 1006, 1006, 284, 303, 285, 304, 78, 871, 1004, 872, 134, 28, 320, 32, 169, 169, 169, 169, 169, 169, 170, 84, 84, 171, 170, 311, 286, 322, 315, 312, 28, 313, 316, 317, 318, 319, 175, 172, 172, 172, 172, 172, 172, 172, 172, 172, 172, 82, 286, 640, 322, 172, 172, 172, 172, 172, 172, 325, 29, 193, 332, 326, 1102, 327, 195, 333, 335, 334, 176, 196, 176, 197, 323, 34, 172, 172, 172, 172, 172, 172, 119, 194, 193, 175, 119, 119, 119, 119, 92, 335, 175, 324, 643, 108, 198, 323, 336, 1004, 109, 32, 117, 117, 117, 110, 117, 175, 117, 340, 84, 117, 117, 117, 175, 993, 563, 108, 198, 28, 564, 336, 109, 341, 355, 117, 117, 117, 117, 876, 117, 877, 340, 117, 117, 117, 176, 176, 176, 176, 176, 176, 176, 176, 176, 176, 341, 355, 357, 425, 176, 176, 176, 176, 176, 176, 993, 291, 291, 291, 292, 291, 34, 337, 346, 992, 426, 338, 347, 339, 348, 357, 425, 176, 176, 176, 176, 176, 176, 177, 176, 176, 176, 178, 176, 179, 176, 176, 176, 426, 640, 992, 92, 176, 176, 176, 176, 176, 176, 744, 72, 427, 209, 745, 989, 180, 210, 213, 211, 989, 73, 214, 215, 216, 217, 428, 176, 176, 176, 176, 176, 176, 72, 212, 427, 220, 434, 180, 175, 221, 222, 223, 224, 218, 207, 208, 350, 641, 219, 418, 351, 352, 353, 354, 435, 281, 212, 419, 307, 434, 175, 175, 308, 282, 309, 218, 207, 225, 358, 226, 219, 418, 359, 360, 361, 362, 369, 281, 310, 984, 370, 366, 371, 175, 175, 367, 744, 368, 984, 225, 745, 226, 117, 117, 117, 232, 117, 480, 374, 481, 982, 310, 374, 374, 374, 374, 175, 175, 499, 233, 234, 234, 234, 235, 236, 237, 238, 234, 234, 880, 480, 881, 481, 234, 234, 234, 234, 234, 234, 175, 424, 499, 254, 420, 257, 415, 258, 342, 421, 255, 422, 176, 343, 176, 344, 92, 234, 234, 234, 234, 234, 234, 234, 982, 254, 239, 240, 241, 234, 242, 243, 244, 423, 175, 429, 501, 523, 245, 430, 246, 431, 387, 345, 388, 389, 374, 374, 374, 374, 399, 399, 399, 400, 399, 423, 175, 981, 92, 501, 523, 245, 482, 246, 234, 345, 524, 239, 240, 241, 234, 242, 243, 244, 482, 482, 482, 483, 482, 250, 387, 251, 388, 389, 374, 374, 374, 374, 432, 524, 148, 465, 263, 91, 264, 440, 284, 100, 285, 268, 63, 269, 250, 101, 251, 148, 148, 148, 275, 148, 981, 459, 436, 971, 92, 91, 437, 460, 438, 100, 530, 971, 276, 276, 276, 276, 276, 276, 276, 276, 276, 276, 439, 459, 531, 537, 276, 276, 276, 276, 276, 276, 65, 530, 415, 415, 415, 416, 415, 889, 461, 890, 944, 538, 462, 439, 463, 531, 537, 276, 276, 276, 276, 276, 276, 278, 278, 278, 278, 278, 278, 278, 278, 278, 278, 374, 538, 944, 464, 278, 278, 278, 278, 278, 278, 442, 910, 938, 911, 141, 142, 143, 144, 640, 938, 467, 550, 92, 374, 158, 464, 159, 278, 278, 278, 278, 278, 278, 293, 293, 293, 293, 293, 293, 293, 293, 293, 293, 912, 550, 291, 605, 293, 293, 293, 293, 293, 293, 912, 445, 445, 445, 446, 445, 470, 477, 643, 513, 75, 478, 76, 479, 514, 605, 515, 293, 293, 293, 293, 293, 293, 296, 296, 296, 296, 296, 296, 296, 296, 296, 296, 72, 72, 552, 136, 296, 296, 296, 296, 296, 296, 73, 73, 92, 257, 502, 258, 397, 397, 397, 398, 397, 137, 72, 72, 932, 552, 136, 296, 296, 296, 296, 296, 296, 297, 297, 297, 297, 297, 297, 297, 297, 297, 297, 102, 103, 909, 932, 297, 297, 297, 297, 297, 297, 482, 482, 482, 483, 482, 640, 97, 403, 403, 403, 404, 403, 102, 92, 98, 106, 99, 297, 297, 297, 297, 297, 297, 175, 175, 175, 300, 175, 97, 484, 107, 909, 520, 485, 903, 486, 98, 521, 106, 522, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 91, 104, 641, 93, 301, 301, 301, 301, 301, 301, 105, 903, 640, 94, 447, 447, 447, 448, 447, 897, 488, 115, 91, 104, 489, 93, 490, 301, 301, 301, 301, 301, 301, 373, 374, 374, 374, 375, 376, 377, 378, 374, 374, 116, 115, 138, 659, 374, 374, 374, 374, 374, 374, 394, 394, 394, 395, 394, 263, 641, 264, 504, 443, 443, 443, 444, 443, 139, 138, 659, 374, 374, 374, 374, 374, 374, 374, 525, 532, 379, 380, 381, 374, 382, 383, 384, 509, 897, 396, 660, 510, 385, 511, 386, 891, 891, 91, 155, 401, 401, 401, 402, 401, 494, 156, 134, 96, 495, 496, 497, 498, 396, 660, 92, 385, 135, 386, 374, 91, 155, 379, 380, 381, 374, 382, 383, 384, 134, 92, 92, 92, 104, 390, 672, 391, 405, 405, 405, 406, 405, 105, 91, 407, 407, 407, 408, 407, 401, 527, 873, 102, 103, 528, 104, 529, 390, 672, 391, 392, 392, 392, 393, 392, 91, 873, 407, 407, 407, 408, 407, 539, 673, 102, 106, 540, 374, 541, 553, 91, 374, 374, 374, 374, 506, 870, 108, 534, 507, 107, 508, 109, 535, 870, 536, 673, 110, 106, 555, 102, 103, 91, 93, 405, 405, 405, 406, 405, 108, 108, 175, 545, 94, 109, 109, 546, 547, 548, 549, 110, 923, 102, 924, 925, 93, 409, 409, 409, 410, 409, 92, 108, 175, 557, 374, 864, 109, 558, 559, 560, 561, 106, 468, 468, 468, 469, 468, 268, 864, 269, 92, 445, 445, 445, 446, 445, 107, 374, 411, 450, 450, 450, 450, 450, 106, 452, 571, 565, 91, 453, 572, 454, 573, 575, 112, 97, 113, 576, 114, 577, 904, 411, 859, 98, 905, 99, 136, 583, 155, 859, 91, 583, 583, 583, 583, 156, 112, 97, 113, 412, 412, 412, 413, 412, 137, 98, 284, 100, 285, 136, 155, 614, 455, 101, 904, 257, 374, 258, 905, 92, 374, 374, 374, 374, 502, 502, 502, 503, 502, 100, 414, 23, 23, 23, 23, 23, 504, 504, 504, 505, 504, 502, 502, 502, 503, 502, 689, 24, 757, 757, 757, 757, 116, 414, 148, 148, 148, 275, 148, 525, 525, 525, 526, 525, 525, 525, 525, 526, 525, 689, 25, 449, 449, 449, 449, 449, 449, 449, 449, 449, 449, 853, 853, 418, 26, 449, 449, 449, 449, 449, 449, 419, 92, 25, 468, 468, 468, 469, 468, 532, 532, 532, 533, 532, 617, 418, 850, 850, 449, 449, 449, 449, 449, 449, 456, 456, 456, 456, 456, 456, 456, 456, 456, 456, 834, 181, 618, 617, 456, 456, 456, 456, 456, 456, 532, 532, 532, 533, 532, 834, 155, 182, 517, 517, 517, 518, 517, 156, 181, 803, 803, 456, 456, 456, 456, 456, 456, 553, 553, 553, 554, 553, 155, 471, 471, 471, 471, 471, 471, 471, 471, 471, 471, 519, 797, 138, 691, 471, 471, 471, 471, 471, 471, 797, 755, 543, 543, 543, 544, 543, 555, 555, 555, 556, 555, 194, 519, 139, 138, 691, 471, 471, 471, 471, 471, 471, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 755, 193, 95, 712, 474, 474, 474, 474, 474, 474, 553, 553, 553, 554, 553, 565, 565, 565, 566, 565, 96, 207, 208, 194, 193, 95, 712, 474, 474, 474, 474, 474, 474, 475, 475, 475, 475, 475, 475, 475, 475, 475, 475, 207, 979, 749, 980, 475, 475, 475, 475, 475, 475, 567, 567, 567, 568, 567, 749, 492, 492, 492, 493, 492, 569, 569, 569, 570, 569, 392, 475, 475, 475, 475, 475, 475, 476, 476, 476, 476, 476, 476, 476, 476, 476, 476, 923, 713, 924, 925, 476, 476, 476, 476, 476, 476, 181, 579, 579, 579, 580, 579, 596, 92, 597, 598, 583, 583, 583, 583, 713, 93, 182, 476, 476, 476, 476, 476, 476, 181, 596, 94, 597, 598, 583, 583, 583, 583, 111, 567, 652, 581, 92, 93, 653, 92, 415, 415, 415, 416, 415, 623, 743, 606, 112, 430, 113, 431, 114, 607, 228, 111, 652, 743, 581, 582, 583, 583, 583, 584, 585, 586, 587, 583, 583, 606, 112, 583, 113, 583, 583, 583, 583, 583, 583, 394, 394, 394, 395, 394, 990, 92, 991, 92, 397, 397, 397, 398, 397, 92, 583, 92, 583, 583, 583, 583, 583, 583, 583, 736, 736, 588, 589, 590, 583, 591, 592, 593, 1009, 612, 601, 717, 1010, 594, 421, 595, 422, 394, 91, 399, 399, 399, 399, 400, 399, 583, 397, 97, 96, 583, 583, 583, 583, 601, 717, 98, 594, 99, 595, 583, 91, 729, 588, 589, 590, 583, 591, 592, 593, 97, 281, 95, 718, 1000, 599, 1001, 600, 98, 282, 405, 608, 729, 92, 91, 609, 100, 610, 100, 97, 96, 403, 101, 281, 101, 95, 718, 98, 599, 99, 600, 392, 392, 392, 393, 392, 91, 569, 100, 611, 100, 97, 401, 401, 401, 402, 401, 106, 583, 98, 207, 208, 583, 583, 583, 583, 403, 403, 403, 404, 403, 611, 107, 92, 104, 405, 405, 405, 406, 405, 106, 678, 207, 105, 93, 459, 407, 407, 407, 408, 407, 460, 706, 108, 94, 91, 104, 412, 109, 92, 1011, 706, 1012, 110, 102, 103, 93, 459, 625, 91, 104, 492, 263, 106, 264, 108, 543, 91, 91, 105, 109, 407, 407, 407, 408, 407, 102, 115, 107, 108, 661, 91, 104, 92, 109, 284, 106, 285, 699, 110, 91, 405, 405, 405, 406, 405, 699, 181, 583, 116, 115, 108, 583, 583, 583, 583, 109, 409, 409, 409, 410, 409, 722, 182, 108, 604, 409, 207, 208, 109, 181, 628, 583, 407, 110, 629, 583, 630, 723, 106, 583, 583, 583, 583, 228, 722, 108, 724, 604, 207, 602, 109, 695, 631, 107, 583, 181, 437, 111, 438, 91, 723, 106, 735, 695, 619, 112, 134, 113, 620, 114, 621, 182, 602, 112, 108, 113, 135, 114, 181, 109, 111, 91, 92, 92, 110, 735, 622, 112, 134, 113, 412, 412, 412, 413, 412, 112, 108, 113, 92, 690, 690, 109, 615, 615, 615, 616, 615, 583, 445, 622, 434, 583, 583, 583, 583, 615, 615, 615, 616, 615, 443, 603, 626, 626, 626, 627, 626, 683, 435, 447, 633, 737, 683, 434, 268, 136, 269, 634, 634, 634, 635, 634, 136, 116, 603, 636, 677, 677, 254, 141, 142, 143, 144, 137, 737, 255, 260, 92, 136, 138, 137, 254, 134, 658, 473, 136, 703, 462, 255, 463, 254, 704, 135, 705, 265, 261, 421, 458, 422, 260, 654, 139, 138, 254, 134, 655, 632, 656, 664, 669, 266, 632, 158, 670, 159, 671, 517, 265, 637, 637, 637, 637, 637, 637, 637, 637, 637, 637, 779, 468, 657, 787, 637, 637, 637, 637, 637, 637, 450, 450, 450, 450, 450, 674, 452, 624, 193, 675, 453, 676, 454, 779, 657, 624, 787, 637, 637, 637, 637, 637, 637, 638, 638, 638, 638, 638, 680, 1102, 194, 193, 681, 1102, 682, 454, 155, 482, 482, 482, 483, 482, 613, 156, 482, 482, 482, 483, 482, 613, 684, 788, 845, 455, 685, 686, 687, 688, 155, 678, 678, 678, 679, 678, 502, 502, 502, 503, 502, 504, 504, 504, 505, 504, 788, 845, 455, 644, 645, 646, 646, 646, 645, 647, 644, 647, 647, 647, 644, 644, 648, 647, 647, 647, 647, 649, 649, 649, 649, 649, 649, 649, 649, 649, 649, 647, 647, 647, 647, 649, 649, 649, 649, 649, 649, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 650, 649, 649, 649, 649, 649, 649, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 67, 67, 67, 151, 67, 92, 578, 662, 662, 662, 663, 662, 492, 492, 492, 493, 492, 651, 651, 651, 651, 651, 651, 651, 651, 651, 651, 757, 757, 757, 757, 651, 651, 651, 651, 651, 651, 502, 502, 502, 503, 502, 692, 708, 578, 574, 693, 696, 694, 181, 574, 697, 281, 698, 651, 651, 651, 651, 651, 651, 282, 562, 324, 709, 562, 182, 708, 714, 710, 551, 711, 715, 181, 716, 281, 163, 78, 78, 164, 163, 551, 28, 525, 525, 525, 526, 525, 700, 700, 700, 701, 700, 665, 665, 665, 665, 665, 665, 665, 665, 665, 665, 757, 757, 757, 757, 665, 665, 665, 665, 665, 665, 525, 525, 525, 526, 525, 430, 542, 431, 254, 702, 532, 532, 532, 533, 532, 255, 29, 665, 665, 665, 665, 665, 665, 82, 82, 82, 168, 82, 324, 719, 254, 780, 702, 542, 720, 781, 721, 30, 92, 516, 666, 666, 666, 666, 666, 666, 666, 666, 666, 666, 437, 516, 438, 780, 666, 666, 666, 666, 666, 666, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 32, 666, 666, 666, 666, 666, 666, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 512, 617, 30, 667, 667, 667, 667, 667, 667, 667, 667, 667, 667, 141, 142, 143, 144, 667, 667, 667, 667, 667, 667, 618, 617, 517, 517, 517, 518, 517, 532, 532, 532, 533, 532, 724, 724, 724, 725, 724, 667, 667, 667, 667, 667, 667, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 707, 462, 265, 463, 668, 668, 668, 668, 668, 668, 724, 724, 724, 725, 724, 726, 512, 827, 266, 727, 746, 728, 194, 707, 747, 265, 748, 668, 668, 668, 668, 668, 668, 543, 543, 543, 544, 543, 730, 828, 827, 846, 731, 732, 733, 734, 553, 553, 553, 554, 553, 555, 555, 555, 556, 555, 553, 553, 553, 554, 553, 738, 579, 750, 846, 739, 740, 741, 742, 565, 565, 565, 566, 565, 567, 567, 567, 568, 567, 569, 569, 569, 570, 569, 92, 207, 208, 750, 750, 750, 751, 750, 752, 227, 500, 227, 753, 500, 754, 579, 579, 579, 580, 579, 786, 1009, 789, 207, 609, 1010, 610, 421, 228, 422, 228, 92, 227, 770, 227, 771, 772, 757, 757, 757, 757, 770, 491, 771, 772, 757, 757, 757, 757, 756, 792, 793, 642, 833, 257, 794, 258, 795, 655, 491, 656, 640, 415, 415, 415, 416, 415, 1049, 228, 1050, 92, 487, 756, 757, 757, 757, 757, 758, 759, 760, 761, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 762, 763, 764, 757, 765, 766, 767, 757, 757, 757, 757, 643, 768, 487, 769, 615, 299, 757, 757, 757, 757, 757, 757, 181, 762, 763, 764, 757, 765, 766, 767, 854, 855, 856, 857, 768, 773, 769, 774, 182, 757, 757, 757, 757, 782, 858, 181, 473, 606, 783, 838, 784, 796, 778, 607, 284, 620, 285, 621, 773, 254, 774, 392, 392, 392, 393, 392, 255, 858, 798, 606, 466, 228, 430, 785, 431, 778, 482, 482, 482, 483, 482, 254, 757, 757, 757, 757, 260, 790, 790, 790, 791, 790, 801, 652, 860, 785, 263, 653, 264, 799, 799, 799, 800, 799, 93, 261, 802, 804, 626, 260, 629, 437, 630, 438, 94, 652, 466, 860, 805, 805, 805, 806, 805, 634, 458, 441, 93, 394, 394, 394, 395, 394, 418, 807, 878, 879, 441, 268, 427, 269, 419, 260, 433, 638, 638, 638, 638, 638, 757, 757, 757, 757, 428, 434, 418, 454, 433, 878, 879, 265, 261, 427, 775, 372, 260, 809, 809, 809, 809, 809, 609, 435, 610, 639, 829, 266, 434, 810, 640, 830, 96, 831, 265, 372, 865, 775, 397, 397, 397, 398, 397, 638, 638, 638, 638, 638, 835, 1102, 92, 92, 462, 1102, 463, 454, 882, 92, 832, 757, 757, 757, 757, 363, 638, 638, 638, 638, 638, 847, 1102, 363, 883, 848, 1102, 849, 454, 641, 92, 882, 832, 97, 646, 646, 646, 646, 646, 662, 92, 98, 620, 99, 621, 92, 454, 883, 455, 356, 638, 638, 638, 638, 638, 97, 1102, 629, 356, 630, 1102, 887, 454, 98, 399, 399, 399, 400, 399, 455, 638, 638, 638, 638, 638, 861, 1102, 349, 349, 862, 1102, 863, 454, 281, 887, 757, 757, 757, 757, 92, 92, 282, 836, 836, 836, 837, 836, 836, 836, 836, 837, 836, 655, 455, 656, 281, 482, 482, 482, 483, 482, 100, 678, 678, 678, 679, 678, 101, 502, 502, 502, 503, 502, 455, 323, 492, 492, 492, 493, 492, 875, 888, 92, 100, 401, 401, 401, 402, 401, 459, 896, 898, 867, 324, 459, 460, 868, 323, 869, 324, 460, 700, 331, 875, 888, 757, 757, 757, 757, 331, 92, 459, 181, 896, 898, 92, 459, 504, 504, 504, 505, 504, 502, 502, 502, 503, 502, 321, 182, 865, 865, 865, 866, 865, 323, 181, 102, 103, 700, 700, 700, 701, 700, 517, 517, 517, 518, 517, 525, 525, 525, 526, 525, 324, 783, 321, 784, 323, 102, 403, 403, 403, 404, 403, 525, 525, 525, 526, 525, 977, 314, 931, 708, 314, 874, 532, 532, 532, 533, 532, 757, 757, 757, 757, 532, 532, 532, 533, 532, 92, 228, 324, 977, 884, 931, 708, 194, 874, 885, 794, 886, 795, 418, 104, 724, 724, 724, 725, 724, 814, 419, 92, 105, 724, 724, 724, 725, 724, 640, 543, 543, 543, 544, 543, 418, 104, 405, 405, 405, 406, 405, 892, 893, 894, 895, 553, 553, 553, 554, 553, 555, 555, 555, 556, 555, 92, 299, 757, 757, 757, 757, 553, 553, 553, 554, 553, 899, 900, 901, 902, 167, 643, 181, 978, 106, 565, 565, 565, 566, 565, 154, 207, 208, 567, 567, 567, 568, 567, 182, 107, 569, 569, 569, 570, 569, 181, 978, 106, 407, 407, 407, 408, 407, 207, 906, 273, 273, 983, 907, 985, 908, 750, 750, 750, 751, 750, 252, 915, 916, 917, 427, 918, 919, 920, 579, 579, 579, 580, 579, 921, 983, 922, 985, 92, 428, 392, 392, 392, 393, 392, 937, 108, 827, 427, 780, 783, 109, 784, 781, 91, 92, 110, 921, 92, 922, 915, 916, 917, 913, 918, 919, 920, 92, 108, 828, 827, 780, 926, 109, 927, 790, 91, 409, 409, 409, 410, 409, 228, 93, 92, 995, 913, 394, 394, 394, 395, 394, 830, 94, 831, 926, 92, 927, 415, 415, 415, 416, 415, 939, 324, 93, 434, 609, 995, 610, 776, 92, 92, 397, 397, 397, 398, 397, 942, 418, 91, 92, 928, 421, 435, 422, 112, 419, 113, 434, 114, 92, 92, 776, 399, 399, 399, 400, 399, 933, 96, 418, 91, 943, 934, 928, 935, 794, 112, 795, 113, 412, 412, 412, 413, 412, 97, 92, 965, 401, 401, 401, 402, 401, 98, 92, 99, 966, 934, 89, 935, 936, 757, 757, 757, 757, 167, 162, 97, 945, 965, 100, 777, 620, 162, 621, 98, 101, 403, 403, 403, 404, 403, 936, 154, 996, 405, 405, 405, 406, 405, 799, 997, 100, 116, 777, 405, 405, 405, 406, 405, 102, 103, 407, 407, 407, 408, 407, 996, 132, 412, 412, 412, 413, 412, 997, 92, 757, 757, 757, 757, 948, 104, 102, 106, 430, 949, 431, 965, 427, 629, 105, 630, 805, 106, 92, 952, 966, 90, 107, 437, 930, 438, 428, 104, 89, 108, 106, 970, 107, 965, 109, 427, 830, 87, 831, 110, 106, 407, 407, 407, 408, 407, 116, 930, 967, 975, 434, 108, 968, 462, 969, 463, 109, 940, 940, 940, 941, 940, 950, 950, 950, 951, 950, 998, 435, 968, 1039, 969, 1040, 434, 405, 405, 405, 406, 405, 809, 809, 809, 809, 809, 108, 999, 81, 71, 972, 109, 998, 810, 91, 655, 110, 656, 407, 407, 407, 408, 407, 986, 70, 459, 606, 987, 108, 988, 999, 460, 607, 109, 106, 1002, 91, 808, 808, 808, 808, 808, 808, 808, 808, 808, 808, 459, 606, 107, 1003, 808, 808, 808, 808, 808, 808, 106, 1002, 69, 1017, 108, 1005, 1007, 1022, 934, 109, 935, 609, 57, 610, 110, 617, 1003, 808, 808, 808, 808, 808, 808, 639, 639, 812, 108, 1053, 1005, 1007, 39, 109, 946, 946, 946, 947, 946, 618, 617, 813, 813, 813, 813, 813, 813, 813, 813, 813, 813, 1102, 1053, 1054, 1072, 813, 813, 813, 813, 813, 813, 1068, 1102, 1069, 1102, 617, 482, 482, 482, 483, 482, 482, 482, 482, 483, 482, 1054, 1072, 813, 813, 813, 813, 813, 813, 642, 642, 815, 618, 617, 678, 678, 678, 679, 678, 502, 502, 502, 503, 502, 1102, 816, 816, 816, 816, 816, 816, 816, 816, 816, 816, 1102, 1073, 1102, 1102, 816, 816, 816, 816, 816, 816, 638, 638, 638, 638, 638, 1023, 1102, 1102, 1019, 794, 1102, 795, 454, 783, 1073, 784, 1102, 816, 816, 816, 816, 816, 816, 817, 817, 817, 818, 817, 606, 1102, 1102, 1102, 1026, 1102, 607, 454, 620, 1102, 621, 1102, 819, 819, 819, 819, 819, 819, 819, 819, 819, 819, 606, 1102, 1102, 455, 819, 819, 819, 819, 819, 819, 504, 504, 504, 505, 504, 502, 502, 502, 503, 502, 865, 865, 865, 866, 865, 1102, 455, 819, 819, 819, 819, 819, 819, 820, 821, 638, 638, 638, 821, 822, 820, 822, 822, 822, 820, 820, 823, 822, 822, 822, 822, 824, 824, 824, 824, 824, 824, 824, 824, 824, 824, 822, 822, 822, 822, 824, 824, 824, 824, 824, 824, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 825, 824, 824, 824, 824, 824, 824, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 826, 826, 826, 826, 826, 826, 826, 826, 826, 826, 1102, 1102, 1102, 1102, 826, 826, 826, 826, 826, 826, 638, 817, 638, 638, 638, 1027, 1102, 1102, 1044, 629, 1102, 630, 454, 830, 1102, 831, 1102, 826, 826, 826, 826, 826, 826, 163, 78, 78, 164, 163, 827, 28, 525, 525, 525, 526, 525, 525, 525, 525, 526, 525, 839, 839, 839, 839, 839, 839, 839, 839, 839, 839, 828, 827, 1102, 455, 839, 839, 839, 839, 839, 839, 646, 646, 646, 646, 646, 973, 973, 973, 974, 973, 1102, 1036, 454, 1102, 1102, 1037, 29, 839, 839, 839, 839, 839, 839, 840, 840, 840, 840, 840, 840, 840, 840, 840, 840, 1036, 1102, 1102, 1102, 840, 840, 840, 840, 840, 840, 638, 638, 638, 638, 638, 517, 517, 517, 518, 517, 962, 652, 454, 1102, 1102, 653, 1102, 840, 840, 840, 840, 840, 840, 841, 841, 841, 841, 841, 841, 841, 841, 841, 841, 652, 1102, 1102, 994, 841, 841, 841, 841, 841, 841, 1102, 638, 638, 638, 638, 638, 1102, 1102, 1042, 1102, 962, 1102, 968, 454, 969, 194, 994, 841, 841, 841, 841, 841, 841, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 1102, 1102, 1102, 1102, 842, 842, 842, 842, 842, 842, 1102, 1102, 940, 940, 940, 941, 940, 836, 1102, 1102, 1102, 455, 532, 532, 532, 533, 532, 842, 842, 842, 842, 842, 842, 914, 914, 914, 914, 914, 914, 914, 914, 914, 914, 1102, 1102, 1102, 1102, 914, 914, 914, 914, 914, 914, 946, 646, 646, 646, 646, 646, 606, 652, 1102, 1102, 459, 653, 607, 454, 1102, 1102, 460, 914, 914, 914, 914, 914, 914, 409, 409, 409, 410, 409, 606, 652, 617, 1102, 459, 163, 78, 78, 164, 163, 1102, 28, 532, 532, 532, 533, 532, 724, 724, 724, 725, 724, 1102, 780, 618, 617, 962, 781, 929, 724, 724, 724, 725, 724, 553, 553, 553, 554, 553, 555, 555, 555, 556, 555, 112, 780, 113, 1102, 114, 1102, 1102, 929, 553, 553, 553, 554, 553, 1102, 29, 565, 565, 565, 566, 565, 1102, 1102, 112, 1102, 113, 953, 953, 953, 953, 953, 953, 953, 953, 953, 953, 1102, 1102, 1102, 1102, 953, 953, 953, 953, 953, 953, 492, 492, 492, 493, 492, 567, 567, 567, 568, 567, 569, 569, 569, 570, 569, 1102, 1102, 953, 953, 953, 953, 953, 953, 954, 811, 811, 955, 954, 1102, 640, 750, 750, 750, 751, 750, 1102, 1102, 181, 1102, 1102, 956, 956, 956, 956, 956, 956, 956, 956, 956, 956, 1102, 1102, 1102, 182, 956, 956, 956, 956, 956, 956, 181, 1024, 1024, 1024, 1025, 1024, 148, 148, 148, 275, 148, 678, 678, 678, 679, 678, 641, 956, 956, 956, 956, 956, 956, 957, 814, 814, 958, 957, 1036, 1047, 1056, 1102, 1037, 640, 655, 934, 656, 935, 1102, 1102, 959, 959, 959, 959, 959, 959, 959, 959, 959, 959, 1036, 1102, 1102, 1102, 959, 959, 959, 959, 959, 959, 700, 700, 700, 701, 700, 1102, 543, 543, 543, 544, 543, 865, 865, 865, 866, 865, 643, 959, 959, 959, 959, 959, 959, 960, 960, 960, 960, 960, 960, 960, 960, 960, 960, 1102, 875, 1102, 1102, 960, 960, 960, 960, 960, 960, 117, 117, 117, 232, 117, 1060, 1102, 1102, 811, 794, 324, 795, 1102, 640, 875, 207, 208, 960, 960, 960, 960, 960, 960, 821, 821, 821, 963, 821, 724, 724, 724, 725, 724, 1102, 1102, 1102, 1102, 207, 1102, 1102, 964, 964, 964, 964, 964, 964, 964, 964, 964, 964, 1102, 1102, 1102, 92, 964, 964, 964, 964, 964, 964, 641, 724, 724, 724, 725, 724, 750, 750, 750, 751, 750, 1057, 1057, 1057, 1058, 1057, 1102, 964, 964, 964, 964, 964, 964, 644, 645, 646, 646, 646, 645, 647, 644, 647, 647, 647, 644, 644, 648, 647, 647, 647, 647, 649, 649, 649, 649, 649, 649, 649, 649, 649, 649, 647, 647, 647, 647, 649, 649, 649, 649, 649, 649, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 650, 649, 649, 649, 649, 649, 649, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 1102, 1102, 1102, 1102, 64, 64, 64, 64, 64, 64, 579, 579, 579, 580, 579, 412, 412, 412, 413, 412, 1071, 1036, 1102, 1102, 1039, 1037, 1040, 64, 64, 64, 64, 64, 64, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 1036, 1015, 1102, 115, 30, 30, 30, 30, 30, 30, 394, 394, 394, 395, 394, 1102, 1102, 1038, 1102, 1102, 228, 1039, 1102, 1040, 1015, 116, 115, 30, 30, 30, 30, 30, 30, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 1102, 1041, 95, 1102, 33, 33, 33, 33, 33, 33, 1102, 392, 392, 392, 393, 392, 1102, 1059, 1102, 1102, 96, 1102, 783, 1041, 784, 95, 1102, 33, 33, 33, 33, 33, 33, 976, 976, 976, 976, 976, 976, 976, 976, 976, 976, 1102, 1102, 1102, 1102, 976, 976, 976, 976, 976, 976, 1102, 93, 397, 397, 397, 398, 397, 1102, 1102, 973, 940, 94, 399, 399, 399, 400, 399, 976, 976, 976, 976, 976, 976, 93, 401, 401, 401, 402, 401, 403, 403, 403, 404, 403, 1102, 1102, 405, 405, 405, 406, 405, 1045, 1102, 1102, 1102, 97, 1102, 1102, 407, 407, 407, 408, 407, 98, 1074, 99, 606, 652, 968, 100, 969, 653, 607, 1077, 1102, 101, 1102, 97, 830, 1102, 831, 827, 1102, 104, 106, 98, 102, 103, 606, 652, 1102, 100, 105, 415, 415, 415, 416, 415, 1102, 107, 1102, 108, 1102, 828, 827, 104, 109, 106, 102, 1102, 1102, 110, 405, 405, 405, 406, 405, 1102, 407, 407, 407, 408, 407, 108, 1075, 1102, 1102, 1102, 109, 409, 409, 409, 410, 409, 1102, 1102, 1102, 117, 117, 117, 232, 117, 1079, 1102, 1102, 1102, 92, 934, 1102, 935, 106, 117, 117, 117, 232, 117, 117, 117, 117, 232, 117, 108, 111, 1085, 1102, 107, 109, 965, 1068, 1102, 1069, 110, 1102, 106, 181, 1087, 966, 1102, 112, 1039, 113, 1040, 114, 108, 1102, 111, 1102, 1102, 109, 965, 182, 117, 117, 117, 232, 117, 1102, 181, 1102, 1090, 112, 1095, 113, 968, 92, 969, 1068, 1102, 1069, 92, 1020, 1020, 1020, 1021, 1020, 646, 646, 646, 646, 646, 638, 821, 638, 638, 638, 227, 1102, 454, 1102, 1102, 1102, 1102, 454, 865, 865, 865, 866, 865, 1102, 1045, 1045, 1045, 1046, 1045, 228, 1102, 1102, 1102, 227, 175, 175, 175, 300, 175, 482, 482, 482, 483, 482, 780, 1098, 1102, 1067, 781, 1039, 1020, 1040, 1068, 962, 1069, 827, 1102, 1102, 962, 1096, 1096, 1096, 1097, 1096, 1102, 1102, 780, 1028, 1028, 1028, 1028, 1028, 1028, 1028, 1028, 1028, 1028, 828, 827, 1070, 1102, 1028, 1028, 1028, 1028, 1028, 1028, 92, 1102, 1102, 1102, 1102, 92, 482, 482, 482, 483, 482, 780, 1102, 1102, 1070, 781, 1102, 1028, 1028, 1028, 1028, 1028, 1028, 1029, 1029, 1029, 1029, 1029, 1029, 1029, 1029, 1029, 1029, 780, 1102, 1102, 1102, 1029, 1029, 1029, 1029, 1029, 1029, 502, 502, 502, 503, 502, 504, 504, 504, 505, 504, 821, 821, 821, 963, 821, 1102, 92, 1029, 1029, 1029, 1029, 1029, 1029, 1030, 1030, 1030, 1030, 1030, 1030, 1030, 1030, 1030, 1030, 1102, 1102, 1102, 1102, 1030, 1030, 1030, 1030, 1030, 1030, 1100, 1102, 1102, 1102, 1102, 1068, 1102, 1069, 1102, 1102, 92, 1102, 1102, 1102, 1102, 92, 1102, 1030, 1030, 1030, 1030, 1030, 1030, 820, 1031, 646, 646, 646, 1031, 1032, 820, 1032, 1032, 1032, 820, 820, 823, 1032, 1032, 1032, 1032, 1033, 1033, 1033, 1033, 1033, 1033, 1033, 1033, 1033, 1033, 1032, 1032, 1032, 1032, 1033, 1033, 1033, 1033, 1033, 1033, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1034, 1033, 1033, 1033, 1033, 1033, 1033, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1035, 1035, 1035, 1035, 1035, 1035, 1035, 1035, 1035, 1035, 1102, 1102, 1102, 1102, 1035, 1035, 1035, 1035, 1035, 1035, 492, 492, 492, 493, 492, 502, 502, 502, 503, 502, 700, 700, 700, 701, 700, 1102, 1102, 1035, 1035, 1035, 1035, 1035, 1035, 517, 517, 517, 518, 517, 175, 175, 175, 300, 175, 1102, 1102, 1102, 1102, 181, 175, 175, 175, 300, 175, 1102, 995, 525, 525, 525, 526, 525, 1102, 1102, 1102, 182, 193, 1102, 1102, 1102, 92, 1102, 181, 1102, 323, 324, 1102, 1102, 1102, 995, 525, 525, 525, 526, 525, 1102, 1102, 1102, 194, 193, 1102, 1102, 1102, 324, 1102, 1102, 1102, 323, 532, 532, 532, 533, 532, 92, 532, 532, 532, 533, 532, 1102, 92, 543, 543, 543, 544, 543, 553, 553, 553, 554, 553, 555, 555, 555, 556, 555, 553, 553, 553, 554, 553, 1102, 1088, 92, 565, 565, 565, 566, 565, 567, 567, 567, 568, 567, 569, 569, 569, 570, 569, 1102, 1102, 92, 579, 579, 579, 580, 579, 92, 175, 175, 175, 300, 175, 207, 208, 1102, 1102, 1102, 1102, 92, 1102, 1102, 1102, 1102, 92, 1102, 1102, 1102, 1102, 92, 1036, 1102, 1102, 1102, 1037, 207, 227, 92, 1102, 1102, 1102, 1102, 92, 1102, 1102, 1102, 1102, 92, 1102, 1102, 1102, 1102, 1036, 1102, 1102, 228, 1102, 1102, 1102, 227, 1102, 92, 1061, 1061, 1061, 1061, 1061, 1061, 1061, 1061, 1061, 1061, 1102, 1102, 1102, 1102, 1061, 1061, 1061, 1061, 1061, 1061, 646, 646, 646, 646, 646, 678, 678, 678, 679, 678, 1102, 1102, 454, 1102, 1102, 1102, 1102, 1061, 1061, 1061, 1061, 1061, 1061, 1062, 1062, 1062, 1062, 1062, 1062, 1062, 1062, 1062, 1062, 1102, 1102, 1102, 1102, 1062, 1062, 1062, 1062, 1062, 1062, 646, 646, 646, 646, 646, 1075, 1075, 1075, 1076, 1075, 962, 1102, 454, 1102, 1102, 92, 1102, 1062, 1062, 1062, 1062, 1062, 1062, 1063, 1063, 1063, 1063, 1063, 1063, 1063, 1063, 1063, 1063, 1102, 1102, 1102, 1102, 1063, 1063, 1063, 1063, 1063, 1063, 724, 724, 724, 725, 724, 1102, 965, 1102, 1102, 1102, 962, 1102, 1102, 1102, 1102, 966, 1102, 1063, 1063, 1063, 1063, 1063, 1063, 1031, 1031, 1031, 1064, 1031, 965, 1102, 1102, 1102, 1102, 1102, 1102, 454, 1102, 1102, 1102, 1102, 1065, 1065, 1065, 1065, 1065, 1065, 1065, 1065, 1065, 1065, 1102, 1102, 1102, 92, 1065, 1065, 1065, 1065, 1065, 1065, 700, 700, 700, 701, 700, 724, 724, 724, 725, 724, 1102, 1102, 1102, 1102, 1102, 1102, 962, 1065, 1065, 1065, 1065, 1065, 1065, 1066, 1066, 1066, 1066, 1066, 1066, 1066, 1066, 1066, 1066, 1102, 323, 1102, 1102, 1066, 1066, 1066, 1066, 1066, 1066, 750, 750, 750, 751, 750, 1102, 1102, 1102, 1102, 1102, 324, 1102, 1102, 1102, 323, 92, 1102, 1066, 1066, 1066, 1066, 1066, 1066, 646, 1031, 646, 646, 646, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 454, 1088, 1088, 1088, 1089, 1088, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 92, 1080, 1080, 1080, 1080, 1080, 1080, 1080, 1080, 1080, 1080, 1102, 1102, 1102, 1102, 1080, 1080, 1080, 1080, 1080, 1080, 1102, 1102, 962, 1102, 1102, 1088, 1088, 1088, 1089, 1088, 1102, 1036, 1102, 1102, 1102, 1037, 1102, 1080, 1080, 1080, 1080, 1080, 1080, 1081, 1081, 1081, 1081, 1081, 1081, 1081, 1081, 1081, 1081, 1036, 1102, 1102, 1102, 1081, 1081, 1081, 1081, 1081, 1081, 865, 865, 865, 866, 865, 1102, 1102, 1102, 1102, 1102, 1102, 1036, 1102, 1102, 1102, 1037, 1102, 1081, 1081, 1081, 1081, 1081, 1081, 1082, 1082, 1082, 1082, 1082, 1082, 1082, 1082, 1082, 1082, 1036, 1102, 1102, 1102, 1082, 1082, 1082, 1082, 1082, 1082, 1031, 1031, 1031, 1064, 1031, 1102, 1102, 1102, 1102, 1102, 92, 1102, 454, 1102, 1102, 1102, 1102, 1082, 1082, 1082, 1082, 1082, 1082, 1083, 1083, 1083, 1083, 1083, 1083, 1083, 1083, 1083, 1083, 1102, 1102, 1102, 1102, 1083, 1083, 1083, 1083, 1083, 1083, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 962, 1102, 1102, 1102, 1102, 1102, 1102, 1083, 1083, 1083, 1083, 1083, 1083, 1084, 1084, 1084, 1084, 1084, 1084, 1084, 1084, 1084, 1084, 1102, 1102, 1102, 1102, 1084, 1084, 1084, 1084, 1084, 1084, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1084, 1084, 1084, 1084, 1084, 1084, 1091, 1091, 1091, 1091, 1091, 1091, 1091, 1091, 1091, 1091, 1102, 1102, 1102, 1102, 1091, 1091, 1091, 1091, 1091, 1091, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1091, 1091, 1091, 1091, 1091, 1091, 1092, 1092, 1092, 1092, 1092, 1092, 1092, 1092, 1092, 1092, 1102, 1102, 1102, 1102, 1092, 1092, 1092, 1092, 1092, 1092, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1092, 1092, 1092, 1092, 1092, 1092, 451, 451, 451, 451, 451, 451, 451, 451, 451, 451, 1102, 1102, 1102, 1102, 451, 451, 451, 451, 451, 451, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 451, 451, 451, 451, 451, 451, 1093, 1093, 1093, 1093, 1093, 1093, 1093, 1093, 1093, 1093, 1102, 1102, 1102, 1102, 1093, 1093, 1093, 1093, 1093, 1093, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1093, 1093, 1093, 1093, 1093, 1093, 1094, 1094, 1094, 1094, 1094, 1094, 1094, 1094, 1094, 1094, 1102, 1102, 1102, 1102, 1094, 1094, 1094, 1094, 1094, 1094, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1094, 1094, 1094, 1094, 1094, 1094, 639, 639, 639, 639, 639, 639, 639, 639, 639, 639, 1102, 1102, 1102, 1102, 639, 639, 639, 639, 639, 639, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 639, 639, 639, 639, 639, 639, 642, 642, 642, 642, 642, 642, 642, 642, 642, 642, 1102, 1102, 1102, 1102, 642, 642, 642, 642, 642, 642, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 642, 642, 642, 642, 642, 642, 1099, 1099, 1099, 1099, 1099, 1099, 1099, 1099, 1099, 1099, 1102, 1102, 1102, 1102, 1099, 1099, 1099, 1099, 1099, 1099, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1099, 1099, 1099, 1099, 1099, 1099, 1101, 1101, 1101, 1101, 1101, 1101, 1101, 1101, 1101, 1101, 1102, 1102, 1102, 1102, 1101, 1101, 1101, 1101, 1101, 1101, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1101, 1101, 1101, 1101, 1101, 1101, 27, 27, 1102, 27, 27, 27, 27, 27, 27, 30, 30, 30, 30, 33, 33, 1102, 33, 33, 33, 33, 33, 33, 36, 1102, 1102, 36, 64, 64, 1102, 64, 64, 67, 67, 1102, 67, 67, 67, 67, 67, 67, 78, 78, 78, 78, 78, 78, 78, 78, 78, 82, 82, 1102, 82, 82, 82, 82, 82, 82, 84, 84, 84, 84, 84, 84, 84, 84, 84, 88, 88, 88, 88, 88, 88, 88, 88, 88, 91, 1102, 91, 91, 148, 148, 1102, 148, 148, 148, 148, 148, 148, 153, 153, 153, 153, 153, 153, 153, 153, 153, 166, 166, 166, 166, 166, 166, 166, 166, 166, 173, 173, 173, 173, 173, 173, 173, 173, 173, 175, 175, 1102, 175, 175, 175, 175, 175, 175, 279, 279, 279, 279, 279, 279, 279, 279, 279, 294, 294, 294, 294, 294, 294, 294, 294, 294, 298, 298, 298, 298, 298, 298, 298, 298, 298, 451, 451, 451, 1102, 451, 451, 451, 451, 457, 457, 457, 457, 457, 457, 457, 457, 457, 472, 472, 472, 472, 472, 472, 472, 472, 472, 173, 173, 173, 173, 173, 173, 173, 173, 173, 639, 639, 1102, 639, 639, 639, 639, 639, 639, 642, 642, 1102, 642, 642, 642, 642, 642, 642, 457, 457, 457, 457, 457, 457, 457, 457, 457, 279, 279, 279, 279, 279, 279, 279, 279, 279, 27, 27, 27, 27, 27, 27, 27, 27, 27, 472, 472, 472, 472, 472, 472, 472, 472, 472, 294, 294, 294, 294, 294, 294, 294, 294, 294, 811, 811, 811, 811, 811, 811, 811, 811, 811, 814, 814, 814, 814, 814, 814, 814, 814, 814, 961, 961, 1102, 1102, 961, 961, 961, 961, 3, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102 } ; static const flex_int16_t yy_chk[7190] = { 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 7, 7, 7, 7, 7, 8, 9, 10, 11, 17, 22, 22, 22, 22, 22, 36, 7, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 25, 30, 82, 470, 1113, 470, 58, 27, 16, 46, 25, 26, 7, 64, 16, 26, 9, 26, 16, 1097, 11, 16, 25, 10, 17, 8, 7, 46, 16, 58, 36, 16, 46, 26, 133, 7, 14, 16, 30, 82, 16, 14, 14, 16, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 27, 64, 26, 133, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 18, 33, 14, 38, 67, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 49, 49, 23, 23, 23, 23, 23, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 48, 23, 1069, 45, 18, 49, 48, 84, 78, 1068, 18, 74, 33, 45, 67, 74, 89, 74, 154, 89, 167, 154, 48, 167, 23, 45, 18, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 51, 23, 170, 50, 19, 19, 19, 19, 19, 19, 23, 1058, 50, 84, 47, 51, 52, 54, 78, 61, 1050, 52, 47, 51, 47, 50, 52, 19, 19, 19, 19, 19, 19, 29, 29, 29, 47, 59, 52, 54, 54, 61, 61, 52, 47, 170, 664, 59, 664, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 59, 60, 148, 72, 29, 29, 29, 29, 29, 29, 72, 150, 79, 85, 66, 299, 195, 79, 299, 60, 1049, 195, 85, 195, 60, 72, 1025, 29, 29, 29, 29, 29, 29, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 53, 148, 66, 126, 32, 32, 32, 32, 32, 32, 150, 73, 126, 66, 1012, 73, 53, 73, 53, 100, 53, 79, 85, 53, 66, 126, 77, 32, 32, 32, 32, 32, 32, 34, 34, 34, 77, 73, 53, 124, 53, 100, 100, 222, 1011, 124, 95, 222, 77, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 73, 151, 124, 95, 34, 34, 34, 34, 34, 34, 95, 151, 224, 991, 990, 163, 224, 96, 105, 988, 453, 96, 105, 96, 105, 987, 980, 34, 34, 34, 34, 34, 34, 37, 37, 37, 37, 37, 37, 37, 37, 37, 37, 96, 105, 105, 130, 37, 37, 37, 37, 37, 37, 151, 119, 979, 106, 106, 119, 119, 119, 119, 163, 453, 969, 96, 105, 105, 130, 130, 37, 37, 37, 37, 37, 37, 56, 56, 106, 968, 951, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 62, 115, 935, 134, 62, 62, 62, 62, 137, 669, 134, 669, 137, 158, 137, 116, 125, 125, 253, 116, 115, 116, 62, 138, 115, 134, 62, 127, 121, 62, 121, 121, 121, 121, 121, 121, 158, 934, 125, 138, 253, 116, 127, 139, 932, 62, 138, 139, 62, 139, 127, 62, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 911, 116, 159, 136, 65, 65, 65, 65, 65, 65, 458, 157, 910, 458, 99, 157, 141, 157, 99, 99, 99, 99, 136, 129, 141, 159, 136, 65, 65, 65, 65, 65, 65, 68, 68, 68, 68, 68, 160, 129, 99, 129, 642, 129, 68, 160, 129, 141, 99, 147, 68, 68, 68, 68, 68, 68, 68, 68, 68, 68, 160, 129, 99, 129, 68, 68, 68, 68, 68, 68, 99, 147, 147, 360, 103, 908, 155, 360, 103, 103, 103, 103, 232, 452, 155, 642, 68, 68, 68, 68, 68, 68, 68, 80, 80, 80, 80, 80, 155, 80, 103, 123, 907, 123, 123, 123, 123, 123, 123, 103, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 473, 178, 103, 473, 80, 80, 80, 80, 80, 80, 452, 103, 135, 901, 232, 140, 135, 143, 135, 140, 140, 140, 140, 146, 178, 143, 80, 80, 80, 80, 80, 80, 80, 83, 83, 83, 83, 83, 135, 161, 179, 146, 680, 161, 680, 161, 146, 899, 143, 83, 83, 83, 83, 83, 83, 83, 83, 83, 83, 83, 135, 145, 187, 179, 83, 83, 83, 83, 83, 83, 156, 145, 177, 895, 893, 156, 177, 156, 177, 164, 703, 890, 703, 145, 164, 187, 83, 83, 83, 83, 83, 83, 83, 86, 86, 86, 86, 86, 183, 156, 189, 186, 183, 86, 183, 186, 186, 186, 186, 300, 86, 86, 86, 86, 86, 86, 86, 86, 86, 86, 168, 156, 814, 189, 86, 86, 86, 86, 86, 86, 194, 164, 198, 199, 194, 168, 194, 101, 199, 200, 199, 101, 101, 101, 101, 193, 86, 86, 86, 86, 86, 86, 86, 90, 198, 198, 194, 90, 90, 90, 90, 300, 200, 101, 193, 814, 128, 101, 193, 201, 889, 128, 168, 90, 90, 90, 128, 90, 194, 90, 204, 171, 90, 90, 90, 101, 872, 362, 128, 101, 171, 362, 201, 128, 205, 214, 90, 90, 90, 90, 709, 90, 709, 204, 90, 90, 90, 92, 92, 92, 92, 92, 92, 92, 92, 92, 92, 205, 214, 216, 257, 92, 92, 92, 92, 92, 92, 871, 162, 162, 162, 162, 162, 171, 203, 209, 869, 258, 203, 209, 203, 209, 216, 257, 92, 92, 92, 92, 92, 92, 94, 94, 94, 94, 94, 94, 94, 94, 94, 94, 258, 639, 868, 865, 94, 94, 94, 94, 94, 94, 559, 162, 260, 107, 559, 863, 94, 107, 110, 107, 862, 162, 110, 110, 110, 110, 260, 94, 94, 94, 94, 94, 94, 162, 107, 260, 114, 265, 94, 107, 114, 114, 114, 114, 110, 212, 212, 213, 639, 110, 254, 213, 213, 213, 213, 265, 286, 107, 254, 182, 265, 107, 114, 182, 286, 182, 110, 212, 114, 220, 114, 110, 254, 220, 220, 220, 220, 229, 286, 182, 857, 229, 228, 229, 114, 182, 228, 561, 228, 855, 114, 561, 114, 118, 118, 118, 118, 118, 303, 234, 304, 852, 182, 234, 234, 234, 234, 182, 228, 316, 118, 118, 118, 118, 118, 118, 118, 118, 118, 118, 714, 303, 714, 304, 118, 118, 118, 118, 118, 118, 228, 256, 316, 259, 255, 256, 416, 256, 208, 255, 259, 255, 208, 208, 208, 208, 118, 118, 118, 118, 118, 118, 118, 120, 851, 259, 120, 120, 120, 120, 120, 120, 120, 255, 208, 261, 318, 333, 120, 261, 120, 261, 236, 208, 236, 236, 236, 236, 236, 236, 242, 242, 242, 242, 242, 255, 208, 849, 416, 318, 333, 120, 483, 120, 122, 208, 334, 122, 122, 122, 122, 122, 122, 122, 305, 305, 305, 305, 305, 122, 238, 122, 238, 238, 238, 238, 238, 238, 262, 334, 275, 283, 262, 242, 262, 267, 283, 242, 283, 267, 275, 267, 122, 242, 122, 149, 149, 149, 149, 149, 848, 281, 266, 831, 483, 242, 266, 281, 266, 242, 338, 830, 149, 149, 149, 149, 149, 149, 149, 149, 149, 149, 266, 281, 339, 343, 149, 149, 149, 149, 149, 149, 275, 338, 249, 249, 249, 249, 249, 726, 282, 726, 795, 344, 282, 266, 282, 339, 343, 149, 149, 149, 149, 149, 149, 152, 152, 152, 152, 152, 152, 152, 152, 152, 152, 249, 344, 794, 282, 152, 152, 152, 152, 152, 152, 270, 752, 784, 752, 270, 270, 270, 270, 957, 783, 287, 351, 249, 249, 287, 282, 287, 152, 152, 152, 152, 152, 152, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 754, 351, 292, 417, 165, 165, 165, 165, 165, 165, 753, 272, 272, 272, 272, 272, 290, 302, 957, 328, 290, 302, 290, 302, 328, 417, 328, 165, 165, 165, 165, 165, 165, 169, 169, 169, 169, 169, 169, 169, 169, 169, 169, 291, 292, 353, 272, 169, 169, 169, 169, 169, 169, 291, 292, 750, 792, 503, 792, 241, 241, 241, 241, 241, 272, 291, 292, 779, 353, 272, 169, 169, 169, 169, 169, 169, 172, 172, 172, 172, 172, 172, 172, 172, 172, 172, 401, 401, 748, 779, 172, 172, 172, 172, 172, 172, 306, 306, 306, 306, 306, 811, 241, 244, 244, 244, 244, 244, 401, 503, 241, 405, 241, 172, 172, 172, 172, 172, 172, 176, 176, 176, 176, 176, 241, 307, 405, 747, 332, 307, 741, 307, 241, 332, 405, 332, 176, 176, 176, 176, 176, 176, 176, 176, 176, 176, 244, 244, 811, 392, 176, 176, 176, 176, 176, 176, 244, 739, 954, 392, 273, 273, 273, 273, 273, 734, 311, 412, 244, 244, 311, 392, 311, 176, 176, 176, 176, 176, 176, 233, 233, 233, 233, 233, 233, 233, 233, 233, 233, 412, 412, 273, 462, 233, 233, 233, 233, 233, 233, 240, 240, 240, 240, 240, 801, 954, 801, 505, 271, 271, 271, 271, 271, 273, 273, 462, 233, 233, 233, 233, 233, 233, 235, 526, 533, 235, 235, 235, 235, 235, 235, 235, 325, 732, 240, 463, 325, 235, 325, 235, 728, 727, 240, 468, 243, 243, 243, 243, 243, 315, 468, 271, 240, 315, 315, 315, 315, 240, 463, 505, 235, 271, 235, 237, 240, 468, 237, 237, 237, 237, 237, 237, 237, 271, 724, 526, 533, 403, 237, 478, 237, 245, 245, 245, 245, 245, 403, 243, 246, 246, 246, 246, 246, 402, 337, 705, 243, 243, 337, 403, 337, 237, 478, 237, 239, 239, 239, 239, 239, 243, 704, 251, 251, 251, 251, 251, 346, 479, 243, 245, 346, 239, 346, 554, 245, 239, 239, 239, 239, 324, 698, 246, 342, 324, 245, 324, 246, 342, 697, 342, 479, 246, 245, 556, 402, 402, 245, 239, 250, 250, 250, 250, 250, 246, 251, 324, 350, 239, 246, 251, 350, 350, 350, 350, 251, 759, 402, 759, 759, 239, 247, 247, 247, 247, 247, 554, 251, 324, 358, 250, 694, 251, 358, 358, 358, 358, 250, 288, 288, 288, 288, 288, 807, 693, 807, 556, 274, 274, 274, 274, 274, 250, 250, 247, 277, 277, 277, 277, 277, 250, 277, 366, 566, 247, 277, 366, 277, 366, 369, 247, 397, 247, 369, 247, 369, 740, 247, 688, 397, 740, 397, 274, 374, 288, 686, 247, 374, 374, 374, 374, 288, 247, 397, 247, 248, 248, 248, 248, 248, 274, 397, 838, 399, 838, 274, 288, 424, 277, 399, 742, 424, 248, 424, 742, 566, 248, 248, 248, 248, 320, 320, 320, 320, 320, 399, 248, 280, 280, 280, 280, 280, 321, 321, 321, 321, 321, 322, 322, 322, 322, 322, 495, 280, 583, 583, 583, 583, 248, 248, 276, 276, 276, 276, 276, 335, 335, 335, 335, 335, 336, 336, 336, 336, 336, 495, 280, 276, 276, 276, 276, 276, 276, 276, 276, 276, 276, 682, 681, 423, 280, 276, 276, 276, 276, 276, 276, 423, 678, 280, 289, 289, 289, 289, 289, 340, 340, 340, 340, 340, 427, 423, 676, 675, 276, 276, 276, 276, 276, 276, 278, 278, 278, 278, 278, 278, 278, 278, 278, 278, 656, 492, 427, 427, 278, 278, 278, 278, 278, 278, 341, 341, 341, 341, 341, 655, 289, 492, 331, 331, 331, 331, 331, 289, 492, 630, 629, 278, 278, 278, 278, 278, 278, 355, 355, 355, 355, 355, 289, 293, 293, 293, 293, 293, 293, 293, 293, 293, 293, 331, 621, 447, 497, 293, 293, 293, 293, 293, 293, 620, 577, 349, 349, 349, 349, 349, 356, 356, 356, 356, 356, 331, 331, 447, 447, 497, 293, 293, 293, 293, 293, 293, 296, 296, 296, 296, 296, 296, 296, 296, 296, 296, 576, 517, 394, 521, 296, 296, 296, 296, 296, 296, 357, 357, 357, 357, 357, 363, 363, 363, 363, 363, 394, 349, 349, 517, 517, 394, 521, 296, 296, 296, 296, 296, 296, 297, 297, 297, 297, 297, 297, 297, 297, 297, 297, 349, 847, 573, 847, 297, 297, 297, 297, 297, 297, 364, 364, 364, 364, 364, 572, 314, 314, 314, 314, 314, 365, 365, 365, 365, 365, 393, 297, 297, 297, 297, 297, 297, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 761, 522, 761, 761, 301, 301, 301, 301, 301, 301, 314, 372, 372, 372, 372, 372, 376, 569, 376, 376, 376, 376, 376, 376, 522, 393, 314, 301, 301, 301, 301, 301, 301, 314, 378, 393, 378, 378, 378, 378, 378, 378, 409, 568, 459, 372, 567, 393, 459, 565, 389, 389, 389, 389, 389, 429, 560, 418, 409, 429, 409, 429, 409, 418, 372, 409, 459, 558, 372, 373, 373, 373, 373, 373, 373, 373, 373, 373, 373, 418, 409, 389, 409, 373, 373, 373, 373, 373, 373, 380, 380, 380, 380, 380, 867, 568, 867, 555, 381, 381, 381, 381, 381, 389, 389, 553, 373, 373, 373, 373, 373, 373, 375, 549, 547, 375, 375, 375, 375, 375, 375, 375, 900, 420, 380, 528, 900, 375, 420, 375, 420, 395, 380, 400, 382, 382, 382, 382, 382, 411, 398, 381, 380, 411, 411, 411, 411, 380, 528, 381, 375, 381, 375, 377, 380, 541, 377, 377, 377, 377, 377, 377, 377, 381, 662, 395, 529, 884, 377, 884, 377, 381, 662, 406, 419, 540, 532, 382, 419, 400, 419, 382, 398, 395, 404, 400, 662, 382, 395, 529, 398, 377, 398, 377, 379, 379, 379, 379, 379, 382, 570, 400, 419, 382, 398, 383, 383, 383, 383, 383, 406, 379, 398, 543, 543, 379, 379, 379, 379, 384, 384, 384, 384, 384, 419, 406, 525, 404, 385, 385, 385, 385, 385, 406, 679, 543, 404, 379, 464, 386, 386, 386, 386, 386, 464, 515, 407, 379, 383, 404, 413, 407, 570, 906, 514, 906, 407, 383, 383, 379, 464, 432, 384, 384, 493, 432, 385, 432, 407, 544, 383, 385, 384, 407, 391, 391, 391, 391, 391, 383, 413, 385, 386, 465, 384, 384, 679, 386, 465, 385, 465, 511, 386, 385, 390, 390, 390, 390, 390, 510, 493, 414, 413, 413, 386, 414, 414, 414, 414, 386, 387, 387, 387, 387, 387, 535, 493, 391, 414, 410, 544, 544, 391, 493, 435, 390, 408, 391, 435, 396, 435, 536, 390, 396, 396, 396, 396, 414, 535, 391, 725, 414, 544, 387, 391, 508, 436, 390, 390, 396, 436, 410, 436, 387, 536, 390, 546, 507, 428, 387, 443, 387, 428, 387, 428, 396, 387, 410, 408, 410, 443, 410, 396, 408, 410, 387, 504, 502, 408, 546, 428, 387, 443, 387, 388, 388, 388, 388, 388, 410, 408, 410, 725, 498, 496, 408, 425, 425, 425, 425, 425, 388, 446, 428, 439, 388, 388, 388, 388, 426, 426, 426, 426, 426, 444, 388, 433, 433, 433, 433, 433, 490, 439, 448, 440, 548, 489, 439, 440, 445, 440, 441, 441, 441, 441, 441, 446, 388, 388, 442, 486, 485, 425, 442, 442, 442, 442, 445, 548, 425, 433, 482, 445, 448, 446, 426, 444, 461, 472, 446, 513, 461, 426, 461, 425, 513, 444, 513, 441, 433, 942, 457, 942, 433, 460, 448, 448, 426, 444, 460, 438, 460, 467, 477, 441, 437, 467, 477, 467, 477, 518, 441, 449, 449, 449, 449, 449, 449, 449, 449, 449, 449, 605, 469, 460, 609, 449, 449, 449, 449, 449, 449, 450, 450, 450, 450, 450, 484, 450, 431, 518, 484, 450, 484, 450, 605, 460, 430, 609, 449, 449, 449, 449, 449, 449, 451, 451, 451, 451, 451, 488, 451, 518, 518, 488, 451, 488, 451, 469, 480, 480, 480, 480, 480, 422, 469, 481, 481, 481, 481, 481, 421, 494, 610, 670, 450, 494, 494, 494, 494, 469, 487, 487, 487, 487, 487, 499, 499, 499, 499, 499, 500, 500, 500, 500, 500, 610, 670, 451, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 456, 456, 456, 456, 456, 415, 371, 466, 466, 466, 466, 466, 491, 491, 491, 491, 491, 456, 456, 456, 456, 456, 456, 456, 456, 456, 456, 602, 602, 602, 602, 456, 456, 456, 456, 456, 456, 501, 501, 501, 501, 501, 506, 519, 370, 368, 506, 509, 506, 491, 367, 509, 466, 509, 456, 456, 456, 456, 456, 456, 466, 361, 519, 520, 359, 491, 519, 527, 520, 354, 520, 527, 491, 527, 466, 471, 471, 471, 471, 471, 352, 471, 523, 523, 523, 523, 523, 512, 512, 512, 512, 512, 471, 471, 471, 471, 471, 471, 471, 471, 471, 471, 604, 604, 604, 604, 471, 471, 471, 471, 471, 471, 524, 524, 524, 524, 524, 948, 348, 948, 615, 512, 530, 530, 530, 530, 530, 615, 471, 471, 471, 471, 471, 471, 471, 474, 474, 474, 474, 474, 512, 534, 615, 606, 512, 347, 534, 606, 534, 474, 345, 330, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 952, 329, 952, 606, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 327, 622, 474, 475, 475, 475, 475, 475, 475, 475, 475, 475, 475, 636, 636, 636, 636, 475, 475, 475, 475, 475, 475, 622, 622, 516, 516, 516, 516, 516, 531, 531, 531, 531, 531, 537, 537, 537, 537, 537, 475, 475, 475, 475, 475, 475, 476, 476, 476, 476, 476, 476, 476, 476, 476, 476, 516, 975, 634, 975, 476, 476, 476, 476, 476, 476, 538, 538, 538, 538, 538, 539, 326, 652, 634, 539, 571, 539, 516, 516, 571, 634, 571, 476, 476, 476, 476, 476, 476, 542, 542, 542, 542, 542, 545, 652, 652, 671, 545, 545, 545, 545, 550, 550, 550, 550, 550, 551, 551, 551, 551, 551, 552, 552, 552, 552, 552, 557, 580, 751, 671, 557, 557, 557, 557, 562, 562, 562, 562, 562, 563, 563, 563, 563, 563, 564, 564, 564, 564, 564, 323, 542, 542, 574, 574, 574, 574, 574, 575, 579, 319, 580, 575, 317, 575, 578, 578, 578, 578, 578, 608, 902, 612, 542, 608, 902, 608, 612, 579, 612, 580, 751, 579, 585, 580, 585, 585, 585, 585, 585, 585, 587, 313, 587, 587, 587, 587, 587, 587, 578, 614, 618, 815, 654, 614, 618, 614, 618, 654, 312, 654, 815, 598, 598, 598, 598, 598, 986, 578, 986, 310, 309, 578, 582, 582, 582, 582, 582, 582, 582, 582, 582, 582, 598, 598, 598, 598, 582, 582, 582, 582, 582, 582, 584, 584, 584, 584, 584, 584, 584, 601, 601, 601, 601, 815, 584, 308, 584, 616, 298, 582, 582, 582, 582, 582, 582, 601, 586, 586, 586, 586, 586, 586, 586, 684, 684, 684, 684, 584, 586, 584, 586, 601, 603, 603, 603, 603, 607, 685, 601, 294, 611, 607, 661, 607, 619, 603, 611, 661, 619, 661, 619, 586, 616, 586, 588, 588, 588, 588, 588, 616, 685, 623, 611, 285, 603, 623, 607, 623, 603, 672, 672, 672, 672, 672, 616, 588, 588, 588, 588, 626, 613, 613, 613, 613, 613, 625, 657, 687, 607, 625, 657, 625, 624, 624, 624, 624, 624, 588, 626, 628, 631, 627, 626, 628, 631, 628, 631, 588, 657, 284, 687, 632, 632, 632, 632, 632, 635, 279, 269, 588, 589, 589, 589, 589, 589, 613, 633, 710, 711, 268, 633, 624, 633, 613, 627, 264, 638, 638, 638, 638, 638, 589, 589, 589, 589, 624, 632, 613, 638, 263, 710, 711, 635, 627, 624, 589, 231, 627, 640, 640, 640, 640, 640, 1022, 632, 1022, 812, 653, 635, 632, 640, 812, 653, 589, 653, 635, 230, 866, 589, 590, 590, 590, 590, 590, 644, 644, 644, 644, 644, 658, 644, 227, 226, 658, 644, 658, 644, 715, 225, 653, 590, 590, 590, 590, 223, 645, 645, 645, 645, 645, 674, 645, 221, 716, 674, 645, 674, 645, 812, 219, 715, 653, 590, 646, 646, 646, 646, 646, 663, 866, 590, 1026, 590, 1026, 218, 646, 716, 644, 217, 647, 647, 647, 647, 647, 590, 647, 1027, 215, 1027, 647, 720, 647, 590, 591, 591, 591, 591, 591, 645, 648, 648, 648, 648, 648, 692, 648, 211, 210, 692, 648, 692, 648, 663, 720, 591, 591, 591, 591, 207, 206, 663, 659, 659, 659, 659, 659, 660, 660, 660, 660, 660, 1047, 647, 1047, 663, 673, 673, 673, 673, 673, 591, 677, 677, 677, 677, 677, 591, 689, 689, 689, 689, 689, 648, 700, 683, 683, 683, 683, 683, 707, 721, 202, 591, 592, 592, 592, 592, 592, 659, 731, 733, 696, 700, 660, 659, 696, 700, 696, 707, 660, 701, 197, 707, 721, 592, 592, 592, 592, 196, 192, 659, 683, 731, 733, 191, 660, 690, 690, 690, 690, 690, 691, 691, 691, 691, 691, 190, 683, 695, 695, 695, 695, 695, 701, 683, 592, 592, 699, 699, 699, 699, 699, 706, 706, 706, 706, 706, 712, 712, 712, 712, 712, 701, 1059, 188, 1059, 701, 592, 593, 593, 593, 593, 593, 713, 713, 713, 713, 713, 843, 185, 777, 699, 184, 706, 717, 717, 717, 717, 717, 593, 593, 593, 593, 718, 718, 718, 718, 718, 181, 777, 699, 843, 719, 777, 699, 706, 706, 719, 1060, 719, 1060, 790, 593, 722, 722, 722, 722, 722, 958, 790, 180, 593, 723, 723, 723, 723, 723, 958, 729, 729, 729, 729, 729, 790, 593, 594, 594, 594, 594, 594, 730, 730, 730, 730, 735, 735, 735, 735, 735, 736, 736, 736, 736, 736, 175, 173, 594, 594, 594, 594, 737, 737, 737, 737, 737, 738, 738, 738, 738, 166, 958, 775, 844, 594, 743, 743, 743, 743, 743, 153, 729, 729, 744, 744, 744, 744, 744, 775, 594, 745, 745, 745, 745, 745, 775, 844, 594, 595, 595, 595, 595, 595, 729, 746, 144, 142, 854, 746, 856, 746, 749, 749, 749, 749, 749, 132, 758, 758, 758, 799, 758, 758, 758, 755, 755, 755, 755, 755, 758, 854, 758, 856, 131, 799, 762, 762, 762, 762, 762, 782, 595, 832, 799, 785, 782, 595, 782, 785, 595, 117, 595, 758, 113, 758, 760, 760, 760, 755, 760, 760, 760, 112, 595, 832, 832, 785, 760, 595, 760, 791, 595, 596, 596, 596, 596, 596, 755, 762, 111, 874, 755, 763, 763, 763, 763, 763, 1077, 762, 1077, 760, 109, 760, 772, 772, 772, 772, 772, 786, 874, 762, 805, 786, 874, 786, 596, 108, 104, 764, 764, 764, 764, 764, 789, 791, 596, 102, 763, 789, 805, 789, 596, 791, 596, 805, 596, 98, 97, 596, 765, 765, 765, 765, 765, 781, 763, 791, 596, 793, 781, 763, 781, 793, 596, 793, 596, 597, 597, 597, 597, 597, 764, 93, 827, 766, 766, 766, 766, 766, 764, 91, 764, 827, 1079, 88, 1079, 781, 597, 597, 597, 597, 81, 76, 764, 796, 827, 765, 597, 796, 75, 796, 764, 765, 767, 767, 767, 767, 767, 781, 71, 876, 768, 768, 768, 768, 768, 800, 877, 765, 597, 597, 599, 599, 599, 599, 599, 766, 766, 769, 769, 769, 769, 769, 876, 57, 771, 771, 771, 771, 771, 877, 55, 599, 599, 599, 599, 798, 767, 766, 768, 798, 802, 798, 1075, 800, 802, 767, 802, 806, 599, 44, 804, 1075, 41, 768, 804, 771, 804, 800, 767, 39, 769, 768, 829, 599, 1075, 769, 800, 829, 35, 829, 769, 599, 600, 600, 600, 600, 600, 771, 771, 828, 835, 806, 769, 828, 835, 828, 835, 769, 787, 787, 787, 787, 787, 803, 803, 803, 803, 803, 880, 806, 1090, 1098, 1090, 1098, 806, 773, 773, 773, 773, 773, 809, 809, 809, 809, 809, 600, 881, 31, 24, 833, 600, 880, 809, 600, 833, 600, 833, 774, 774, 774, 774, 774, 861, 21, 836, 787, 861, 600, 861, 881, 836, 787, 600, 773, 885, 600, 637, 637, 637, 637, 637, 637, 637, 637, 637, 637, 836, 787, 773, 886, 637, 637, 637, 637, 637, 637, 773, 885, 20, 933, 774, 892, 894, 939, 933, 774, 933, 939, 15, 939, 774, 946, 886, 637, 637, 637, 637, 637, 637, 641, 641, 641, 774, 1000, 892, 894, 13, 774, 797, 797, 797, 797, 797, 946, 946, 641, 641, 641, 641, 641, 641, 641, 641, 641, 641, 3, 1000, 1001, 1039, 641, 641, 641, 641, 641, 641, 1100, 0, 1100, 0, 797, 845, 845, 845, 845, 845, 846, 846, 846, 846, 846, 1001, 1039, 641, 641, 641, 641, 641, 641, 643, 643, 643, 797, 797, 850, 850, 850, 850, 850, 858, 858, 858, 858, 858, 0, 643, 643, 643, 643, 643, 643, 643, 643, 643, 643, 0, 1040, 0, 0, 643, 643, 643, 643, 643, 643, 817, 817, 817, 817, 817, 943, 817, 0, 937, 943, 817, 943, 817, 937, 1040, 937, 0, 643, 643, 643, 643, 643, 643, 649, 649, 649, 649, 649, 940, 649, 0, 0, 945, 649, 940, 649, 945, 0, 945, 0, 649, 649, 649, 649, 649, 649, 649, 649, 649, 649, 940, 0, 0, 817, 649, 649, 649, 649, 649, 649, 859, 859, 859, 859, 859, 860, 860, 860, 860, 860, 864, 864, 864, 864, 864, 0, 649, 649, 649, 649, 649, 649, 649, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 651, 651, 651, 651, 651, 651, 651, 651, 651, 651, 0, 0, 0, 0, 651, 651, 651, 651, 651, 651, 818, 818, 818, 818, 818, 949, 818, 0, 970, 949, 818, 949, 818, 970, 0, 970, 0, 651, 651, 651, 651, 651, 651, 665, 665, 665, 665, 665, 1045, 665, 878, 878, 878, 878, 878, 879, 879, 879, 879, 879, 665, 665, 665, 665, 665, 665, 665, 665, 665, 665, 1045, 1045, 0, 818, 665, 665, 665, 665, 665, 665, 820, 820, 820, 820, 820, 834, 834, 834, 834, 834, 0, 965, 820, 0, 0, 965, 665, 665, 665, 665, 665, 665, 665, 666, 666, 666, 666, 666, 666, 666, 666, 666, 666, 965, 0, 0, 0, 666, 666, 666, 666, 666, 666, 821, 821, 821, 821, 821, 873, 873, 873, 873, 873, 820, 834, 821, 0, 0, 834, 0, 666, 666, 666, 666, 666, 666, 667, 667, 667, 667, 667, 667, 667, 667, 667, 667, 834, 0, 0, 873, 667, 667, 667, 667, 667, 667, 0, 822, 822, 822, 822, 822, 0, 822, 967, 0, 821, 822, 967, 822, 967, 873, 873, 667, 667, 667, 667, 667, 667, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 0, 0, 0, 0, 668, 668, 668, 668, 668, 668, 0, 0, 788, 788, 788, 788, 788, 837, 0, 0, 0, 822, 882, 882, 882, 882, 882, 668, 668, 668, 668, 668, 668, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 0, 0, 0, 0, 757, 757, 757, 757, 757, 757, 947, 823, 823, 823, 823, 823, 788, 973, 0, 0, 837, 973, 788, 823, 0, 0, 837, 757, 757, 757, 757, 757, 757, 770, 770, 770, 770, 770, 788, 973, 947, 0, 837, 839, 839, 839, 839, 839, 0, 839, 883, 883, 883, 883, 883, 887, 887, 887, 887, 887, 0, 1020, 947, 947, 823, 1020, 770, 888, 888, 888, 888, 888, 896, 896, 896, 896, 896, 897, 897, 897, 897, 897, 770, 1020, 770, 0, 770, 0, 0, 770, 898, 898, 898, 898, 898, 0, 839, 903, 903, 903, 903, 903, 0, 0, 770, 0, 770, 808, 808, 808, 808, 808, 808, 808, 808, 808, 808, 0, 0, 0, 0, 808, 808, 808, 808, 808, 808, 853, 853, 853, 853, 853, 904, 904, 904, 904, 904, 905, 905, 905, 905, 905, 0, 0, 808, 808, 808, 808, 808, 808, 813, 813, 813, 813, 813, 0, 813, 909, 909, 909, 909, 909, 0, 0, 853, 0, 0, 813, 813, 813, 813, 813, 813, 813, 813, 813, 813, 0, 0, 0, 853, 813, 813, 813, 813, 813, 813, 853, 944, 944, 944, 944, 944, 953, 953, 953, 953, 953, 981, 981, 981, 981, 981, 813, 813, 813, 813, 813, 813, 813, 816, 816, 816, 816, 816, 1041, 972, 1017, 0, 1041, 816, 972, 1017, 972, 1017, 0, 0, 816, 816, 816, 816, 816, 816, 816, 816, 816, 816, 1041, 0, 0, 0, 816, 816, 816, 816, 816, 816, 870, 870, 870, 870, 870, 0, 891, 891, 891, 891, 891, 989, 989, 989, 989, 989, 816, 816, 816, 816, 816, 816, 816, 819, 819, 819, 819, 819, 819, 819, 819, 819, 819, 0, 870, 0, 0, 819, 819, 819, 819, 819, 819, 914, 914, 914, 914, 914, 1023, 0, 0, 955, 1023, 870, 1023, 0, 955, 870, 891, 891, 819, 819, 819, 819, 819, 819, 824, 824, 824, 824, 824, 1002, 1002, 1002, 1002, 1002, 0, 0, 0, 0, 891, 0, 0, 824, 824, 824, 824, 824, 824, 824, 824, 824, 824, 0, 0, 0, 914, 824, 824, 824, 824, 824, 824, 955, 1003, 1003, 1003, 1003, 1003, 1013, 1013, 1013, 1013, 1013, 1018, 1018, 1018, 1018, 1018, 0, 824, 824, 824, 824, 824, 824, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 826, 826, 826, 826, 826, 826, 826, 826, 826, 826, 0, 0, 0, 0, 826, 826, 826, 826, 826, 826, 912, 912, 912, 912, 912, 924, 924, 924, 924, 924, 1038, 1088, 0, 0, 1038, 1088, 1038, 826, 826, 826, 826, 826, 826, 840, 840, 840, 840, 840, 840, 840, 840, 840, 840, 1088, 912, 0, 924, 840, 840, 840, 840, 840, 840, 916, 916, 916, 916, 916, 0, 0, 966, 0, 0, 912, 966, 0, 966, 912, 924, 924, 840, 840, 840, 840, 840, 840, 841, 841, 841, 841, 841, 841, 841, 841, 841, 841, 0, 966, 916, 0, 841, 841, 841, 841, 841, 841, 0, 915, 915, 915, 915, 915, 0, 1019, 0, 0, 916, 0, 1019, 966, 1019, 916, 0, 841, 841, 841, 841, 841, 841, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 0, 0, 0, 0, 842, 842, 842, 842, 842, 842, 0, 915, 917, 917, 917, 917, 917, 0, 0, 974, 941, 915, 918, 918, 918, 918, 918, 842, 842, 842, 842, 842, 842, 915, 919, 919, 919, 919, 919, 920, 920, 920, 920, 920, 0, 0, 921, 921, 921, 921, 921, 1046, 0, 0, 0, 917, 0, 0, 922, 922, 922, 922, 922, 917, 1042, 917, 941, 974, 1042, 918, 1042, 974, 941, 1044, 0, 918, 0, 917, 1044, 0, 1044, 1046, 0, 920, 921, 917, 919, 919, 941, 974, 0, 918, 920, 925, 925, 925, 925, 925, 0, 921, 0, 922, 0, 1046, 1046, 920, 922, 921, 919, 0, 0, 922, 926, 926, 926, 926, 926, 0, 927, 927, 927, 927, 927, 922, 1076, 0, 0, 0, 922, 923, 923, 923, 923, 923, 0, 0, 0, 928, 928, 928, 928, 928, 1056, 0, 0, 0, 925, 1056, 0, 1056, 926, 929, 929, 929, 929, 929, 931, 931, 931, 931, 931, 927, 923, 1067, 0, 926, 927, 1076, 1067, 0, 1067, 927, 0, 926, 928, 1071, 1076, 0, 923, 1071, 923, 1071, 923, 927, 0, 923, 0, 0, 927, 1076, 928, 930, 930, 930, 930, 930, 0, 928, 0, 1074, 923, 1085, 923, 1074, 929, 1074, 1085, 0, 1085, 931, 938, 938, 938, 938, 938, 961, 961, 961, 961, 961, 963, 963, 963, 963, 963, 930, 0, 961, 0, 0, 0, 0, 963, 1051, 1051, 1051, 1051, 1051, 0, 971, 971, 971, 971, 971, 930, 0, 0, 0, 930, 976, 976, 976, 976, 976, 977, 977, 977, 977, 977, 938, 1087, 0, 1037, 938, 1087, 1021, 1087, 1037, 961, 1037, 971, 0, 0, 963, 1086, 1086, 1086, 1086, 1086, 0, 0, 938, 956, 956, 956, 956, 956, 956, 956, 956, 956, 956, 971, 971, 1037, 0, 956, 956, 956, 956, 956, 956, 976, 0, 0, 0, 0, 977, 978, 978, 978, 978, 978, 1021, 0, 0, 1037, 1021, 0, 956, 956, 956, 956, 956, 956, 959, 959, 959, 959, 959, 959, 959, 959, 959, 959, 1021, 0, 0, 0, 959, 959, 959, 959, 959, 959, 983, 983, 983, 983, 983, 984, 984, 984, 984, 984, 1094, 1094, 1094, 1094, 1094, 0, 978, 959, 959, 959, 959, 959, 959, 960, 960, 960, 960, 960, 960, 960, 960, 960, 960, 0, 0, 0, 0, 960, 960, 960, 960, 960, 960, 1095, 0, 0, 0, 0, 1095, 0, 1095, 0, 0, 983, 0, 0, 0, 0, 984, 0, 960, 960, 960, 960, 960, 960, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 964, 964, 964, 964, 964, 964, 964, 964, 964, 964, 0, 0, 0, 0, 964, 964, 964, 964, 964, 964, 982, 982, 982, 982, 982, 985, 985, 985, 985, 985, 992, 992, 992, 992, 992, 0, 0, 964, 964, 964, 964, 964, 964, 993, 993, 993, 993, 993, 994, 994, 994, 994, 994, 0, 0, 0, 0, 982, 995, 995, 995, 995, 995, 0, 992, 996, 996, 996, 996, 996, 0, 0, 0, 982, 993, 0, 0, 0, 985, 0, 982, 0, 994, 992, 0, 0, 0, 992, 997, 997, 997, 997, 997, 0, 0, 0, 993, 993, 0, 0, 0, 994, 0, 0, 0, 994, 998, 998, 998, 998, 998, 995, 999, 999, 999, 999, 999, 0, 996, 1004, 1004, 1004, 1004, 1004, 1005, 1005, 1005, 1005, 1005, 1006, 1006, 1006, 1006, 1006, 1007, 1007, 1007, 1007, 1007, 0, 1089, 997, 1008, 1008, 1008, 1008, 1008, 1009, 1009, 1009, 1009, 1009, 1010, 1010, 1010, 1010, 1010, 0, 0, 998, 1014, 1014, 1014, 1014, 1014, 999, 1015, 1015, 1015, 1015, 1015, 1004, 1004, 0, 0, 0, 0, 1005, 0, 0, 0, 0, 1006, 0, 0, 0, 0, 1007, 1089, 0, 0, 0, 1089, 1004, 1014, 1008, 0, 0, 0, 0, 1009, 0, 0, 0, 0, 1010, 0, 0, 0, 0, 1089, 0, 0, 1014, 0, 0, 0, 1014, 0, 1015, 1028, 1028, 1028, 1028, 1028, 1028, 1028, 1028, 1028, 1028, 0, 0, 0, 0, 1028, 1028, 1028, 1028, 1028, 1028, 1031, 1031, 1031, 1031, 1031, 1048, 1048, 1048, 1048, 1048, 0, 0, 1031, 0, 0, 0, 0, 1028, 1028, 1028, 1028, 1028, 1028, 1029, 1029, 1029, 1029, 1029, 1029, 1029, 1029, 1029, 1029, 0, 0, 0, 0, 1029, 1029, 1029, 1029, 1029, 1029, 1032, 1032, 1032, 1032, 1032, 1043, 1043, 1043, 1043, 1043, 1031, 0, 1032, 0, 0, 1048, 0, 1029, 1029, 1029, 1029, 1029, 1029, 1030, 1030, 1030, 1030, 1030, 1030, 1030, 1030, 1030, 1030, 0, 0, 0, 0, 1030, 1030, 1030, 1030, 1030, 1030, 1053, 1053, 1053, 1053, 1053, 0, 1043, 0, 0, 0, 1032, 0, 0, 0, 0, 1043, 0, 1030, 1030, 1030, 1030, 1030, 1030, 1033, 1033, 1033, 1033, 1033, 1043, 0, 0, 0, 0, 0, 0, 1033, 0, 0, 0, 0, 1033, 1033, 1033, 1033, 1033, 1033, 1033, 1033, 1033, 1033, 0, 0, 0, 1053, 1033, 1033, 1033, 1033, 1033, 1033, 1052, 1052, 1052, 1052, 1052, 1054, 1054, 1054, 1054, 1054, 0, 0, 0, 0, 0, 0, 1033, 1033, 1033, 1033, 1033, 1033, 1033, 1035, 1035, 1035, 1035, 1035, 1035, 1035, 1035, 1035, 1035, 0, 1052, 0, 0, 1035, 1035, 1035, 1035, 1035, 1035, 1055, 1055, 1055, 1055, 1055, 0, 0, 0, 0, 0, 1052, 0, 0, 0, 1052, 1054, 0, 1035, 1035, 1035, 1035, 1035, 1035, 1064, 1064, 1064, 1064, 1064, 0, 0, 0, 0, 0, 0, 0, 1064, 1072, 1072, 1072, 1072, 1072, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1055, 1061, 1061, 1061, 1061, 1061, 1061, 1061, 1061, 1061, 1061, 0, 0, 0, 0, 1061, 1061, 1061, 1061, 1061, 1061, 0, 0, 1064, 0, 0, 1073, 1073, 1073, 1073, 1073, 0, 1072, 0, 0, 0, 1072, 0, 1061, 1061, 1061, 1061, 1061, 1061, 1062, 1062, 1062, 1062, 1062, 1062, 1062, 1062, 1062, 1062, 1072, 0, 0, 0, 1062, 1062, 1062, 1062, 1062, 1062, 1078, 1078, 1078, 1078, 1078, 0, 0, 0, 0, 0, 0, 1073, 0, 0, 0, 1073, 0, 1062, 1062, 1062, 1062, 1062, 1062, 1063, 1063, 1063, 1063, 1063, 1063, 1063, 1063, 1063, 1063, 1073, 0, 0, 0, 1063, 1063, 1063, 1063, 1063, 1063, 1101, 1101, 1101, 1101, 1101, 0, 0, 0, 0, 0, 1078, 0, 1101, 0, 0, 0, 0, 1063, 1063, 1063, 1063, 1063, 1063, 1065, 1065, 1065, 1065, 1065, 1065, 1065, 1065, 1065, 1065, 0, 0, 0, 0, 1065, 1065, 1065, 1065, 1065, 1065, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1101, 0, 0, 0, 0, 0, 0, 1065, 1065, 1065, 1065, 1065, 1065, 1066, 1066, 1066, 1066, 1066, 1066, 1066, 1066, 1066, 1066, 0, 0, 0, 0, 1066, 1066, 1066, 1066, 1066, 1066, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1066, 1066, 1066, 1066, 1066, 1066, 1080, 1080, 1080, 1080, 1080, 1080, 1080, 1080, 1080, 1080, 0, 0, 0, 0, 1080, 1080, 1080, 1080, 1080, 1080, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1080, 1080, 1080, 1080, 1080, 1080, 1081, 1081, 1081, 1081, 1081, 1081, 1081, 1081, 1081, 1081, 0, 0, 0, 0, 1081, 1081, 1081, 1081, 1081, 1081, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1081, 1081, 1081, 1081, 1081, 1081, 1082, 1082, 1082, 1082, 1082, 1082, 1082, 1082, 1082, 1082, 0, 0, 0, 0, 1082, 1082, 1082, 1082, 1082, 1082, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1082, 1082, 1082, 1082, 1082, 1082, 1083, 1083, 1083, 1083, 1083, 1083, 1083, 1083, 1083, 1083, 0, 0, 0, 0, 1083, 1083, 1083, 1083, 1083, 1083, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1083, 1083, 1083, 1083, 1083, 1083, 1084, 1084, 1084, 1084, 1084, 1084, 1084, 1084, 1084, 1084, 0, 0, 0, 0, 1084, 1084, 1084, 1084, 1084, 1084, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1084, 1084, 1084, 1084, 1084, 1084, 1091, 1091, 1091, 1091, 1091, 1091, 1091, 1091, 1091, 1091, 0, 0, 0, 0, 1091, 1091, 1091, 1091, 1091, 1091, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1091, 1091, 1091, 1091, 1091, 1091, 1092, 1092, 1092, 1092, 1092, 1092, 1092, 1092, 1092, 1092, 0, 0, 0, 0, 1092, 1092, 1092, 1092, 1092, 1092, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1092, 1092, 1092, 1092, 1092, 1092, 1093, 1093, 1093, 1093, 1093, 1093, 1093, 1093, 1093, 1093, 0, 0, 0, 0, 1093, 1093, 1093, 1093, 1093, 1093, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1093, 1093, 1093, 1093, 1093, 1093, 1099, 1099, 1099, 1099, 1099, 1099, 1099, 1099, 1099, 1099, 0, 0, 0, 0, 1099, 1099, 1099, 1099, 1099, 1099, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1099, 1099, 1099, 1099, 1099, 1099, 1103, 1103, 0, 1103, 1103, 1103, 1103, 1103, 1103, 1104, 1104, 1104, 1104, 1105, 1105, 0, 1105, 1105, 1105, 1105, 1105, 1105, 1106, 0, 0, 1106, 1107, 1107, 0, 1107, 1107, 1108, 1108, 0, 1108, 1108, 1108, 1108, 1108, 1108, 1109, 1109, 1109, 1109, 1109, 1109, 1109, 1109, 1109, 1110, 1110, 0, 1110, 1110, 1110, 1110, 1110, 1110, 1111, 1111, 1111, 1111, 1111, 1111, 1111, 1111, 1111, 1112, 1112, 1112, 1112, 1112, 1112, 1112, 1112, 1112, 1114, 0, 1114, 1114, 1115, 1115, 0, 1115, 1115, 1115, 1115, 1115, 1115, 1116, 1116, 1116, 1116, 1116, 1116, 1116, 1116, 1116, 1117, 1117, 1117, 1117, 1117, 1117, 1117, 1117, 1117, 1118, 1118, 1118, 1118, 1118, 1118, 1118, 1118, 1118, 1119, 1119, 0, 1119, 1119, 1119, 1119, 1119, 1119, 1120, 1120, 1120, 1120, 1120, 1120, 1120, 1120, 1120, 1121, 1121, 1121, 1121, 1121, 1121, 1121, 1121, 1121, 1122, 1122, 1122, 1122, 1122, 1122, 1122, 1122, 1122, 1123, 1123, 1123, 0, 1123, 1123, 1123, 1123, 1124, 1124, 1124, 1124, 1124, 1124, 1124, 1124, 1124, 1125, 1125, 1125, 1125, 1125, 1125, 1125, 1125, 1125, 1126, 1126, 1126, 1126, 1126, 1126, 1126, 1126, 1126, 1127, 1127, 0, 1127, 1127, 1127, 1127, 1127, 1127, 1128, 1128, 0, 1128, 1128, 1128, 1128, 1128, 1128, 1129, 1129, 1129, 1129, 1129, 1129, 1129, 1129, 1129, 1130, 1130, 1130, 1130, 1130, 1130, 1130, 1130, 1130, 1131, 1131, 1131, 1131, 1131, 1131, 1131, 1131, 1131, 1132, 1132, 1132, 1132, 1132, 1132, 1132, 1132, 1132, 1133, 1133, 1133, 1133, 1133, 1133, 1133, 1133, 1133, 1134, 1134, 1134, 1134, 1134, 1134, 1134, 1134, 1134, 1135, 1135, 1135, 1135, 1135, 1135, 1135, 1135, 1135, 1136, 1136, 0, 0, 1136, 1136, 1136, 1136, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102 } ; static yy_state_type yy_last_accepting_state; static char *yy_last_accepting_cpos; extern int yy_flex_debug; int yy_flex_debug = 0; /* The intent behind this definition is that it'll catch * any uses of REJECT which flex missed. */ #define REJECT reject_used_but_not_detected #define yymore() yymore_used_but_not_detected #define YY_MORE_ADJ 0 #define YY_RESTORE_YY_MORE_OFFSET char *yytext; #line 1 "css.l" #line 13 "css.l" /* Lex source for CSS tokenizing. Taken from http://www.w3.org/TR/CSS21/grammar.html#q2 Copyright (C) 2006, 2009-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #define YY_NO_INPUT #include "css-tokens.h" #if defined __clang__ || __GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 5) #pragma GCC diagnostic ignored "-Wunknown-pragmas" // clang mourns about the next one #pragma GCC diagnostic ignored "-Wunused-function" #pragma GCC diagnostic ignored "-Wunused-macros" #pragma GCC diagnostic ignored "-Wunused-parameter" #pragma GCC diagnostic ignored "-Wsign-compare" #pragma GCC diagnostic ignored "-Wswitch-default" #pragma GCC diagnostic ignored "-Wunreachable-code" // clang #pragma clang diagnostic ignored "-Wshorten-64-to-32" #ifndef __clang__ #pragma GCC diagnostic ignored "-Wsuggest-attribute=pure" #endif #endif #line 2452 "css.c" #line 2453 "css.c" #define INITIAL 0 #ifndef YY_NO_UNISTD_H /* Special case for "unistd.h", since it is non-ANSI. We include it way * down here because we want the user's section 1 to have been scanned first. * The user has a chance to override it with an option. */ #include #endif #ifndef YY_EXTRA_TYPE #define YY_EXTRA_TYPE void * #endif static int yy_init_globals ( void ); /* Accessor methods to globals. These are made visible to non-reentrant scanners for convenience. */ int yylex_destroy ( void ); int yyget_debug ( void ); void yyset_debug ( int debug_flag ); YY_EXTRA_TYPE yyget_extra ( void ); void yyset_extra ( YY_EXTRA_TYPE user_defined ); FILE *yyget_in ( void ); void yyset_in ( FILE * _in_str ); FILE *yyget_out ( void ); void yyset_out ( FILE * _out_str ); int yyget_leng ( void ); char *yyget_text ( void ); int yyget_lineno ( void ); void yyset_lineno ( int _line_number ); /* Macros after this point can all be overridden by user definitions in * section 1. */ #ifndef YY_SKIP_YYWRAP #ifdef __cplusplus extern "C" int yywrap ( void ); #else extern int yywrap ( void ); #endif #endif #ifndef YY_NO_UNPUT #endif #ifndef yytext_ptr static void yy_flex_strncpy ( char *, const char *, int ); #endif #ifdef YY_NEED_STRLEN static int yy_flex_strlen ( const char * ); #endif #ifndef YY_NO_INPUT #ifdef __cplusplus static int yyinput ( void ); #else static int input ( void ); #endif #endif /* Amount of stuff to slurp up with each read. */ #ifndef YY_READ_BUF_SIZE #ifdef __ia64__ /* On IA-64, the buffer size is 16k, not 8k */ #define YY_READ_BUF_SIZE 16384 #else #define YY_READ_BUF_SIZE 8192 #endif /* __ia64__ */ #endif /* Copy whatever the last rule matched to the standard output. */ #ifndef ECHO /* This used to be an fputs(), but since the string might contain NUL's, * we now use fwrite(). */ #define ECHO do { if (fwrite( yytext, (size_t) yyleng, 1, yyout )) {} } while (0) #endif /* Gets input and stuffs it into "buf". number of characters read, or YY_NULL, * is returned in "result". */ #ifndef YY_INPUT #define YY_INPUT(buf,result,max_size) \ if ( YY_CURRENT_BUFFER_LVALUE->yy_is_interactive ) \ { \ int c = '*'; \ int n; \ for ( n = 0; n < max_size && \ (c = getc( yyin )) != EOF && c != '\n'; ++n ) \ buf[n] = (char) c; \ if ( c == '\n' ) \ buf[n++] = (char) c; \ if ( c == EOF && ferror( yyin ) ) \ YY_FATAL_ERROR( "input in flex scanner failed" ); \ result = n; \ } \ else \ { \ errno=0; \ while ( (result = (int) fread(buf, 1, (yy_size_t) max_size, yyin)) == 0 && ferror(yyin)) \ { \ if( errno != EINTR) \ { \ YY_FATAL_ERROR( "input in flex scanner failed" ); \ break; \ } \ errno=0; \ clearerr(yyin); \ } \ }\ \ #endif /* No semi-colon after return; correct usage is to write "yyterminate();" - * we don't want an extra ';' after the "return" because that will cause * some compilers to complain about unreachable statements. */ #ifndef yyterminate #define yyterminate() return YY_NULL #endif /* Number of entries by which start-condition stack grows. */ #ifndef YY_START_STACK_INCR #define YY_START_STACK_INCR 25 #endif /* Report a fatal error. */ #ifndef YY_FATAL_ERROR #define YY_FATAL_ERROR(msg) yy_fatal_error( msg ) #endif /* end tables serialization structures and prototypes */ /* Default declaration of generated scanner - a define so the user can * easily add parameters. */ #ifndef YY_DECL #define YY_DECL_IS_OURS 1 extern int yylex (void); #define YY_DECL int yylex (void) #endif /* !YY_DECL */ /* Code executed at the beginning of each rule, after yytext and yyleng * have been set up. */ #ifndef YY_USER_ACTION #define YY_USER_ACTION #endif /* Code executed at the end of each rule. */ #ifndef YY_BREAK #define YY_BREAK /*LINTED*/break; #endif #define YY_RULE_SETUP \ YY_USER_ACTION /** The main scanner function which does all the work. */ YY_DECL { yy_state_type yy_current_state; char *yy_cp, *yy_bp; int yy_act; if ( !(yy_init) ) { (yy_init) = 1; #ifdef YY_USER_INIT YY_USER_INIT; #endif if ( ! (yy_start) ) (yy_start) = 1; /* first start state */ if ( ! yyin ) yyin = stdin; if ( ! yyout ) yyout = stdout; if ( ! YY_CURRENT_BUFFER ) { yyensure_buffer_stack (); YY_CURRENT_BUFFER_LVALUE = yy_create_buffer( yyin, YY_BUF_SIZE ); } yy_load_buffer_state( ); } { #line 112 "css.l" #line 2671 "css.c" while ( /*CONSTCOND*/1 ) /* loops until end-of-file is reached */ { yy_cp = (yy_c_buf_p); /* Support of yytext. */ *yy_cp = (yy_hold_char); /* yy_bp points to the position in yy_ch_buf of the start of * the current run. */ yy_bp = yy_cp; yy_current_state = (yy_start); yy_match: do { YY_CHAR yy_c = yy_ec[YY_SC_TO_UI(*yy_cp)] ; if ( yy_accept[yy_current_state] ) { (yy_last_accepting_state) = yy_current_state; (yy_last_accepting_cpos) = yy_cp; } while ( yy_chk[yy_base[yy_current_state] + yy_c] != yy_current_state ) { yy_current_state = (int) yy_def[yy_current_state]; if ( yy_current_state >= 1103 ) yy_c = yy_meta[yy_c]; } yy_current_state = yy_nxt[yy_base[yy_current_state] + yy_c]; ++yy_cp; } while ( yy_current_state != 1102 ); yy_cp = (yy_last_accepting_cpos); yy_current_state = (yy_last_accepting_state); yy_find_action: yy_act = yy_accept[yy_current_state]; YY_DO_BEFORE_ACTION; do_action: /* This label is used only to access EOF actions. */ switch ( yy_act ) { /* beginning of action switch */ case 0: /* must back up */ /* undo the effects of YY_DO_BEFORE_ACTION */ *yy_cp = (yy_hold_char); yy_cp = (yy_last_accepting_cpos); yy_current_state = (yy_last_accepting_state); goto yy_find_action; case 1: /* rule 1 can match eol */ YY_RULE_SETUP #line 114 "css.l" {return S;} YY_BREAK case 2: /* rule 2 can match eol */ YY_RULE_SETUP #line 116 "css.l" {return COMMENT;} YY_BREAK case 3: /* rule 3 can match eol */ YY_RULE_SETUP #line 117 "css.l" /* ignore comments */ YY_BREAK case 4: /* rule 4 can match eol */ YY_RULE_SETUP #line 118 "css.l" /* unclosed comment at EOF */ YY_BREAK case 5: YY_RULE_SETUP #line 120 "css.l" {return CDO;} YY_BREAK case 6: YY_RULE_SETUP #line 121 "css.l" {return CDC;} YY_BREAK case 7: YY_RULE_SETUP #line 122 "css.l" {return INCLUDES;} YY_BREAK case 8: YY_RULE_SETUP #line 123 "css.l" {return DASHMATCH;} YY_BREAK case 9: /* rule 9 can match eol */ YY_RULE_SETUP #line 125 "css.l" {return STRING;} YY_BREAK case 10: /* rule 10 can match eol */ YY_RULE_SETUP #line 126 "css.l" {return BAD_STRING;} YY_BREAK case 11: /* rule 11 can match eol */ YY_RULE_SETUP #line 128 "css.l" {return IDENT;} YY_BREAK case 12: /* rule 12 can match eol */ YY_RULE_SETUP #line 130 "css.l" {return HASH;} YY_BREAK case 13: /* rule 13 can match eol */ YY_RULE_SETUP #line 132 "css.l" {return IMPORT_SYM;} YY_BREAK case 14: /* rule 14 can match eol */ YY_RULE_SETUP #line 133 "css.l" {return PAGE_SYM;} YY_BREAK case 15: /* rule 15 can match eol */ YY_RULE_SETUP #line 134 "css.l" {return MEDIA_SYM;} YY_BREAK case 16: YY_RULE_SETUP #line 135 "css.l" {return CHARSET_SYM;} YY_BREAK case 17: /* rule 17 can match eol */ YY_RULE_SETUP #line 137 "css.l" {return IMPORTANT_SYM;} YY_BREAK case 18: /* rule 18 can match eol */ YY_RULE_SETUP #line 139 "css.l" {return EMS;} YY_BREAK case 19: /* rule 19 can match eol */ YY_RULE_SETUP #line 140 "css.l" {return EXS;} YY_BREAK case 20: /* rule 20 can match eol */ YY_RULE_SETUP #line 141 "css.l" {return LENGTH;} YY_BREAK case 21: /* rule 21 can match eol */ YY_RULE_SETUP #line 142 "css.l" {return LENGTH;} YY_BREAK case 22: /* rule 22 can match eol */ YY_RULE_SETUP #line 143 "css.l" {return LENGTH;} YY_BREAK case 23: /* rule 23 can match eol */ YY_RULE_SETUP #line 144 "css.l" {return LENGTH;} YY_BREAK case 24: /* rule 24 can match eol */ YY_RULE_SETUP #line 145 "css.l" {return LENGTH;} YY_BREAK case 25: /* rule 25 can match eol */ YY_RULE_SETUP #line 146 "css.l" {return LENGTH;} YY_BREAK case 26: /* rule 26 can match eol */ YY_RULE_SETUP #line 147 "css.l" {return ANGLE;} YY_BREAK case 27: /* rule 27 can match eol */ YY_RULE_SETUP #line 148 "css.l" {return ANGLE;} YY_BREAK case 28: /* rule 28 can match eol */ YY_RULE_SETUP #line 149 "css.l" {return ANGLE;} YY_BREAK case 29: /* rule 29 can match eol */ YY_RULE_SETUP #line 150 "css.l" {return TIME;} YY_BREAK case 30: /* rule 30 can match eol */ YY_RULE_SETUP #line 151 "css.l" {return TIME;} YY_BREAK case 31: /* rule 31 can match eol */ YY_RULE_SETUP #line 152 "css.l" {return FREQ;} YY_BREAK case 32: /* rule 32 can match eol */ YY_RULE_SETUP #line 153 "css.l" {return FREQ;} YY_BREAK case 33: /* rule 33 can match eol */ YY_RULE_SETUP #line 154 "css.l" {return DIMENSION;} YY_BREAK case 34: YY_RULE_SETUP #line 156 "css.l" {return PERCENTAGE;} YY_BREAK case 35: YY_RULE_SETUP #line 157 "css.l" {return NUMBER;} YY_BREAK case 36: /* rule 36 can match eol */ YY_RULE_SETUP #line 159 "css.l" {return URI;} YY_BREAK case 37: /* rule 37 can match eol */ YY_RULE_SETUP #line 160 "css.l" {return URI;} YY_BREAK case 38: /* rule 38 can match eol */ YY_RULE_SETUP #line 161 "css.l" {return BAD_URI;} YY_BREAK case 39: /* rule 39 can match eol */ YY_RULE_SETUP #line 163 "css.l" {return FUNCTION;} YY_BREAK case 40: YY_RULE_SETUP #line 165 "css.l" {return *yytext;} YY_BREAK case 41: YY_RULE_SETUP #line 167 "css.l" ECHO; YY_BREAK #line 2961 "css.c" case YY_STATE_EOF(INITIAL): yyterminate(); case YY_END_OF_BUFFER: { /* Amount of text matched not including the EOB char. */ int yy_amount_of_matched_text = (int) (yy_cp - (yytext_ptr)) - 1; /* Undo the effects of YY_DO_BEFORE_ACTION. */ *yy_cp = (yy_hold_char); YY_RESTORE_YY_MORE_OFFSET if ( YY_CURRENT_BUFFER_LVALUE->yy_buffer_status == YY_BUFFER_NEW ) { /* We're scanning a new file or input source. It's * possible that this happened because the user * just pointed yyin at a new source and called * yylex(). If so, then we have to assure * consistency between YY_CURRENT_BUFFER and our * globals. Here is the right place to do so, because * this is the first action (other than possibly a * back-up) that will match for the new input source. */ (yy_n_chars) = YY_CURRENT_BUFFER_LVALUE->yy_n_chars; YY_CURRENT_BUFFER_LVALUE->yy_input_file = yyin; YY_CURRENT_BUFFER_LVALUE->yy_buffer_status = YY_BUFFER_NORMAL; } /* Note that here we test for yy_c_buf_p "<=" to the position * of the first EOB in the buffer, since yy_c_buf_p will * already have been incremented past the NUL character * (since all states make transitions on EOB to the * end-of-buffer state). Contrast this with the test * in input(). */ if ( (yy_c_buf_p) <= &YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[(yy_n_chars)] ) { /* This was really a NUL. */ yy_state_type yy_next_state; (yy_c_buf_p) = (yytext_ptr) + yy_amount_of_matched_text; yy_current_state = yy_get_previous_state( ); /* Okay, we're now positioned to make the NUL * transition. We couldn't have * yy_get_previous_state() go ahead and do it * for us because it doesn't know how to deal * with the possibility of jamming (and we don't * want to build jamming into it because then it * will run more slowly). */ yy_next_state = yy_try_NUL_trans( yy_current_state ); yy_bp = (yytext_ptr) + YY_MORE_ADJ; if ( yy_next_state ) { /* Consume the NUL. */ yy_cp = ++(yy_c_buf_p); yy_current_state = yy_next_state; goto yy_match; } else { yy_cp = (yy_last_accepting_cpos); yy_current_state = (yy_last_accepting_state); goto yy_find_action; } } else switch ( yy_get_next_buffer( ) ) { case EOB_ACT_END_OF_FILE: { (yy_did_buffer_switch_on_eof) = 0; if ( yywrap( ) ) { /* Note: because we've taken care in * yy_get_next_buffer() to have set up * yytext, we can now set up * yy_c_buf_p so that if some total * hoser (like flex itself) wants to * call the scanner after we return the * YY_NULL, it'll still work - another * YY_NULL will get returned. */ (yy_c_buf_p) = (yytext_ptr) + YY_MORE_ADJ; yy_act = YY_STATE_EOF(YY_START); goto do_action; } else { if ( ! (yy_did_buffer_switch_on_eof) ) YY_NEW_FILE; } break; } case EOB_ACT_CONTINUE_SCAN: (yy_c_buf_p) = (yytext_ptr) + yy_amount_of_matched_text; yy_current_state = yy_get_previous_state( ); yy_cp = (yy_c_buf_p); yy_bp = (yytext_ptr) + YY_MORE_ADJ; goto yy_match; case EOB_ACT_LAST_MATCH: (yy_c_buf_p) = &YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[(yy_n_chars)]; yy_current_state = yy_get_previous_state( ); yy_cp = (yy_c_buf_p); yy_bp = (yytext_ptr) + YY_MORE_ADJ; goto yy_find_action; } break; } default: YY_FATAL_ERROR( "fatal flex scanner internal error--no action found" ); } /* end of action switch */ } /* end of scanning one token */ } /* end of user's declarations */ } /* end of yylex */ /* yy_get_next_buffer - try to read in a new buffer * * Returns a code representing an action: * EOB_ACT_LAST_MATCH - * EOB_ACT_CONTINUE_SCAN - continue scanning from current position * EOB_ACT_END_OF_FILE - end of file */ static int yy_get_next_buffer (void) { char *dest = YY_CURRENT_BUFFER_LVALUE->yy_ch_buf; char *source = (yytext_ptr); int number_to_move, i; int ret_val; if ( (yy_c_buf_p) > &YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[(yy_n_chars) + 1] ) YY_FATAL_ERROR( "fatal flex scanner internal error--end of buffer missed" ); if ( YY_CURRENT_BUFFER_LVALUE->yy_fill_buffer == 0 ) { /* Don't try to fill the buffer, so this is an EOF. */ if ( (yy_c_buf_p) - (yytext_ptr) - YY_MORE_ADJ == 1 ) { /* We matched a single character, the EOB, so * treat this as a final EOF. */ return EOB_ACT_END_OF_FILE; } else { /* We matched some text prior to the EOB, first * process it. */ return EOB_ACT_LAST_MATCH; } } /* Try to read more data. */ /* First move last chars to start of buffer. */ number_to_move = (int) ((yy_c_buf_p) - (yytext_ptr) - 1); for ( i = 0; i < number_to_move; ++i ) *(dest++) = *(source++); if ( YY_CURRENT_BUFFER_LVALUE->yy_buffer_status == YY_BUFFER_EOF_PENDING ) /* don't do the read, it's not guaranteed to return an EOF, * just force an EOF */ YY_CURRENT_BUFFER_LVALUE->yy_n_chars = (yy_n_chars) = 0; else { int num_to_read = YY_CURRENT_BUFFER_LVALUE->yy_buf_size - number_to_move - 1; while ( num_to_read <= 0 ) { /* Not enough room in the buffer - grow it. */ /* just a shorter name for the current buffer */ YY_BUFFER_STATE b = YY_CURRENT_BUFFER_LVALUE; int yy_c_buf_p_offset = (int) ((yy_c_buf_p) - b->yy_ch_buf); if ( b->yy_is_our_buffer ) { int new_size = b->yy_buf_size * 2; if ( new_size <= 0 ) b->yy_buf_size += b->yy_buf_size / 8; else b->yy_buf_size *= 2; b->yy_ch_buf = (char *) /* Include room in for 2 EOB chars. */ yyrealloc( (void *) b->yy_ch_buf, (yy_size_t) (b->yy_buf_size + 2) ); } else /* Can't grow it, we don't own it. */ b->yy_ch_buf = NULL; if ( ! b->yy_ch_buf ) YY_FATAL_ERROR( "fatal error - scanner input buffer overflow" ); (yy_c_buf_p) = &b->yy_ch_buf[yy_c_buf_p_offset]; num_to_read = YY_CURRENT_BUFFER_LVALUE->yy_buf_size - number_to_move - 1; } if ( num_to_read > YY_READ_BUF_SIZE ) num_to_read = YY_READ_BUF_SIZE; /* Read in more data. */ YY_INPUT( (&YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[number_to_move]), (yy_n_chars), num_to_read ); YY_CURRENT_BUFFER_LVALUE->yy_n_chars = (yy_n_chars); } if ( (yy_n_chars) == 0 ) { if ( number_to_move == YY_MORE_ADJ ) { ret_val = EOB_ACT_END_OF_FILE; yyrestart( yyin ); } else { ret_val = EOB_ACT_LAST_MATCH; YY_CURRENT_BUFFER_LVALUE->yy_buffer_status = YY_BUFFER_EOF_PENDING; } } else ret_val = EOB_ACT_CONTINUE_SCAN; if (((yy_n_chars) + number_to_move) > YY_CURRENT_BUFFER_LVALUE->yy_buf_size) { /* Extend the array by 50%, plus the number we really need. */ int new_size = (yy_n_chars) + number_to_move + ((yy_n_chars) >> 1); YY_CURRENT_BUFFER_LVALUE->yy_ch_buf = (char *) yyrealloc( (void *) YY_CURRENT_BUFFER_LVALUE->yy_ch_buf, (yy_size_t) new_size ); if ( ! YY_CURRENT_BUFFER_LVALUE->yy_ch_buf ) YY_FATAL_ERROR( "out of dynamic memory in yy_get_next_buffer()" ); /* "- 2" to take care of EOB's */ YY_CURRENT_BUFFER_LVALUE->yy_buf_size = (int) (new_size - 2); } (yy_n_chars) += number_to_move; YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[(yy_n_chars)] = YY_END_OF_BUFFER_CHAR; YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[(yy_n_chars) + 1] = YY_END_OF_BUFFER_CHAR; (yytext_ptr) = &YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[0]; return ret_val; } /* yy_get_previous_state - get the state just before the EOB char was reached */ static yy_state_type yy_get_previous_state (void) { yy_state_type yy_current_state; char *yy_cp; yy_current_state = (yy_start); for ( yy_cp = (yytext_ptr) + YY_MORE_ADJ; yy_cp < (yy_c_buf_p); ++yy_cp ) { YY_CHAR yy_c = (*yy_cp ? yy_ec[YY_SC_TO_UI(*yy_cp)] : 1); if ( yy_accept[yy_current_state] ) { (yy_last_accepting_state) = yy_current_state; (yy_last_accepting_cpos) = yy_cp; } while ( yy_chk[yy_base[yy_current_state] + yy_c] != yy_current_state ) { yy_current_state = (int) yy_def[yy_current_state]; if ( yy_current_state >= 1103 ) yy_c = yy_meta[yy_c]; } yy_current_state = yy_nxt[yy_base[yy_current_state] + yy_c]; } return yy_current_state; } /* yy_try_NUL_trans - try to make a transition on the NUL character * * synopsis * next_state = yy_try_NUL_trans( current_state ); */ static yy_state_type yy_try_NUL_trans (yy_state_type yy_current_state ) { int yy_is_jam; char *yy_cp = (yy_c_buf_p); YY_CHAR yy_c = 1; if ( yy_accept[yy_current_state] ) { (yy_last_accepting_state) = yy_current_state; (yy_last_accepting_cpos) = yy_cp; } while ( yy_chk[yy_base[yy_current_state] + yy_c] != yy_current_state ) { yy_current_state = (int) yy_def[yy_current_state]; if ( yy_current_state >= 1103 ) yy_c = yy_meta[yy_c]; } yy_current_state = yy_nxt[yy_base[yy_current_state] + yy_c]; yy_is_jam = (yy_current_state == 1102); return yy_is_jam ? 0 : yy_current_state; } #ifndef YY_NO_UNPUT #endif #ifndef YY_NO_INPUT #ifdef __cplusplus static int yyinput (void) #else static int input (void) #endif { int c; *(yy_c_buf_p) = (yy_hold_char); if ( *(yy_c_buf_p) == YY_END_OF_BUFFER_CHAR ) { /* yy_c_buf_p now points to the character we want to return. * If this occurs *before* the EOB characters, then it's a * valid NUL; if not, then we've hit the end of the buffer. */ if ( (yy_c_buf_p) < &YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[(yy_n_chars)] ) /* This was really a NUL. */ *(yy_c_buf_p) = '\0'; else { /* need more input */ int offset = (int) ((yy_c_buf_p) - (yytext_ptr)); ++(yy_c_buf_p); switch ( yy_get_next_buffer( ) ) { case EOB_ACT_LAST_MATCH: /* This happens because yy_g_n_b() * sees that we've accumulated a * token and flags that we need to * try matching the token before * proceeding. But for input(), * there's no matching to consider. * So convert the EOB_ACT_LAST_MATCH * to EOB_ACT_END_OF_FILE. */ /* Reset buffer status. */ yyrestart( yyin ); /*FALLTHROUGH*/ case EOB_ACT_END_OF_FILE: { if ( yywrap( ) ) return 0; if ( ! (yy_did_buffer_switch_on_eof) ) YY_NEW_FILE; #ifdef __cplusplus return yyinput(); #else return input(); #endif } case EOB_ACT_CONTINUE_SCAN: (yy_c_buf_p) = (yytext_ptr) + offset; break; } } } c = *(unsigned char *) (yy_c_buf_p); /* cast for 8-bit char's */ *(yy_c_buf_p) = '\0'; /* preserve yytext */ (yy_hold_char) = *++(yy_c_buf_p); return c; } #endif /* ifndef YY_NO_INPUT */ /** Immediately switch to a different input stream. * @param input_file A readable stream. * * @note This function does not reset the start condition to @c INITIAL . */ void yyrestart (FILE * input_file ) { if ( ! YY_CURRENT_BUFFER ){ yyensure_buffer_stack (); YY_CURRENT_BUFFER_LVALUE = yy_create_buffer( yyin, YY_BUF_SIZE ); } yy_init_buffer( YY_CURRENT_BUFFER, input_file ); yy_load_buffer_state( ); } /** Switch to a different input buffer. * @param new_buffer The new input buffer. * */ void yy_switch_to_buffer (YY_BUFFER_STATE new_buffer ) { /* TODO. We should be able to replace this entire function body * with * yypop_buffer_state(); * yypush_buffer_state(new_buffer); */ yyensure_buffer_stack (); if ( YY_CURRENT_BUFFER == new_buffer ) return; if ( YY_CURRENT_BUFFER ) { /* Flush out information for old buffer. */ *(yy_c_buf_p) = (yy_hold_char); YY_CURRENT_BUFFER_LVALUE->yy_buf_pos = (yy_c_buf_p); YY_CURRENT_BUFFER_LVALUE->yy_n_chars = (yy_n_chars); } YY_CURRENT_BUFFER_LVALUE = new_buffer; yy_load_buffer_state( ); /* We don't actually know whether we did this switch during * EOF (yywrap()) processing, but the only time this flag * is looked at is after yywrap() is called, so it's safe * to go ahead and always set it. */ (yy_did_buffer_switch_on_eof) = 1; } static void yy_load_buffer_state (void) { (yy_n_chars) = YY_CURRENT_BUFFER_LVALUE->yy_n_chars; (yytext_ptr) = (yy_c_buf_p) = YY_CURRENT_BUFFER_LVALUE->yy_buf_pos; yyin = YY_CURRENT_BUFFER_LVALUE->yy_input_file; (yy_hold_char) = *(yy_c_buf_p); } /** Allocate and initialize an input buffer state. * @param file A readable stream. * @param size The character buffer size in bytes. When in doubt, use @c YY_BUF_SIZE. * * @return the allocated buffer state. */ YY_BUFFER_STATE yy_create_buffer (FILE * file, int size ) { YY_BUFFER_STATE b; b = (YY_BUFFER_STATE) yyalloc( sizeof( struct yy_buffer_state ) ); if ( ! b ) YY_FATAL_ERROR( "out of dynamic memory in yy_create_buffer()" ); b->yy_buf_size = size; /* yy_ch_buf has to be 2 characters longer than the size given because * we need to put in 2 end-of-buffer characters. */ b->yy_ch_buf = (char *) yyalloc( (yy_size_t) (b->yy_buf_size + 2) ); if ( ! b->yy_ch_buf ) YY_FATAL_ERROR( "out of dynamic memory in yy_create_buffer()" ); b->yy_is_our_buffer = 1; yy_init_buffer( b, file ); return b; } /** Destroy the buffer. * @param b a buffer created with yy_create_buffer() * */ void yy_delete_buffer (YY_BUFFER_STATE b ) { if ( ! b ) return; if ( b == YY_CURRENT_BUFFER ) /* Not sure if we should pop here. */ YY_CURRENT_BUFFER_LVALUE = (YY_BUFFER_STATE) 0; if ( b->yy_is_our_buffer ) yyfree( (void *) b->yy_ch_buf ); yyfree( (void *) b ); } /* Initializes or reinitializes a buffer. * This function is sometimes called more than once on the same buffer, * such as during a yyrestart() or at EOF. */ static void yy_init_buffer (YY_BUFFER_STATE b, FILE * file ) { int oerrno = errno; yy_flush_buffer( b ); b->yy_input_file = file; b->yy_fill_buffer = 1; /* If b is the current buffer, then yy_init_buffer was _probably_ * called from yyrestart() or through yy_get_next_buffer. * In that case, we don't want to reset the lineno or column. */ if (b != YY_CURRENT_BUFFER){ b->yy_bs_lineno = 1; b->yy_bs_column = 0; } b->yy_is_interactive = 0; errno = oerrno; } /** Discard all buffered characters. On the next scan, YY_INPUT will be called. * @param b the buffer state to be flushed, usually @c YY_CURRENT_BUFFER. * */ void yy_flush_buffer (YY_BUFFER_STATE b ) { if ( ! b ) return; b->yy_n_chars = 0; /* We always need two end-of-buffer characters. The first causes * a transition to the end-of-buffer state. The second causes * a jam in that state. */ b->yy_ch_buf[0] = YY_END_OF_BUFFER_CHAR; b->yy_ch_buf[1] = YY_END_OF_BUFFER_CHAR; b->yy_buf_pos = &b->yy_ch_buf[0]; b->yy_at_bol = 1; b->yy_buffer_status = YY_BUFFER_NEW; if ( b == YY_CURRENT_BUFFER ) yy_load_buffer_state( ); } /** Pushes the new state onto the stack. The new state becomes * the current state. This function will allocate the stack * if necessary. * @param new_buffer The new state. * */ void yypush_buffer_state (YY_BUFFER_STATE new_buffer ) { if (new_buffer == NULL) return; yyensure_buffer_stack(); /* This block is copied from yy_switch_to_buffer. */ if ( YY_CURRENT_BUFFER ) { /* Flush out information for old buffer. */ *(yy_c_buf_p) = (yy_hold_char); YY_CURRENT_BUFFER_LVALUE->yy_buf_pos = (yy_c_buf_p); YY_CURRENT_BUFFER_LVALUE->yy_n_chars = (yy_n_chars); } /* Only push if top exists. Otherwise, replace top. */ if (YY_CURRENT_BUFFER) (yy_buffer_stack_top)++; YY_CURRENT_BUFFER_LVALUE = new_buffer; /* copied from yy_switch_to_buffer. */ yy_load_buffer_state( ); (yy_did_buffer_switch_on_eof) = 1; } /** Removes and deletes the top of the stack, if present. * The next element becomes the new top. * */ void yypop_buffer_state (void) { if (!YY_CURRENT_BUFFER) return; yy_delete_buffer(YY_CURRENT_BUFFER ); YY_CURRENT_BUFFER_LVALUE = NULL; if ((yy_buffer_stack_top) > 0) --(yy_buffer_stack_top); if (YY_CURRENT_BUFFER) { yy_load_buffer_state( ); (yy_did_buffer_switch_on_eof) = 1; } } /* Allocates the stack if it does not exist. * Guarantees space for at least one push. */ static void yyensure_buffer_stack (void) { yy_size_t num_to_alloc; if (!(yy_buffer_stack)) { /* First allocation is just for 2 elements, since we don't know if this * scanner will even need a stack. We use 2 instead of 1 to avoid an * immediate realloc on the next call. */ num_to_alloc = 1; /* After all that talk, this was set to 1 anyways... */ (yy_buffer_stack) = (struct yy_buffer_state**)yyalloc (num_to_alloc * sizeof(struct yy_buffer_state*) ); if ( ! (yy_buffer_stack) ) YY_FATAL_ERROR( "out of dynamic memory in yyensure_buffer_stack()" ); memset((yy_buffer_stack), 0, num_to_alloc * sizeof(struct yy_buffer_state*)); (yy_buffer_stack_max) = num_to_alloc; (yy_buffer_stack_top) = 0; return; } if ((yy_buffer_stack_top) >= ((yy_buffer_stack_max)) - 1){ /* Increase the buffer to prepare for a possible push. */ yy_size_t grow_size = 8 /* arbitrary grow size */; num_to_alloc = (yy_buffer_stack_max) + grow_size; (yy_buffer_stack) = (struct yy_buffer_state**)yyrealloc ((yy_buffer_stack), num_to_alloc * sizeof(struct yy_buffer_state*) ); if ( ! (yy_buffer_stack) ) YY_FATAL_ERROR( "out of dynamic memory in yyensure_buffer_stack()" ); /* zero only the new slots.*/ memset((yy_buffer_stack) + (yy_buffer_stack_max), 0, grow_size * sizeof(struct yy_buffer_state*)); (yy_buffer_stack_max) = num_to_alloc; } } /** Setup the input buffer state to scan directly from a user-specified character buffer. * @param base the character buffer * @param size the size in bytes of the character buffer * * @return the newly allocated buffer state object. */ YY_BUFFER_STATE yy_scan_buffer (char * base, yy_size_t size ) { YY_BUFFER_STATE b; if ( size < 2 || base[size-2] != YY_END_OF_BUFFER_CHAR || base[size-1] != YY_END_OF_BUFFER_CHAR ) /* They forgot to leave room for the EOB's. */ return NULL; b = (YY_BUFFER_STATE) yyalloc( sizeof( struct yy_buffer_state ) ); if ( ! b ) YY_FATAL_ERROR( "out of dynamic memory in yy_scan_buffer()" ); b->yy_buf_size = (int) (size - 2); /* "- 2" to take care of EOB's */ b->yy_buf_pos = b->yy_ch_buf = base; b->yy_is_our_buffer = 0; b->yy_input_file = NULL; b->yy_n_chars = b->yy_buf_size; b->yy_is_interactive = 0; b->yy_at_bol = 1; b->yy_fill_buffer = 0; b->yy_buffer_status = YY_BUFFER_NEW; yy_switch_to_buffer( b ); return b; } /** Setup the input buffer state to scan a string. The next call to yylex() will * scan from a @e copy of @a str. * @param yystr a NUL-terminated string to scan * * @return the newly allocated buffer state object. * @note If you want to scan bytes that may contain NUL values, then use * yy_scan_bytes() instead. */ YY_BUFFER_STATE yy_scan_string (const char * yystr ) { return yy_scan_bytes( yystr, (int) strlen(yystr) ); } /** Setup the input buffer state to scan the given bytes. The next call to yylex() will * scan from a @e copy of @a bytes. * @param yybytes the byte buffer to scan * @param _yybytes_len the number of bytes in the buffer pointed to by @a bytes. * * @return the newly allocated buffer state object. */ YY_BUFFER_STATE yy_scan_bytes (const char * yybytes, int _yybytes_len ) { YY_BUFFER_STATE b; char *buf; yy_size_t n; int i; /* Get memory for full buffer, including space for trailing EOB's. */ n = (yy_size_t) (_yybytes_len + 2); buf = (char *) yyalloc( n ); if ( ! buf ) YY_FATAL_ERROR( "out of dynamic memory in yy_scan_bytes()" ); for ( i = 0; i < _yybytes_len; ++i ) buf[i] = yybytes[i]; buf[_yybytes_len] = buf[_yybytes_len+1] = YY_END_OF_BUFFER_CHAR; b = yy_scan_buffer( buf, n ); if ( ! b ) YY_FATAL_ERROR( "bad buffer in yy_scan_bytes()" ); /* It's okay to grow etc. this buffer, and we should throw it * away when we're done. */ b->yy_is_our_buffer = 1; return b; } #ifndef YY_EXIT_FAILURE #define YY_EXIT_FAILURE 2 #endif static void yynoreturn yy_fatal_error (const char* msg ) { fprintf( stderr, "%s\n", msg ); exit( YY_EXIT_FAILURE ); } /* Redefine yyless() so it works in section 3 code. */ #undef yyless #define yyless(n) \ do \ { \ /* Undo effects of setting up yytext. */ \ int yyless_macro_arg = (n); \ YY_LESS_LINENO(yyless_macro_arg);\ yytext[yyleng] = (yy_hold_char); \ (yy_c_buf_p) = yytext + yyless_macro_arg; \ (yy_hold_char) = *(yy_c_buf_p); \ *(yy_c_buf_p) = '\0'; \ yyleng = yyless_macro_arg; \ } \ while ( 0 ) /* Accessor methods (get/set functions) to struct members. */ /** Get the current line number. * */ int yyget_lineno (void) { return yylineno; } /** Get the input stream. * */ FILE *yyget_in (void) { return yyin; } /** Get the output stream. * */ FILE *yyget_out (void) { return yyout; } /** Get the length of the current token. * */ int yyget_leng (void) { return yyleng; } /** Get the current token. * */ char *yyget_text (void) { return yytext; } /** Set the current line number. * @param _line_number line number * */ void yyset_lineno (int _line_number ) { yylineno = _line_number; } /** Set the input stream. This does not discard the current * input buffer. * @param _in_str A readable stream. * * @see yy_switch_to_buffer */ void yyset_in (FILE * _in_str ) { yyin = _in_str ; } void yyset_out (FILE * _out_str ) { yyout = _out_str ; } int yyget_debug (void) { return yy_flex_debug; } void yyset_debug (int _bdebug ) { yy_flex_debug = _bdebug ; } static int yy_init_globals (void) { /* Initialization is the same as for the non-reentrant scanner. * This function is called from yylex_destroy(), so don't allocate here. */ (yy_buffer_stack) = NULL; (yy_buffer_stack_top) = 0; (yy_buffer_stack_max) = 0; (yy_c_buf_p) = NULL; (yy_init) = 0; (yy_start) = 0; /* Defined in main.c */ #ifdef YY_STDINIT yyin = stdin; yyout = stdout; #else yyin = NULL; yyout = NULL; #endif /* For future reference: Set errno on error, since we are called by * yylex_init() */ return 0; } /* yylex_destroy is for both reentrant and non-reentrant scanners. */ int yylex_destroy (void) { /* Pop the buffer stack, destroying each element. */ while(YY_CURRENT_BUFFER){ yy_delete_buffer( YY_CURRENT_BUFFER ); YY_CURRENT_BUFFER_LVALUE = NULL; yypop_buffer_state(); } /* Destroy the stack itself. */ yyfree((yy_buffer_stack) ); (yy_buffer_stack) = NULL; /* Reset the globals. This is important in a non-reentrant scanner so the next time * yylex() is called, initialization will occur. */ yy_init_globals( ); return 0; } /* * Internal utility routines. */ #ifndef yytext_ptr static void yy_flex_strncpy (char* s1, const char * s2, int n ) { int i; for ( i = 0; i < n; ++i ) s1[i] = s2[i]; } #endif #ifdef YY_NEED_STRLEN static int yy_flex_strlen (const char * s ) { int n; for ( n = 0; s[n]; ++n ) ; return n; } #endif void *yyalloc (yy_size_t size ) { return malloc(size); } void *yyrealloc (void * ptr, yy_size_t size ) { /* The cast to (char *) in the following accommodates both * implementations that use char* generic pointers, and those * that use void* generic pointers. It works with the latter * because both ANSI C and C++ allow castless assignment from * any pointer type to void*, and deal with argument conversions * as though doing an assignment. */ return realloc(ptr, size); } void yyfree (void * ptr ) { free( (char *) ptr ); /* see yyrealloc() for (char *) cast */ } #define YYTABLES_NAME "yytables" #line 167 "css.l" wget-1.21.2/src/Makefile.am0000644000000000000000000001015014057013220012244 00000000000000# Makefile for `wget' utility # Copyright (C) 1995-2011, 2015, 2018-2021 Free Software Foundation, # Inc. # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 3 of the License, or # (at your option) any later version. # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # You should have received a copy of the GNU General Public License # along with Wget. If not, see . # Additional permission under GNU GPL version 3 section 7 # If you modify this program, or any covered work, by linking or # combining it with the OpenSSL project's OpenSSL library (or a # modified version of that library), containing parts covered by the # terms of the OpenSSL or SSLeay licenses, the Free Software Foundation # grants you additional permission to convey the resulting work. # Corresponding Source for a non-source form of such a combination # shall include the source code for the parts of OpenSSL used as well # as that of the covered work. # # Version: @VERSION@ # if IRI_IS_ENABLED IRI_OBJ = iri.c endif if METALINK_IS_ENABLED METALINK_OBJ = metalink.c endif if WITH_XATTR XATTR_OBJ = xattr.c endif # The following line is losing on some versions of make! DEFS = @DEFS@ -DSYSTEM_WGETRC=\"$(sysconfdir)/wgetrc\" -DLOCALEDIR=\"$(localedir)\" EXTRA_DIST = css.l css.c css_.c build_info.c.in build_info.c bin_PROGRAMS = wget wget_SOURCES = connect.c convert.c cookies.c ftp.c \ css_.c css-url.c \ ftp-basic.c ftp-ls.c hash.c host.c hsts.c html-parse.c html-url.c \ http.c init.c log.c main.c netrc.c progress.c ptimer.c \ recur.c res.c retr.c spider.c url.c warc.c $(XATTR_OBJ) \ utils.c exits.c build_info.c $(IRI_OBJ) $(METALINK_OBJ) \ css-url.h css-tokens.h connect.h convert.h cookies.h \ ftp.h hash.h host.h hsts.h html-parse.h html-url.h \ http.h http-ntlm.h init.h log.h mswindows.h netrc.h \ options.h progress.h ptimer.h recur.h res.h retr.h \ spider.h ssl.h sysdep.h url.h warc.h utils.h wget.h iri.h \ exits.h version.h metalink.h xattr.h nodist_wget_SOURCES = version.c EXTRA_wget_SOURCES = iri.c metalink.c xattr.c LDADD = $(CODE_COVERAGE_LIBS) $(LIBOBJS) ../lib/libgnu.a $(GETADDRINFO_LIB) $(HOSTENT_LIB)\ $(INET_NTOP_LIB) $(LIBSOCKET) $(LIB_CLOCK_GETTIME) $(LIB_CRYPTO)\ $(LIB_NANOSLEEP) $(LIB_POSIX_SPAWN) $(LIB_SELECT) $(LIBICONV) $(LIBINTL)\ $(LIBTHREAD) $(LIBUNISTRING) $(SERVENT_LIB) AM_CPPFLAGS = -I$(top_builddir)/lib -I$(top_srcdir)/lib $(CODE_COVERAGE_CPPFLAGS) AM_CFLAGS = $(WERROR_CFLAGS) $(WARN_CFLAGS) $(CODE_COVERAGE_CFLAGS) ../lib/libgnu.a: cd ../lib && $(MAKE) $(AM_MAKEFLAGS) build_info.c: $(srcdir)/Makefile.am $(srcdir)/build_info.c.in if test -n "$(VPATH)"; then cp "$(srcdir)/build_info.c.in" .; fi $(PERL) "$(top_srcdir)/build-aux/build_info.pl" \ "$(top_builddir)/src/build_info.c" if test -n "$(VPATH)"; then rm -f build_info.c.in; fi ESCAPEQUOTE = sed -e 's/[\\"]/\\&/g' -e 's/\\"/"/' -e 's/\\";$$/";/' version.c: $(wget_SOURCES) ../lib/libgnu.a echo '/* version.c */' > $@ echo '/* Autogenerated by Makefile - DO NOT EDIT */' >> $@ echo '' >> $@ echo '#include "version.h"' >> $@ echo 'const char *version_string = "@VERSION@";' >> $@ echo 'const char *compilation_string = "'$(COMPILE)'";' \ | $(ESCAPEQUOTE) >> $@ echo 'const char *link_string = "'$(CCLD) $(AM_CFLAGS) $(CFLAGS) \ $(AM_LDFLAGS) $(LDFLAGS) $(LIBS) $(wget_LDADD)'";' \ | $(ESCAPEQUOTE) >> $@ css.c: $(srcdir)/css.l $(LEX) $(LFLAGS) -o$@ $^ css_.c: css.c echo '#include "wget.h"' > $@ cat css.c >> $@ distclean-local: rm -f css.c css_.c check_LIBRARIES = libunittest.a libunittest_a_SOURCES = $(wget_SOURCES) build_info.c nodist_libunittest_a_SOURCES = version.c libunittest_a_CPPFLAGS = -DTESTING "-I$(top_builddir)/lib" "-I$(top_srcdir)/lib" $(CODE_COVERAGE_CPPLAGS) libunittest_a_LIBADD = $(LIBOBJS) CLEANFILES = *~ *.bak core core.[0-9]* build_info.c version.c wget-1.21.2/src/warc.c0000644000000000000000000014746714115732710011345 00000000000000/* Utility functions for writing WARC files. Copyright (C) 2011-2012, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #include "wget.h" #include "hash.h" #include "utils.h" #include "version.h" #include "dirname.h" #include "url.h" #include #include #include #include #include #include #include #include #ifdef HAVE_LIBZ #include #endif #ifdef HAVE_LIBUUID #include #elif HAVE_UUID_CREATE #include #endif #include "warc.h" #include "exits.h" #ifdef WINDOWS /* we need this on Windows to have O_TEMPORARY defined */ # include # include #endif #ifndef O_TEMPORARY #define O_TEMPORARY 0 #endif #include "warc.h" #include "exits.h" /* The log file (a temporary file that contains a copy of the wget log). */ static FILE *warc_log_fp; /* The manifest file (a temporary file that contains the warcinfo uuid of every file in this crawl). */ static FILE *warc_manifest_fp; /* The current WARC file (or NULL, if WARC is disabled). */ static FILE *warc_current_file; #ifdef HAVE_LIBZ /* The gzip stream for the current WARC file (or NULL, if WARC or gzip is disabled). */ static gzFile warc_current_gzfile; /* The offset of the current gzip record in the WARC file. */ static off_t warc_current_gzfile_offset; /* The uncompressed size (so far) of the current record. */ static off_t warc_current_gzfile_uncompressed_size; # endif /* This is true until a warc_write_* method fails. */ static bool warc_write_ok; /* The current CDX file (or NULL, if CDX is disabled). */ static FILE *warc_current_cdx_file; /* The record id of the warcinfo record of the current WARC file. */ static char warc_current_warcinfo_uuid_str[48]; /* The file name of the current WARC file. */ static char *warc_current_filename; /* The serial number of the current WARC file. This number is incremented each time a new file is opened and is used in the WARC file's filename. */ static int warc_current_file_number; /* The table of CDX records, if deduplication is enabled. */ static struct hash_table * warc_cdx_dedup_table; static bool warc_start_new_file (bool meta); struct warc_cdx_record { char *url; char *uuid; char digest[SHA1_DIGEST_SIZE]; }; static unsigned long warc_hash_sha1_digest (const void *key) { /* We just use some of the first bytes of the digest. */ unsigned long v = 0; memcpy (&v, key, sizeof (unsigned long)); return v; } static int warc_cmp_sha1_digest (const void *digest1, const void *digest2) { return !memcmp (digest1, digest2, SHA1_DIGEST_SIZE); } /* Writes SIZE bytes from BUFFER to the current WARC file, through gzwrite if compression is enabled. Returns the number of uncompressed bytes written. */ static size_t warc_write_buffer (const char *buffer, size_t size) { #ifdef HAVE_LIBZ if (warc_current_gzfile) { warc_current_gzfile_uncompressed_size += size; return gzwrite (warc_current_gzfile, buffer, size); } else #endif return fwrite (buffer, 1, size, warc_current_file); } /* Writes STR to the current WARC file. Returns false and set warc_write_ok to false if there is an error. */ static bool warc_write_string (const char *str) { size_t n; if (!warc_write_ok) return false; n = strlen (str); if (n != warc_write_buffer (str, n)) warc_write_ok = false; return warc_write_ok; } #define EXTRA_GZIP_HEADER_SIZE 14 #define GZIP_STATIC_HEADER_SIZE 10 #define FLG_FEXTRA 0x04 #define OFF_FLG 3 /* Starts a new WARC record. Writes the version header. If opt.warc_maxsize is set and the current file is becoming too large, this will open a new WARC file. If compression is enabled, this will start a new gzip stream in the current WARC file. Returns false and set warc_write_ok to false if there is an error. */ static bool warc_write_start_record (void) { if (!warc_write_ok) return false; fflush (warc_current_file); if (opt.warc_maxsize > 0 && ftello (warc_current_file) >= opt.warc_maxsize) warc_start_new_file (false); #ifdef HAVE_LIBZ /* Start a GZIP stream, if required. */ if (opt.warc_compression_enabled) { int dup_fd; /* Record the starting offset of the new record. */ warc_current_gzfile_offset = ftello (warc_current_file); /* Reserve space for the extra GZIP header field. In warc_write_end_record we will fill this space with information about the uncompressed and compressed size of the record. */ if (fseek (warc_current_file, EXTRA_GZIP_HEADER_SIZE, SEEK_CUR) < 0) { logprintf (LOG_NOTQUIET, _("Error setting WARC file position.\n")); warc_write_ok = false; return false; } if (fflush (warc_current_file) != 0) { logprintf (LOG_NOTQUIET, _("Error flushing WARC file to disk.\n")); warc_write_ok = false; return false; } /* Start a new GZIP stream. */ dup_fd = dup (fileno (warc_current_file)); if (dup_fd < 0) { logprintf (LOG_NOTQUIET, _("Error duplicating WARC file file descriptor.\n")); warc_write_ok = false; return false; } warc_current_gzfile = gzdopen (dup_fd, "wb9"); warc_current_gzfile_uncompressed_size = 0; if (warc_current_gzfile == NULL) { logprintf (LOG_NOTQUIET, _("Error opening GZIP stream to WARC file.\n")); close (dup_fd); warc_write_ok = false; return false; } } #endif warc_write_string ("WARC/1.0\r\n"); return warc_write_ok; } /* Writes a WARC header to the current WARC record. This method may be run after warc_write_start_record and before warc_write_block_from_file. */ static bool warc_write_header (const char *name, const char *value) { if (value) { warc_write_string (name); warc_write_string (": "); warc_write_string (value); warc_write_string ("\r\n"); } return warc_write_ok; } /* Writes a WARC header with a URI as value to the current WARC record. This method may be run after warc_write_start_record and before warc_write_block_from_file. */ static bool warc_write_header_uri (const char *name, const char *value) { if (value) { warc_write_string (name); warc_write_string (": <"); warc_write_string (value); warc_write_string (">\r\n"); } return warc_write_ok; } /* Copies the contents of DATA_IN to the WARC record. Adds a Content-Length header to the WARC record. Run this method after warc_write_header, then run warc_write_end_record. */ static bool warc_write_block_from_file (FILE *data_in) { /* Add the Content-Length header. */ char content_length[MAX_INT_TO_STRING_LEN(off_t)]; char buffer[BUFSIZ]; size_t s; fseeko (data_in, 0L, SEEK_END); number_to_string (content_length, ftello (data_in)); warc_write_header ("Content-Length", content_length); /* End of the WARC header section. */ warc_write_string ("\r\n"); if (fseeko (data_in, 0L, SEEK_SET) != 0) warc_write_ok = false; /* Copy the data in the file to the WARC record. */ while (warc_write_ok && (s = fread (buffer, 1, BUFSIZ, data_in)) > 0) { if (warc_write_buffer (buffer, s) < s) warc_write_ok = false; } return warc_write_ok; } /* Run this method to close the current WARC record. If compression is enabled, this method closes the current GZIP stream and fills the extra GZIP header with the uncompressed and compressed length of the record. */ static bool warc_write_end_record (void) { if (warc_write_buffer ("\r\n\r\n", 4) != 4) { warc_write_ok = false; return false; } #ifdef HAVE_LIBZ /* We start a new gzip stream for each record. */ if (warc_write_ok && warc_current_gzfile) { char extra_header[EXTRA_GZIP_HEADER_SIZE]; char static_header[GZIP_STATIC_HEADER_SIZE]; off_t current_offset, uncompressed_size, compressed_size; size_t result; if (gzclose (warc_current_gzfile) != Z_OK) { warc_write_ok = false; return false; } fflush (warc_current_file); fseeko (warc_current_file, 0, SEEK_END); /* The WARC standard suggests that we add 'skip length' data in the extra header field of the GZIP stream. In warc_write_start_record we reserved space for this extra header. This extra space starts at warc_current_gzfile_offset and fills EXTRA_GZIP_HEADER_SIZE bytes. The static GZIP header starts at warc_current_gzfile_offset + EXTRA_GZIP_HEADER_SIZE. We need to do three things: 1. Move the static GZIP header to warc_current_gzfile_offset; 2. Set the FEXTRA flag in the GZIP header; 3. Write the extra GZIP header after the static header, that is, starting at warc_current_gzfile_offset + GZIP_STATIC_HEADER_SIZE. */ /* Calculate the uncompressed and compressed sizes. */ current_offset = ftello (warc_current_file); uncompressed_size = current_offset - warc_current_gzfile_offset; compressed_size = warc_current_gzfile_uncompressed_size; /* Go back to the static GZIP header. */ result = fseeko (warc_current_file, warc_current_gzfile_offset + EXTRA_GZIP_HEADER_SIZE, SEEK_SET); if (result != 0) { warc_write_ok = false; return false; } /* Read the header. */ result = fread (static_header, 1, GZIP_STATIC_HEADER_SIZE, warc_current_file); if (result != GZIP_STATIC_HEADER_SIZE) { warc_write_ok = false; return false; } /* Set the FEXTRA flag in the flags byte of the header. */ static_header[OFF_FLG] = static_header[OFF_FLG] | FLG_FEXTRA; /* Write the header back to the file, but starting at warc_current_gzfile_offset. */ fseeko (warc_current_file, warc_current_gzfile_offset, SEEK_SET); fwrite (static_header, 1, GZIP_STATIC_HEADER_SIZE, warc_current_file); /* Prepare the extra GZIP header. */ /* XLEN, the length of the extra header fields. */ extra_header[0] = ((EXTRA_GZIP_HEADER_SIZE - 2) & 255); extra_header[1] = ((EXTRA_GZIP_HEADER_SIZE - 2) >> 8) & 255; /* The extra header field identifier for the WARC skip length. */ extra_header[2] = 's'; extra_header[3] = 'l'; /* The size of the field value (8 bytes). */ extra_header[4] = (8 & 255); extra_header[5] = ((8 >> 8) & 255); /* The size of the uncompressed record. */ extra_header[6] = (uncompressed_size & 255); extra_header[7] = (uncompressed_size >> 8) & 255; extra_header[8] = (uncompressed_size >> 16) & 255; extra_header[9] = (uncompressed_size >> 24) & 255; /* The size of the compressed record. */ extra_header[10] = (compressed_size & 255); extra_header[11] = (compressed_size >> 8) & 255; extra_header[12] = (compressed_size >> 16) & 255; extra_header[13] = (compressed_size >> 24) & 255; /* Write the extra header after the static header. */ fseeko (warc_current_file, warc_current_gzfile_offset + GZIP_STATIC_HEADER_SIZE, SEEK_SET); fwrite (extra_header, 1, EXTRA_GZIP_HEADER_SIZE, warc_current_file); /* Done, move back to the end of the file. */ fflush (warc_current_file); fseeko (warc_current_file, 0, SEEK_END); } #endif /* HAVE_LIBZ */ return warc_write_ok; } /* Writes the WARC-Date header for the given timestamp to the current WARC record. If timestamp is NULL, the current time will be used. */ static bool warc_write_date_header (const char *timestamp) { char current_timestamp[21]; return warc_write_header ("WARC-Date", timestamp ? timestamp : warc_timestamp (current_timestamp, sizeof(current_timestamp))); } /* Writes the WARC-IP-Address header for the given IP to the current WARC record. If IP is NULL, no header will be written. */ static bool warc_write_ip_header (const ip_address *ip) { if (ip != NULL) return warc_write_header ("WARC-IP-Address", print_address (ip)); else return warc_write_ok; } /* warc_sha1_stream_with_payload is a modified copy of sha1_stream from gnulib/sha1.c. This version calculates two digests in one go. Compute SHA1 message digests for bytes read from STREAM. The digest of the complete file will be written into the 16 bytes beginning at RES_BLOCK. If payload_offset >= 0, a second digest will be calculated of the portion of the file starting at payload_offset and continuing to the end of the file. The digest number will be written into the 16 bytes beginning ad RES_PAYLOAD. */ static int warc_sha1_stream_with_payload (FILE *stream, void *res_block, void *res_payload, off_t payload_offset) { #define BLOCKSIZE 32768 struct sha1_ctx ctx_block; struct sha1_ctx ctx_payload; off_t pos; off_t sum; char *buffer = xmalloc (BLOCKSIZE + 72); /* Initialize the computation context. */ sha1_init_ctx (&ctx_block); if (payload_offset >= 0) sha1_init_ctx (&ctx_payload); pos = 0; /* Iterate over full file contents. */ while (1) { /* We read the file in blocks of BLOCKSIZE bytes. One call of the computation function processes the whole buffer so that with the next round of the loop another block can be read. */ off_t n; sum = 0; /* Read block. Take care for partial reads. */ while (1) { n = fread (buffer + sum, 1, BLOCKSIZE - sum, stream); sum += n; pos += n; if (sum == BLOCKSIZE) break; if (n == 0) { /* Check for the error flag IF N == 0, so that we don't exit the loop after a partial read due to e.g., EAGAIN or EWOULDBLOCK. */ if (ferror (stream)) { xfree (buffer); return 1; } goto process_partial_block; } /* We've read at least one byte, so ignore errors. But always check for EOF, since feof may be true even though N > 0. Otherwise, we could end up calling fread after EOF. */ if (feof (stream)) goto process_partial_block; } /* Process buffer with BLOCKSIZE bytes. Note that BLOCKSIZE % 64 == 0 */ sha1_process_block (buffer, BLOCKSIZE, &ctx_block); if (payload_offset >= 0 && payload_offset < pos) { /* At least part of the buffer contains data from payload. */ off_t start_of_payload = payload_offset - (pos - BLOCKSIZE); if (start_of_payload <= 0) /* All bytes in the buffer belong to the payload. */ start_of_payload = 0; /* Process the payload part of the buffer. Note: we can't use sha1_process_block here even if we process the complete buffer. Because the payload doesn't have to start with a full block, there may still be some bytes left from the previous buffer. Therefore, we need to continue with sha1_process_bytes. */ sha1_process_bytes (buffer + start_of_payload, BLOCKSIZE - start_of_payload, &ctx_payload); } } process_partial_block:; /* Process any remaining bytes. */ if (sum > 0) { sha1_process_bytes (buffer, sum, &ctx_block); if (payload_offset >= 0 && payload_offset < pos) { /* At least part of the buffer contains data from payload. */ off_t start_of_payload = payload_offset - (pos - sum); if (start_of_payload <= 0) /* All bytes in the buffer belong to the payload. */ start_of_payload = 0; /* Process the payload part of the buffer. */ sha1_process_bytes (buffer + start_of_payload, sum - start_of_payload, &ctx_payload); } } /* Construct result in desired memory. */ sha1_finish_ctx (&ctx_block, res_block); if (payload_offset >= 0) sha1_finish_ctx (&ctx_payload, res_payload); xfree (buffer); return 0; #undef BLOCKSIZE } /* Converts the SHA1 digest to a base32-encoded string. "sha1:DIGEST\0" (Allocates a new string for the response.) */ static char * warc_base32_sha1_digest (const char *sha1_digest, char *sha1_base32, size_t sha1_base32_size) { if (sha1_base32_size >= BASE32_LENGTH(SHA1_DIGEST_SIZE) + 5 + 1) { memcpy (sha1_base32, "sha1:", 5); base32_encode (sha1_digest, SHA1_DIGEST_SIZE, sha1_base32 + 5, sha1_base32_size - 5); } else *sha1_base32 = 0; return sha1_base32; } /* Sets the digest headers of the record. This method will calculate the block digest and, if payload_offset >= 0, will also calculate the payload digest of the payload starting at the provided offset. */ static void warc_write_digest_headers (FILE *file, long payload_offset) { if (opt.warc_digests_enabled) { /* Calculate the block and payload digests. */ char sha1_res_block[SHA1_DIGEST_SIZE]; char sha1_res_payload[SHA1_DIGEST_SIZE]; rewind (file); if (warc_sha1_stream_with_payload (file, sha1_res_block, sha1_res_payload, payload_offset) == 0) { char digest[BASE32_LENGTH(SHA1_DIGEST_SIZE) + 1 + 5]; warc_write_header ("WARC-Block-Digest", warc_base32_sha1_digest (sha1_res_block, digest, sizeof(digest))); if (payload_offset >= 0) warc_write_header ("WARC-Payload-Digest", warc_base32_sha1_digest (sha1_res_payload, digest, sizeof(digest))); } } } /* Fills timestamp with the current time and date. The UTC time is formatted following ISO 8601, as required for use in the WARC-Date header. The timestamp will be 21 characters long. */ char * warc_timestamp (char *timestamp, size_t timestamp_size) { time_t rawtime = time (NULL); struct tm * timeinfo = gmtime (&rawtime); if (strftime (timestamp, timestamp_size, "%Y-%m-%dT%H:%M:%SZ", timeinfo) == 0 && timestamp_size > 0) *timestamp = 0; return timestamp; } /* Fills urn_str with a UUID in the format required for the WARC-Record-Id header. The string will be 47 characters long. */ #if HAVE_LIBUUID void warc_uuid_str (char *urn_str, size_t urn_size) { char uuid_str[37]; uuid_t record_id; uuid_generate (record_id); uuid_unparse (record_id, uuid_str); snprintf (urn_str, urn_size, "", uuid_str); } #elif HAVE_UUID_CREATE void warc_uuid_str (char *urn_str, size_t urn_size) { char *uuid_str; uuid_t record_id; uuid_create (&record_id, NULL); uuid_to_string (&record_id, &uuid_str, NULL); snprintf (urn_str, urn_size, "", uuid_str); xfree (uuid_str); } #else # ifdef WINDOWS typedef RPC_STATUS (RPC_ENTRY * UuidCreate_proc) (UUID *); typedef RPC_STATUS (RPC_ENTRY * UuidToString_proc) (UUID *, unsigned char **); typedef RPC_STATUS (RPC_ENTRY * RpcStringFree_proc) (unsigned char **); static int windows_uuid_str (char *urn_str, size_t urn_size) { static UuidCreate_proc pfn_UuidCreate = NULL; static UuidToString_proc pfn_UuidToString = NULL; static RpcStringFree_proc pfn_RpcStringFree = NULL; static int rpc_uuid_avail = -1; /* Rpcrt4.dll is not available on older versions of Windows, so we need to test its availability at run time. */ if (rpc_uuid_avail == -1) { HMODULE hm_rpcrt4 = LoadLibrary ("Rpcrt4.dll"); if (hm_rpcrt4) { pfn_UuidCreate = (UuidCreate_proc) GetProcAddress (hm_rpcrt4, "UuidCreate"); pfn_UuidToString = (UuidToString_proc) GetProcAddress (hm_rpcrt4, "UuidToStringA"); pfn_RpcStringFree = (RpcStringFree_proc) GetProcAddress (hm_rpcrt4, "RpcStringFreeA"); if (pfn_UuidCreate && pfn_UuidToString && pfn_RpcStringFree) rpc_uuid_avail = 1; else rpc_uuid_avail = 0; } else rpc_uuid_avail = 0; } if (rpc_uuid_avail) { BYTE *uuid_str; UUID uuid; if (pfn_UuidCreate (&uuid) == RPC_S_OK) { if (pfn_UuidToString (&uuid, &uuid_str) == RPC_S_OK) { snprintf (urn_str, urn_size, "", uuid_str); pfn_RpcStringFree (&uuid_str); return 1; } } } return 0; } #endif /* Fills urn_str with a UUID based on random numbers in the format required for the WARC-Record-Id header. (See RFC 4122, UUID version 4.) Note: this is a fallback method, it is much better to use the methods provided by libuuid. The string will be 47 characters long. */ void warc_uuid_str (char *urn_str, size_t urn_size) { /* RFC 4122, a version 4 UUID with only random numbers */ unsigned char uuid_data[16]; int i; #ifdef WINDOWS /* If the native method fails (expected on older Windows versions), use the fallback below. */ if (windows_uuid_str (urn_str, urn_size)) return; #endif for (i=0; i<16; i++) uuid_data[i] = random_number (255); /* Set the four most significant bits (bits 12 through 15) of the * time_hi_and_version field to the 4-bit version number */ uuid_data[6] = (uuid_data[6] & 0x0F) | 0x40; /* Set the two most significant bits (bits 6 and 7) of the * clock_seq_hi_and_reserved to zero and one, respectively. */ uuid_data[8] = (uuid_data[8] & 0xBF) | 0x80; snprintf (urn_str, urn_size, "", uuid_data[0], uuid_data[1], uuid_data[2], uuid_data[3], uuid_data[4], uuid_data[5], uuid_data[6], uuid_data[7], uuid_data[8], uuid_data[9], uuid_data[10], uuid_data[11], uuid_data[12], uuid_data[13], uuid_data[14], uuid_data[15]); } #endif /* Write a warcinfo record to the current file. Updates warc_current_warcinfo_uuid_str. */ static bool warc_write_warcinfo_record (const char *filename) { FILE *warc_tmp; char timestamp[22]; char *filename_basename; /* Write warc-info record as the first record of the file. */ /* We add the record id of this info record to the other records in the file. */ warc_uuid_str (warc_current_warcinfo_uuid_str, sizeof (warc_current_warcinfo_uuid_str)); warc_timestamp (timestamp, sizeof(timestamp)); filename_basename = base_name (filename); warc_write_start_record (); warc_write_header ("WARC-Type", "warcinfo"); warc_write_header ("Content-Type", "application/warc-fields"); warc_write_header ("WARC-Date", timestamp); warc_write_header ("WARC-Record-ID", warc_current_warcinfo_uuid_str); warc_write_header ("WARC-Filename", filename_basename); xfree (filename_basename); /* Create content. */ warc_tmp = warc_tempfile (); if (warc_tmp == NULL) { return false; } fprintf (warc_tmp, "software: Wget/%s (%s)\r\n", version_string, OS_TYPE); fprintf (warc_tmp, "format: WARC File Format 1.0\r\n"); fprintf (warc_tmp, "conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf\r\n"); fprintf (warc_tmp, "robots: %s\r\n", (opt.use_robots ? "classic" : "off")); fprintf (warc_tmp, "wget-arguments: %s\r\n", program_argstring); /* Add the user headers, if any. */ if (opt.warc_user_headers) { int i; for (i = 0; opt.warc_user_headers[i]; i++) fprintf (warc_tmp, "%s\r\n", opt.warc_user_headers[i]); } fprintf(warc_tmp, "\r\n"); warc_write_digest_headers (warc_tmp, -1); warc_write_block_from_file (warc_tmp); warc_write_end_record (); if (! warc_write_ok) logprintf (LOG_NOTQUIET, _("Error writing warcinfo record to WARC file.\n")); fclose (warc_tmp); return warc_write_ok; } /* Opens a new WARC file. If META is true, generates a filename ending with 'meta.warc.gz'. This method will: 1. close the current WARC file (if there is one); 2. increment warc_current_file_number; 3. open a new WARC file; 4. write the initial warcinfo record. Returns true on success, false otherwise. */ static bool warc_start_new_file (bool meta) { #ifdef __VMS # define WARC_GZ "warc-gz" #else /* def __VMS */ # define WARC_GZ "warc.gz" #endif /* def __VMS [else] */ #ifdef HAVE_LIBZ const char *extension = (opt.warc_compression_enabled ? WARC_GZ : "warc"); #else const char *extension = "warc"; #endif int base_filename_length; char *new_filename; if (opt.warc_filename == NULL) return false; if (warc_current_file != NULL) fclose (warc_current_file); *warc_current_warcinfo_uuid_str = 0; xfree (warc_current_filename); warc_current_file_number++; base_filename_length = strlen (opt.warc_filename); /* filename format: base + "-" + 5 digit serial number + ".warc.gz" */ new_filename = xmalloc (base_filename_length + 1 + 5 + 8 + 1); warc_current_filename = new_filename; /* If max size is enabled, we add a serial number to the file names. */ if (meta) sprintf (new_filename, "%s-meta.%s", opt.warc_filename, extension); else if (opt.warc_maxsize > 0) { sprintf (new_filename, "%s-%05d.%s", opt.warc_filename, warc_current_file_number, extension); } else sprintf (new_filename, "%s.%s", opt.warc_filename, extension); logprintf (LOG_VERBOSE, _("Opening WARC file %s.\n\n"), quote (new_filename)); /* Open the WARC file. */ warc_current_file = fopen (new_filename, "wb+"); if (warc_current_file == NULL) { logprintf (LOG_NOTQUIET, _("Error opening WARC file %s.\n"), quote (new_filename)); return false; } if (! warc_write_warcinfo_record (new_filename)) return false; /* Add warcinfo uuid to manifest. */ if (warc_manifest_fp) fprintf (warc_manifest_fp, "%s\n", warc_current_warcinfo_uuid_str); return true; } /* Opens the CDX file for output. */ static bool warc_start_cdx_file (void) { char *cdx_filename = aprintf("%s.cdx", opt.warc_filename); warc_current_cdx_file = fopen (cdx_filename, "a+"); free(cdx_filename); if (warc_current_cdx_file == NULL) return false; /* Print the CDX header. * * a - original url * b - date * m - mime type * s - response code * k - new style checksum * r - redirect * M - meta tags * V - compressed arc file offset * g - file name * u - record-id */ fprintf (warc_current_cdx_file, " CDX a b a m s k r M V g u\n"); fflush (warc_current_cdx_file); return true; } #define CDX_FIELDSEP " \t\r\n" /* Parse the CDX header and find the field numbers of the original url, checksum and record ID fields. */ static bool warc_parse_cdx_header (char *lineptr, int *field_num_original_url, int *field_num_checksum, int *field_num_record_id) { char *token; char *save_ptr; *field_num_original_url = -1; *field_num_checksum = -1; *field_num_record_id = -1; token = strtok_r (lineptr, CDX_FIELDSEP, &save_ptr); if (token != NULL && strcmp (token, "CDX") == 0) { int field_num = 0; while (token != NULL) { token = strtok_r (NULL, CDX_FIELDSEP, &save_ptr); if (token != NULL) { switch (token[0]) { case 'a': *field_num_original_url = field_num; break; case 'k': *field_num_checksum = field_num; break; case 'u': *field_num_record_id = field_num; break; } } field_num++; } } return *field_num_original_url != -1 && *field_num_checksum != -1 && *field_num_record_id != -1; } /* Parse the CDX record and add it to the warc_cdx_dedup_table hash table. */ static void warc_process_cdx_line (char *lineptr, int field_num_original_url, int field_num_checksum, int field_num_record_id) { char *original_url = NULL; char *checksum = NULL; char *record_id = NULL; char *token; char *save_ptr; int field_num = 0; /* Read this line to get the fields we need. */ token = strtok_r (lineptr, CDX_FIELDSEP, &save_ptr); while (token != NULL) { char **val; if (field_num == field_num_original_url) val = &original_url; else if (field_num == field_num_checksum) val = &checksum; else if (field_num == field_num_record_id) val = &record_id; else val = NULL; if (val != NULL) *val = strdup (token); token = strtok_r (NULL, CDX_FIELDSEP, &save_ptr); field_num++; } if (original_url != NULL && checksum != NULL && record_id != NULL) { /* For some extra efficiency, we decode the base32 encoded checksum value. This should produce exactly SHA1_DIGEST_SIZE bytes. */ size_t checksum_l; char * checksum_v; base32_decode_alloc (checksum, strlen (checksum), &checksum_v, &checksum_l); xfree (checksum); if (checksum_v != NULL && checksum_l == SHA1_DIGEST_SIZE) { /* This is a valid line with a valid checksum. */ struct warc_cdx_record *rec; rec = xmalloc (sizeof (struct warc_cdx_record)); rec->url = original_url; rec->uuid = record_id; memcpy (rec->digest, checksum_v, SHA1_DIGEST_SIZE); hash_table_put (warc_cdx_dedup_table, rec->digest, rec); xfree (checksum_v); } else { xfree (original_url); xfree (checksum_v); xfree (record_id); } } else { xfree(checksum); xfree(original_url); xfree(record_id); } } /* Loads the CDX file from opt.warc_cdx_dedup_filename and fills the warc_cdx_dedup_table. */ static bool warc_load_cdx_dedup_file (void) { FILE *f; char *lineptr = NULL; size_t n = 0; ssize_t line_length; int field_num_original_url = -1; int field_num_checksum = -1; int field_num_record_id = -1; f = fopen (opt.warc_cdx_dedup_filename, "r"); if (f == NULL) return false; /* The first line should contain the CDX header. Format: " CDX x x x x x" where x are field type indicators. For our purposes, we only need 'a' (the original url), 'k' (the SHA1 checksum) and 'u' (the WARC record id). */ line_length = getline (&lineptr, &n, f); if (line_length != -1) warc_parse_cdx_header (lineptr, &field_num_original_url, &field_num_checksum, &field_num_record_id); /* If the file contains all three fields, read the complete file. */ if (field_num_original_url == -1 || field_num_checksum == -1 || field_num_record_id == -1) { if (field_num_original_url == -1) logprintf (LOG_NOTQUIET, _("CDX file does not list original urls. (Missing column 'a'.)\n")); if (field_num_checksum == -1) logprintf (LOG_NOTQUIET, _("CDX file does not list checksums. (Missing column 'k'.)\n")); if (field_num_record_id == -1) logprintf (LOG_NOTQUIET, _("CDX file does not list record ids. (Missing column 'u'.)\n")); } else { int nrecords; /* Initialize the table. */ warc_cdx_dedup_table = hash_table_new (1000, warc_hash_sha1_digest, warc_cmp_sha1_digest); do { line_length = getline (&lineptr, &n, f); if (line_length != -1) { warc_process_cdx_line (lineptr, field_num_original_url, field_num_checksum, field_num_record_id); } } while (line_length != -1); /* Print results. */ nrecords = hash_table_count (warc_cdx_dedup_table); logprintf (LOG_VERBOSE, ngettext ("Loaded %d record from CDX.\n\n", "Loaded %d records from CDX.\n\n", nrecords), nrecords); } xfree (lineptr); fclose (f); return true; } #undef CDX_FIELDSEP /* Returns the existing duplicate CDX record for the given url and payload digest. Returns NULL if the url is not found or if the payload digest does not match, or if CDX deduplication is disabled. */ static struct warc_cdx_record * warc_find_duplicate_cdx_record (const char *url, char *sha1_digest_payload) { struct warc_cdx_record *rec_existing; if (warc_cdx_dedup_table == NULL) return NULL; rec_existing = hash_table_get (warc_cdx_dedup_table, sha1_digest_payload); if (rec_existing && strcmp (rec_existing->url, url) == 0) return rec_existing; else return NULL; } /* Initializes the WARC writer (if opt.warc_filename is set). This should be called before any WARC record is written. */ void warc_init (void) { warc_write_ok = true; if (opt.warc_filename != NULL) { if (opt.warc_cdx_dedup_filename != NULL) { if (! warc_load_cdx_dedup_file ()) { logprintf (LOG_NOTQUIET, _("Could not read CDX file %s for deduplication.\n"), quote (opt.warc_cdx_dedup_filename)); exit (WGET_EXIT_GENERIC_ERROR); } } warc_manifest_fp = warc_tempfile (); if (warc_manifest_fp == NULL) { logprintf (LOG_NOTQUIET, _("Could not open temporary WARC manifest file.\n")); exit (WGET_EXIT_GENERIC_ERROR); } if (opt.warc_keep_log) { warc_log_fp = warc_tempfile (); if (warc_log_fp == NULL) { logprintf (LOG_NOTQUIET, _("Could not open temporary WARC log file.\n")); exit (WGET_EXIT_GENERIC_ERROR); } log_set_warc_log_fp (warc_log_fp); } warc_current_file_number = -1; if (! warc_start_new_file (false)) { logprintf (LOG_NOTQUIET, _("Could not open WARC file.\n")); exit (WGET_EXIT_GENERIC_ERROR); } if (opt.warc_cdx_enabled) { if (! warc_start_cdx_file ()) { logprintf (LOG_NOTQUIET, _("Could not open CDX file for output.\n")); exit (WGET_EXIT_GENERIC_ERROR); } } } } /* Writes metadata (manifest, configuration, log file) to the WARC file. */ static void warc_write_metadata (void) { char manifest_uuid[48]; FILE *warc_tmp_fp; /* If there are multiple WARC files, the metadata should be written to a separate file. */ if (opt.warc_maxsize > 0) warc_start_new_file (true); warc_uuid_str (manifest_uuid, sizeof (manifest_uuid)); fflush (warc_manifest_fp); warc_write_metadata_record (manifest_uuid, "metadata://gnu.org/software/wget/warc/MANIFEST.txt", NULL, NULL, NULL, "text/plain", warc_manifest_fp, -1); /* warc_write_resource_record has closed warc_manifest_fp. */ warc_tmp_fp = warc_tempfile (); if (warc_tmp_fp == NULL) { logprintf (LOG_NOTQUIET, _("Could not open temporary WARC file.\n")); exit (WGET_EXIT_GENERIC_ERROR); } fflush (warc_tmp_fp); fprintf (warc_tmp_fp, "%s\n", program_argstring); warc_write_resource_record (NULL, "metadata://gnu.org/software/wget/warc/wget_arguments.txt", NULL, manifest_uuid, NULL, "text/plain", warc_tmp_fp, -1); /* warc_write_resource_record has closed warc_tmp_fp. */ if (warc_log_fp != NULL) { warc_write_resource_record (NULL, "metadata://gnu.org/software/wget/warc/wget.log", NULL, manifest_uuid, NULL, "text/plain", warc_log_fp, -1); /* warc_write_resource_record has closed warc_log_fp. */ warc_log_fp = NULL; log_set_warc_log_fp (NULL); } } /* Finishes the WARC writing. This should be called at the end of the program. */ void warc_close (void) { if (warc_current_file != NULL) { warc_write_metadata (); *warc_current_warcinfo_uuid_str = 0; fclose (warc_current_file); warc_current_file = NULL; } if (warc_current_cdx_file != NULL) { fclose (warc_current_cdx_file); warc_current_cdx_file = NULL; } if (warc_log_fp != NULL) { fclose (warc_log_fp); log_set_warc_log_fp (NULL); } } /* Creates a temporary file for writing WARC output. The temporary file will be created in opt.warc_tempdir. Returns the pointer to the temporary file, or NULL. */ FILE * warc_tempfile (void) { char filename[100]; int fd; if (path_search (filename, 100, opt.warc_tempdir, "wget", true) == -1) return NULL; #ifdef __VMS /* 2013-07-12 SMS. * mkostemp()+unlink()+fdopen() scheme causes trouble on VMS, so use * mktemp() to uniquify the (VMS-style) name, and then use a normal * fopen() with a "create temp file marked for delete" option. */ { char *tfn; tfn = mktemp (filename); /* Get unique name from template. */ if (tfn == NULL) return NULL; return fopen (tfn, "w+", "fop=tmd"); /* Create auto-delete temp file. */ } #else /* def __VMS */ fd = mkostemp (filename, O_TEMPORARY); if (fd < 0) return NULL; #if !O_TEMPORARY if (unlink (filename) < 0) { close(fd); return NULL; } #endif return fdopen (fd, "wb+"); #endif /* def __VMS [else] */ } /* Writes a request record to the WARC file. url is the target uri of the request, timestamp_str is the timestamp of the request (generated with warc_timestamp), record_uuid is the uuid of the request (generated with warc_uuid_str), body is a pointer to a file containing the request headers and body. ip is the ip address of the server (or NULL), Calling this function will close body. Returns true on success, false on error. */ bool warc_write_request_record (const char *url, const char *timestamp_str, const char *record_uuid, const ip_address *ip, FILE *body, off_t payload_offset) { warc_write_start_record (); warc_write_header ("WARC-Type", "request"); warc_write_header_uri ("WARC-Target-URI", url); warc_write_header ("Content-Type", "application/http;msgtype=request"); warc_write_date_header (timestamp_str); warc_write_header ("WARC-Record-ID", record_uuid); warc_write_ip_header (ip); warc_write_header ("WARC-Warcinfo-ID", warc_current_warcinfo_uuid_str); warc_write_digest_headers (body, payload_offset); warc_write_block_from_file (body); warc_write_end_record (); fclose (body); return warc_write_ok; } /* Writes a response record to the CDX file. url is the target uri of the request/response, timestamp_str is the timestamp of the request that generated this response, (generated with warc_timestamp), mime_type is the mime type of the response body (will be printed to CDX), response_code is the HTTP response code (will be printed to CDX), payload_digest is the sha1 digest of the payload, redirect_location is the contents of the Location: header, or NULL (will be printed to CDX), offset is the position of the WARC record in the WARC file, warc_filename is the filename of the WARC, response_uuid is the uuid of the response. Returns true on success, false on error. */ static bool warc_write_cdx_record (const char *url, const char *timestamp_str, const char *mime_type, int response_code, const char *payload_digest, const char *redirect_location, off_t offset, const char *warc_filename _GL_UNUSED, const char *response_uuid) { /* Transform the timestamp. */ char timestamp_str_cdx[15]; char offset_string[MAX_INT_TO_STRING_LEN(off_t)]; const char *checksum; char *tmp_location = NULL; memcpy (timestamp_str_cdx , timestamp_str , 4); /* "YYYY" "-" */ memcpy (timestamp_str_cdx + 4, timestamp_str + 5, 2); /* "mm" "-" */ memcpy (timestamp_str_cdx + 6, timestamp_str + 8, 2); /* "dd" "T" */ memcpy (timestamp_str_cdx + 8, timestamp_str + 11, 2); /* "HH" ":" */ memcpy (timestamp_str_cdx + 10, timestamp_str + 14, 2); /* "MM" ":" */ memcpy (timestamp_str_cdx + 12, timestamp_str + 17, 2); /* "SS" "Z" */ timestamp_str_cdx[14] = '\0'; /* Rewrite the checksum. */ if (payload_digest != NULL) checksum = payload_digest + 5; /* Skip the "sha1:" */ else checksum = "-"; if (mime_type == NULL || strlen(mime_type) == 0) mime_type = "-"; if (redirect_location == NULL || strlen(redirect_location) == 0) tmp_location = strdup ("-"); else tmp_location = url_escape(redirect_location); number_to_string (offset_string, offset); /* Print the CDX line. */ fprintf (warc_current_cdx_file, "%s %s %s %s %d %s %s - %s %s %s\n", url, timestamp_str_cdx, url, mime_type, response_code, checksum, tmp_location, offset_string, warc_current_filename, response_uuid); fflush (warc_current_cdx_file); free (tmp_location); return true; } /* Writes a revisit record to the WARC file. url is the target uri of the request/response, timestamp_str is the timestamp of the request that generated this response (generated with warc_timestamp), concurrent_to_uuid is the uuid of the request for that generated this response (generated with warc_uuid_str), refers_to_uuid is the uuid of the original response (generated with warc_uuid_str), payload_digest is the sha1 digest of the payload, ip is the ip address of the server (or NULL), body is a pointer to a file containing the response headers (without payload). Calling this function will close body. Returns true on success, false on error. */ static bool warc_write_revisit_record (const char *url, const char *timestamp_str, const char *concurrent_to_uuid, const char *payload_digest, const char *refers_to, const ip_address *ip, FILE *body) { char revisit_uuid [48]; char block_digest[BASE32_LENGTH(SHA1_DIGEST_SIZE) + 1 + 5]; char sha1_res_block[SHA1_DIGEST_SIZE]; warc_uuid_str (revisit_uuid, sizeof (revisit_uuid)); sha1_stream (body, sha1_res_block); warc_base32_sha1_digest (sha1_res_block, block_digest, sizeof(block_digest)); warc_write_start_record (); warc_write_header ("WARC-Type", "revisit"); warc_write_header ("WARC-Record-ID", revisit_uuid); warc_write_header ("WARC-Warcinfo-ID", warc_current_warcinfo_uuid_str); warc_write_header ("WARC-Concurrent-To", concurrent_to_uuid); warc_write_header ("WARC-Refers-To", refers_to); warc_write_header ("WARC-Profile", "http://netpreserve.org/warc/1.0/revisit/identical-payload-digest"); warc_write_header ("WARC-Truncated", "length"); warc_write_header_uri ("WARC-Target-URI", url); warc_write_date_header (timestamp_str); warc_write_ip_header (ip); warc_write_header ("Content-Type", "application/http;msgtype=response"); warc_write_header ("WARC-Block-Digest", block_digest); warc_write_header ("WARC-Payload-Digest", payload_digest); warc_write_block_from_file (body); warc_write_end_record (); fclose (body); return warc_write_ok; } /* Writes a response record to the WARC file. url is the target uri of the request/response, timestamp_str is the timestamp of the request that generated this response (generated with warc_timestamp), concurrent_to_uuid is the uuid of the request for that generated this response (generated with warc_uuid_str), ip is the ip address of the server (or NULL), body is a pointer to a file containing the response headers and body. mime_type is the mime type of the response body (will be printed to CDX), response_code is the HTTP response code (will be printed to CDX), redirect_location is the contents of the Location: header, or NULL (will be printed to CDX), Calling this function will close body. Returns true on success, false on error. */ bool warc_write_response_record (const char *url, const char *timestamp_str, const char *concurrent_to_uuid, const ip_address *ip, FILE *body, off_t payload_offset, const char *mime_type, int response_code, const char *redirect_location) { char block_digest[BASE32_LENGTH(SHA1_DIGEST_SIZE) + 1 + 5]; char payload_digest[BASE32_LENGTH(SHA1_DIGEST_SIZE) + 1 + 5]; char sha1_res_block[SHA1_DIGEST_SIZE]; char sha1_res_payload[SHA1_DIGEST_SIZE]; char response_uuid [48]; off_t offset; if (opt.warc_digests_enabled) { /* Calculate the block and payload digests. */ rewind (body); if (warc_sha1_stream_with_payload (body, sha1_res_block, sha1_res_payload, payload_offset) == 0) { /* Decide (based on url + payload digest) if we have seen this data before. */ struct warc_cdx_record *rec_existing; rec_existing = warc_find_duplicate_cdx_record (url, sha1_res_payload); if (rec_existing != NULL) { bool result; /* Found an existing record. */ logprintf (LOG_VERBOSE, _("Found exact match in CDX file. Saving revisit record to WARC.\n")); /* Remove the payload from the file. */ if (payload_offset > 0) { if (ftruncate (fileno (body), payload_offset) == -1) return false; } /* Send the original payload digest. */ warc_base32_sha1_digest (sha1_res_payload, payload_digest, sizeof(payload_digest)); result = warc_write_revisit_record (url, timestamp_str, concurrent_to_uuid, payload_digest, rec_existing->uuid, ip, body); return result; } warc_base32_sha1_digest (sha1_res_block, block_digest, sizeof(block_digest)); warc_base32_sha1_digest (sha1_res_payload, payload_digest, sizeof(payload_digest)); } } /* Not a revisit, just store the record. */ warc_uuid_str (response_uuid, sizeof (response_uuid)); fseeko (warc_current_file, 0L, SEEK_END); offset = ftello (warc_current_file); warc_write_start_record (); warc_write_header ("WARC-Type", "response"); warc_write_header ("WARC-Record-ID", response_uuid); warc_write_header ("WARC-Warcinfo-ID", warc_current_warcinfo_uuid_str); warc_write_header ("WARC-Concurrent-To", concurrent_to_uuid); warc_write_header_uri ("WARC-Target-URI", url); warc_write_date_header (timestamp_str); warc_write_ip_header (ip); warc_write_header ("WARC-Block-Digest", block_digest); warc_write_header ("WARC-Payload-Digest", payload_digest); warc_write_header ("Content-Type", "application/http;msgtype=response"); warc_write_block_from_file (body); warc_write_end_record (); fclose (body); if (warc_write_ok && opt.warc_cdx_enabled) { /* Add this record to the CDX. */ warc_write_cdx_record (url, timestamp_str, mime_type, response_code, payload_digest, redirect_location, offset, warc_current_filename, response_uuid); } return warc_write_ok; } /* Writes a resource or metadata record to the WARC file. warc_type is either "resource" or "metadata", resource_uuid is the uuid of the resource (or NULL), url is the target uri of the resource, timestamp_str is the timestamp (generated with warc_timestamp), concurrent_to_uuid is the uuid of the record that generated this, resource (generated with warc_uuid_str) or NULL, ip is the ip address of the server (or NULL), content_type is the mime type of the body (or NULL), body is a pointer to a file containing the resource data. Calling this function will close body. Returns true on success, false on error. */ static bool warc_write_record (const char *record_type, const char *resource_uuid, const char *url, const char *timestamp_str, const char *concurrent_to_uuid, const ip_address *ip, const char *content_type, FILE *body, off_t payload_offset) { char uuid_buf[48]; if (resource_uuid == NULL) { warc_uuid_str (uuid_buf, sizeof (uuid_buf)); resource_uuid = uuid_buf; } if (content_type == NULL) content_type = "application/octet-stream"; warc_write_start_record (); warc_write_header ("WARC-Type", record_type); warc_write_header ("WARC-Record-ID", resource_uuid); warc_write_header ("WARC-Warcinfo-ID", warc_current_warcinfo_uuid_str); warc_write_header ("WARC-Concurrent-To", concurrent_to_uuid); warc_write_header_uri ("WARC-Target-URI", url); warc_write_date_header (timestamp_str); warc_write_ip_header (ip); warc_write_digest_headers (body, payload_offset); warc_write_header ("Content-Type", content_type); warc_write_block_from_file (body); warc_write_end_record (); fclose (body); return warc_write_ok; } /* Writes a resource record to the WARC file. resource_uuid is the uuid of the resource (or NULL), url is the target uri of the resource, timestamp_str is the timestamp (generated with warc_timestamp), concurrent_to_uuid is the uuid of the record that generated this, resource (generated with warc_uuid_str) or NULL, ip is the ip address of the server (or NULL), content_type is the mime type of the body (or NULL), body is a pointer to a file containing the resource data. Calling this function will close body. Returns true on success, false on error. */ bool warc_write_resource_record (const char *resource_uuid, const char *url, const char *timestamp_str, const char *concurrent_to_uuid, const ip_address *ip, const char *content_type, FILE *body, off_t payload_offset) { return warc_write_record ("resource", resource_uuid, url, timestamp_str, concurrent_to_uuid, ip, content_type, body, payload_offset); } /* Writes a metadata record to the WARC file. record_uuid is the uuid of the record (or NULL), url is the target uri of the record, timestamp_str is the timestamp (generated with warc_timestamp), concurrent_to_uuid is the uuid of the record that generated this, record (generated with warc_uuid_str) or NULL, ip is the ip address of the server (or NULL), content_type is the mime type of the body (or NULL), body is a pointer to a file containing the record data. Calling this function will close body. Returns true on success, false on error. */ bool warc_write_metadata_record (const char *record_uuid, const char *url, const char *timestamp_str, const char *concurrent_to_uuid, ip_address *ip, const char *content_type, FILE *body, off_t payload_offset) { return warc_write_record ("metadata", record_uuid, url, timestamp_str, concurrent_to_uuid, ip, content_type, body, payload_offset); } wget-1.21.2/src/ftp.h0000644000000000000000000001250614115732710011170 00000000000000/* Declarations for FTP support. Copyright (C) 1996-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #ifndef FTP_H #define FTP_H #include #include #include "host.h" #include "url.h" /* System types. */ enum stype { ST_UNIX, ST_VMS, ST_WINNT, ST_MACOS, ST_OS400, ST_OTHER }; /* Extensions of the ST_UNIX */ enum ustype { UST_TYPE_L8, UST_MULTINET, UST_OTHER }; #ifdef HAVE_SSL /* Data channel protection levels (to be used with PBSZ) */ enum prot_level { PROT_CLEAR = 'C', PROT_SAFE = 'S', PROT_CONFIDENTIAL = 'E', PROT_PRIVATE = 'P' }; #endif uerr_t ftp_response (int, char **); uerr_t ftp_greeting (int); uerr_t ftp_login (int, const char *, const char *); uerr_t ftp_port (int, int *); uerr_t ftp_pasv (int, ip_address *, int *); #ifdef HAVE_SSL uerr_t ftp_auth (int, enum url_scheme); uerr_t ftp_pbsz (int, int); uerr_t ftp_prot (int, enum prot_level); #endif #ifdef ENABLE_IPV6 uerr_t ftp_lprt (int, int *); uerr_t ftp_lpsv (int, ip_address *, int *); uerr_t ftp_eprt (int, int *); uerr_t ftp_epsv (int, ip_address *, int *); #endif uerr_t ftp_type (int, int); uerr_t ftp_cwd (int, const char *); uerr_t ftp_retr (int, const char *); uerr_t ftp_rest (int, wgint); uerr_t ftp_list (int, const char *, bool, bool, bool *); uerr_t ftp_syst (int, enum stype *, enum ustype *); uerr_t ftp_pwd (int, char **); uerr_t ftp_size (int, const char *, wgint *); #ifdef ENABLE_OPIE const char *skey_response (int, const char *, const char *); #endif struct url; /* File types. */ enum ftype { FT_PLAINFILE, FT_DIRECTORY, FT_SYMLINK, FT_UNKNOWN }; /* Globbing (used by ftp_retrieve_glob). */ enum { GLOB_GLOBALL, GLOB_GETALL, GLOB_GETONE }; /* Used by to test if time parsed includes hours and minutes. */ enum parsetype { TT_HOUR_MIN, TT_DAY }; /* Information about one filename in a linked list. */ struct fileinfo { enum ftype type; /* file type */ char *name; /* file name */ wgint size; /* file size */ long tstamp; /* time-stamp */ enum parsetype ptype; /* time parsing */ int perms; /* file permissions */ char *linkto; /* link to which file points */ struct fileinfo *prev; /* previous... */ struct fileinfo *next; /* ...and next structure. */ }; /* Commands for FTP functions. */ enum wget_ftp_command { DO_LOGIN = 0x0001, /* Connect and login to the server. */ DO_CWD = 0x0002, /* Change current directory. */ DO_RETR = 0x0004, /* Retrieve the file. */ DO_LIST = 0x0008, /* Retrieve the directory list. */ LEAVE_PENDING = 0x0010 /* Do not close the socket. */ }; enum wget_ftp_fstatus { NOTHING = 0x0000, /* Nothing done yet. */ ON_YOUR_OWN = 0x0001, /* The ftp_loop_internal sets the defaults. */ DONE_CWD = 0x0002, /* The current working directory is correct. */ /* 2013-10-17 Andrea Urbani (matfanjol) For more information about the following entries, please, look at ftp.c, function getftp, text "__LIST_A_EXPLANATION__". */ AVOID_LIST_A = 0x0004, /* It tells us if during this session we have to avoid the use of "LIST -a".*/ AVOID_LIST = 0x0008, /* It tells us if during this session we have to avoid to use "LIST". */ LIST_AFTER_LIST_A_CHECK_DONE = 0x0010, /* It tells us if we have already checked "LIST" after the first "LIST -a" to handle the case of file/folders named "-a". */ DATA_CHANNEL_SECURITY = 0x0020 /* Establish a secure data channel */ }; struct fileinfo *ftp_parse_ls (const char *, const enum stype); struct fileinfo *ftp_parse_ls_fp (FILE *, const enum stype); void freefileinfo(struct fileinfo *); uerr_t ftp_loop (struct url *, struct url *, char **, int *, struct url *, bool, bool); uerr_t ftp_index (const char *, struct url *, struct fileinfo *); char ftp_process_type (const char *); #endif /* FTP_H */ wget-1.21.2/src/netrc.c0000644000000000000000000003406514115732710011511 00000000000000/* Read and parse the .netrc file to get hosts, accounts, and passwords. Copyright (C) 1996, 2007-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ /* This file used to be kept in synch with the code in Fetchmail, but the latter has diverged since. */ #include "wget.h" #include #include #include #include #include "utils.h" #include "netrc.h" #include "init.h" #ifdef WINDOWS # define NETRC_FILE_NAME "_netrc" #else # define NETRC_FILE_NAME ".netrc" #endif typedef struct _acc_t { char *host; /* NULL if this is the default machine entry. */ char *acc; char *passwd; /* NULL if there is no password. */ struct _acc_t *next; } acc_t; static acc_t *parse_netrc (const char *); static acc_t *parse_netrc_fp (const char *, FILE *); static acc_t *netrc_list; static int processed_netrc; #if defined DEBUG_MALLOC || defined TESTING static void free_netrc(acc_t *); void netrc_cleanup (void) { free_netrc (netrc_list); processed_netrc = 0; } #endif /* Return the correct user and password, given the host, user (as given in the URL), and password (as given in the URL). May return NULL. If SLACK_DEFAULT is set, allow looking for a "default" account. You will typically turn it off for HTTP. */ void search_netrc (const char *host, const char **acc, const char **passwd, int slack_default, FILE *fp_netrc) { acc_t *l; if (!opt.netrc) return; /* Find ~/.netrc. */ if (!processed_netrc) { #ifdef __VMS int err; struct stat buf; char *path = "SYS$LOGIN:.netrc"; netrc_list = NULL; processed_netrc = 1; err = stat (path, &buf); if (err == 0) netrc_list = parse_netrc (path); #else /* def __VMS */ netrc_list = NULL; processed_netrc = 1; if (fp_netrc) netrc_list = parse_netrc_fp (".netrc", fp_netrc); else if (opt.homedir) { struct stat buf; char *path = aprintf ("%s/%s", opt.homedir, NETRC_FILE_NAME); if (stat (path, &buf) == 0) netrc_list = parse_netrc (path); xfree (path); } #endif /* def __VMS [else] */ } /* If nothing to do... */ if (!netrc_list) return; /* Acc and password found; all OK. */ if (*acc && *passwd) return; /* Some data not given -- try finding the host. */ for (l = netrc_list; l; l = l->next) { if (!l->host) continue; else if (!strcasecmp (l->host, host)) break; } if (l) { if (*acc) { /* Looking for password in .netrc. */ if (!strcmp (l->acc, *acc)) *passwd = l->passwd; /* usernames match; password OK */ else *passwd = NULL; /* usernames don't match */ } else /* NOT *acc */ { /* If password was given, use it. The account is l->acc. */ *acc = l->acc; if (l->passwd) *passwd = l->passwd; } return; } else { if (!slack_default) return; if (*acc) return; /* Try looking for the default account. */ for (l = netrc_list; l; l = l->next) if (!l->host) break; if (!l) return; *acc = l->acc; if (!*passwd) *passwd = l->passwd; return; } } #ifdef STANDALONE /* Normally, these functions would be defined by your package. */ # define xmalloc malloc # define xfree(p) do { free ((void *) (p)); p = NULL; } while (0) # define xstrdup strdup # define xrealloc realloc #endif /* STANDALONE */ /* Maybe add NEWENTRY to the account information list, LIST. NEWENTRY is set to a ready-to-use acc_t, in any event. */ static void maybe_add_to_list (acc_t **newentry, acc_t **list) { acc_t *a, *l; a = *newentry; l = *list; /* We need an account name in order to add the entry to the list. */ if (a && ! a->acc) { /* Free any allocated space. */ xfree (a->host); xfree (a->acc); xfree (a->passwd); } else { if (a) { /* Add the current machine into our list. */ a->next = l; l = a; } /* Allocate a new acc_t structure. */ a = xmalloc (sizeof (acc_t)); } /* Zero the structure, so that it is ready to use. */ memset (a, 0, sizeof(*a)); /* Return the new pointers. */ *newentry = a; *list = l; return; } /* Helper function for the parser, shifts contents of null-terminated string once character to the left. Used in processing \ and " constructs in the netrc file */ static void shift_left(char *string) { char *p; for (p=string; *p; ++p) *p = *(p+1); } /* Parse a .netrc file (as described in the ftp(1) manual page). */ static acc_t * parse_netrc_fp (const char *path, FILE *fp) { char *line = NULL, *p, *tok; const char *premature_token = NULL; acc_t *current = NULL, *retval = NULL; int ln = 0, qmark; size_t bufsize = 0; /* The latest token we've seen in the file. */ enum { tok_nothing, tok_account, tok_login, tok_macdef, tok_machine, tok_password, tok_port, tok_force } last_token = tok_nothing; /* While there are lines in the file... */ while (getline (&line, &bufsize, fp) > 0) { ln ++; /* Parse the line. */ p = line; qmark = 0; /* Skip leading whitespace. */ while (*p && c_isspace (*p)) p ++; /* If the line is empty, then end any macro definition. */ if (last_token == tok_macdef && !*p) /* End of macro if the line is empty. */ last_token = tok_nothing; /* If we are defining macros, then skip parsing the line. */ while (*p && last_token != tok_macdef) { /* Skip any whitespace. */ while (*p && c_isspace (*p)) p ++; /* Discard end-of-line comments; also, stop processing if the above `while' merely skipped trailing whitespace. */ if (*p == '#' || !*p) break; /* If the token starts with quotation mark, note this fact, and squash the quotation character */ if (*p == '"'){ qmark = 1; shift_left (p); } tok = p; /* Find the end of the token, handling quotes and escapes. */ while (*p && (qmark ? *p != '"' : !c_isspace (*p))){ if (*p == '\\') shift_left (p); p ++; } /* If field was quoted, squash the trailing quotation mark and reset qmark flag. */ if (qmark) { shift_left (p); qmark = 0; } /* Null-terminate the token, if it isn't already. */ if (*p) *p ++ = '\0'; switch (last_token) { case tok_login: if (current) { xfree (current->acc); current->acc = xstrdup (tok); } else premature_token = "login"; break; case tok_machine: /* Start a new machine entry. */ maybe_add_to_list (¤t, &retval); current->host = xstrdup (tok); break; case tok_password: if (current) { xfree (current->passwd); current->passwd = xstrdup (tok); } else premature_token = "password"; break; /* We handle most of tok_macdef above. */ case tok_macdef: if (!current) premature_token = "macdef"; break; /* We don't handle the account keyword at all. */ case tok_account: if (!current) premature_token = "account"; break; /* We don't handle the port keyword at all. */ case tok_port: if (!current) premature_token = "port"; break; /* We don't handle the force keyword at all. */ case tok_force: if (!current) premature_token = "force"; break; /* We handle tok_nothing below this switch. */ case tok_nothing: break; } if (premature_token) { fprintf (stderr, _("\ %s: %s:%d: warning: %s token appears before any machine name\n"), exec_name, path, ln, quote (premature_token)); premature_token = NULL; } if (last_token != tok_nothing) /* We got a value, so reset the token state. */ last_token = tok_nothing; else { /* Fetch the next token. */ if (!strcmp (tok, "account")) last_token = tok_account; else if (!strcmp (tok, "default")) maybe_add_to_list (¤t, &retval); else if (!strcmp (tok, "login")) last_token = tok_login; else if (!strcmp (tok, "macdef")) last_token = tok_macdef; else if (!strcmp (tok, "machine")) last_token = tok_machine; else if (!strcmp (tok, "password")) last_token = tok_password; /* GNU extensions 'port' and 'force', not operational * see https://www.gnu.org/software/emacs/manual/html_node/gnus/NNTP.html#index-nntp_002dauthinfo_002dfunction-2003 * see https://savannah.gnu.org/bugs/index.php?52066 */ else if (!strcmp (tok, "port")) last_token = tok_port; else if (!strcmp (tok, "force")) last_token = tok_force; else fprintf (stderr, _("%s: %s:%d: unknown token \"%s\"\n"), exec_name, path, ln, tok); } } } xfree (line); /* Finalize the last machine entry we found. */ maybe_add_to_list (¤t, &retval); xfree (current); /* Reverse the order of the list so that it appears in file order. */ current = retval; retval = NULL; while (current) { acc_t *saved_reference; /* Change the direction of the pointers. */ saved_reference = current->next; current->next = retval; /* Advance to the next node. */ retval = current; current = saved_reference; } return retval; } static acc_t * parse_netrc (const char *path) { FILE *fp; acc_t *acc; fp = fopen (path, "r"); if (!fp) { fprintf (stderr, _("%s: Cannot read %s (%s).\n"), exec_name, path, strerror (errno)); return NULL; } acc = parse_netrc_fp (path, fp); fclose(fp); return acc; } #if defined DEBUG_MALLOC || defined TESTING /* Free a netrc list. */ static void free_netrc(acc_t *l) { acc_t *t; while (l) { t = l->next; xfree (l->acc); xfree (l->passwd); xfree (l->host); xfree (l); l = t; } } #endif #ifdef STANDALONE #include #include #include "exits.h" const char *program_argstring = NULL; /* Needed by warc.c */ int main (int argc, char **argv) { struct stat sb; char *program_name, *file, *target; acc_t *head, *a; if (argc < 2 || argc > 3) { fprintf (stderr, _("Usage: %s NETRC [HOSTNAME]\n"), argv[0]); exit (WGET_EXIT_GENERIC_ERROR); } program_name = argv[0]; file = argv[1]; target = argv[2]; #ifdef ENABLE_NLS /* Set the current locale. */ setlocale (LC_ALL, ""); /* Set the text message domain. */ bindtextdomain ("wget", LOCALEDIR); textdomain ("wget"); #endif /* ENABLE_NLS */ if (stat (file, &sb)) { fprintf (stderr, _("%s: cannot stat %s: %s\n"), argv[0], file, strerror (errno)); exit (WGET_EXIT_GENERIC_ERROR); } head = parse_netrc (file); a = head; while (a) { /* Skip if we have a target and this isn't it. */ if (target && a->host && strcmp (target, a->host)) { a = a->next; continue; } if (!target) { /* Print the host name if we have no target. */ if (a->host) fputs (a->host, stdout); else fputs ("DEFAULT", stdout); fputc (' ', stdout); } /* Print the account name. */ fputs (a->acc, stdout); if (a->passwd) { /* Print the password, if there is any. */ fputc (' ', stdout); fputs (a->passwd, stdout); } fputc ('\n', stdout); /* Exit if we found the target. */ if (target) exit (WGET_EXIT_SUCCESS); a = a->next; } /* Exit with failure if we had a target, success otherwise. */ if (target) exit (WGET_EXIT_GENERIC_ERROR); exit (WGET_EXIT_SUCCESS); } #endif /* STANDALONE */ wget-1.21.2/src/build_info.c.in0000644000000000000000000000106514115732710013107 00000000000000digest defined ENABLE_DIGEST https defined HAVE_SSL ipv6 defined ENABLE_IPV6 iri defined ENABLE_IRI large-file SIZEOF_OFF_T >= 8 || defined WINDOWS nls defined ENABLE_NLS ntlm defined ENABLE_NTLM opie defined ENABLE_OPIE psl defined HAVE_LIBPSL cares defined HAVE_LIBCARES metalink defined HAVE_METALINK gpgme defined HAVE_GPGME ssl choice: openssl defined HAVE_LIBSSL || defined HAVE_LIBSSL32 gnutls defined HAVE_LIBGNUTLS wget-1.21.2/src/ftp-basic.c0000644000000000000000000007775314115732710012261 00000000000000/* Basic FTP routines. Copyright (C) 1996-2011, 2014-2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #include "wget.h" #include #include #include #include #include #include #include "utils.h" #include "connect.h" #include "host.h" #include "ftp.h" #include "retr.h" #include "c-strcase.h" /* Get the response of FTP server and allocate enough room to handle it. and characters are stripped from the line, and the line is 0-terminated. All the response lines but the last one are skipped. The last line is determined as described in RFC959. If the line is successfully read, FTPOK is returned, and *ret_line is assigned a freshly allocated line. Otherwise, FTPRERR is returned, and the value of *ret_line should be ignored. */ uerr_t ftp_response (int fd, char **ret_line) { for (;;) { char *p; char *line = fd_read_line (fd); if (!line) return FTPRERR; /* Strip trailing CRLF before printing the line, so that quoting doesn't include bogus \012 and \015. */ if ((p = strpbrk(line , "\r\n"))) *p = 0; if (opt.server_response) logprintf (LOG_NOTQUIET, "%s\n", quotearg_style (escape_quoting_style, line)); else DEBUGP (("%s\n", quotearg_style (escape_quoting_style, line))); /* The last line of output is the one that begins with "ddd ". */ if (c_isdigit (line[0]) && c_isdigit (line[1]) && c_isdigit (line[2]) && line[3] == ' ') { *ret_line = line; return FTPOK; } xfree (line); } } /* Returns the malloc-ed FTP request, ending with , printing it if printing is required. If VALUE is NULL, just use command. */ static char * ftp_request (const char *command, const char *value) { char *res; if (value) { char *defanged = NULL, buf[256]; /* Check for newlines in VALUE (possibly injected by the %0A URL escape) making the callers inadvertently send multiple FTP commands at once. Without this check an attacker could intentionally redirect to ftp://server/fakedir%0Acommand.../ and execute arbitrary FTP command on a remote FTP server. */ if (strpbrk (value, "\r\n")) { /* Copy VALUE to the stack and modify CR/LF to space. */ char *p; size_t len = strlen(value); if (len < sizeof (buf)) defanged = buf; else defanged = xmalloc (len + 1); memcpy (defanged, value, len + 1); for (p = defanged; *p; p++) if (*p == '\r' || *p == '\n') *p = ' '; DEBUGP (("\nDetected newlines in %s \"%s\"; changing to %s \"%s\"\n", command, quotearg_style (escape_quoting_style, value), command, quotearg_style (escape_quoting_style, defanged))); /* Make VALUE point to the defanged copy of the string. */ value = defanged; } res = concat_strings (command, " ", value, "\r\n", (char *) 0); if (defanged != buf) xfree (defanged); } else res = concat_strings (command, "\r\n", (char *) 0); if (opt.server_response) { /* Hack: don't print out password. */ if (strncmp (res, "PASS", 4) != 0) logprintf (LOG_ALWAYS, "--> %s\n", res); else logputs (LOG_ALWAYS, "--> PASS Turtle Power!\n\n"); } else DEBUGP (("\n--> %s\n", res)); return res; } uerr_t ftp_greeting (int csock) { uerr_t err = FTPOK; char *response = NULL; err = ftp_response (csock, &response); if (err != FTPOK) goto bail; if (*response != '2') err = FTPSRVERR; bail: if (response) xfree (response); return err; } /* Sends the USER and PASS commands to the server, to control connection socket csock. */ uerr_t ftp_login (int csock, const char *acc, const char *pass) { uerr_t err; char *request, *respline; int nwritten; /* Send USER username. */ request = ftp_request ("USER", acc); nwritten = fd_write (csock, request, strlen (request), -1); if (nwritten < 0) { xfree (request); return WRITEFAILED; } xfree (request); /* Get appropriate response. */ err = ftp_response (csock, &respline); if (err != FTPOK) return err; /* An unprobable possibility of logging without a password. */ if (*respline == '2') { xfree (respline); return FTPOK; } /* Else, only response 3 is appropriate. */ if (*respline != '3') { xfree (respline); return FTPLOGREFUSED; } #ifdef ENABLE_OPIE { static const char *skey_head[] = { "331 s/key ", "331 opiekey " }; size_t i; const char *seed = NULL; for (i = 0; i < countof (skey_head); i++) { int l = strlen (skey_head[i]); if (0 == c_strncasecmp (skey_head[i], respline, l)) { seed = respline + l; break; } } if (seed) { int skey_sequence = 0; /* Extract the sequence from SEED. */ for (; c_isdigit (*seed); seed++) skey_sequence = 10 * skey_sequence + *seed - '0'; if (*seed == ' ') ++seed; else { xfree (respline); return FTPLOGREFUSED; } /* Replace the password with the SKEY response to the challenge. */ pass = skey_response (skey_sequence, seed, pass); } } #endif /* ENABLE_OPIE */ xfree (respline); /* Send PASS password. */ request = ftp_request ("PASS", pass); nwritten = fd_write (csock, request, strlen (request), -1); if (nwritten < 0) { xfree (request); return WRITEFAILED; } xfree (request); /* Get appropriate response. */ err = ftp_response (csock, &respline); if (err != FTPOK) return err; if (*respline != '2') { xfree (respline); return FTPLOGINC; } xfree (respline); /* All OK. */ return FTPOK; } static void ip_address_to_port_repr (const ip_address *addr, int port, char *buf, size_t buflen) { unsigned char *ptr; assert (addr->family == AF_INET); /* buf must contain the argument of PORT (of the form a,b,c,d,e,f). */ assert (buflen >= 6 * 4); ptr = IP_INADDR_DATA (addr); snprintf (buf, buflen, "%d,%d,%d,%d,%d,%d", ptr[0], ptr[1], ptr[2], ptr[3], (port & 0xff00) >> 8, port & 0xff); buf[buflen - 1] = '\0'; } /* Bind a port and send the appropriate PORT command to the FTP server. Use acceptport after RETR, to get the socket of data connection. */ uerr_t ftp_port (int csock, int *local_sock) { uerr_t err; char *request, *respline; ip_address addr; int nwritten; int port; /* Must contain the argument of PORT (of the form a,b,c,d,e,f). */ char bytes[6 * 4 + 1]; /* Get the address of this side of the connection. */ if (!socket_ip_address (csock, &addr, ENDPOINT_LOCAL)) return FTPSYSERR; assert (addr.family == AF_INET); /* Setting port to 0 lets the system choose a free port. */ port = 0; /* Bind the port. */ *local_sock = bind_local (&addr, &port); if (*local_sock < 0) return FTPSYSERR; /* Construct the argument of PORT (of the form a,b,c,d,e,f). */ ip_address_to_port_repr (&addr, port, bytes, sizeof (bytes)); /* Send PORT request. */ request = ftp_request ("PORT", bytes); nwritten = fd_write (csock, request, strlen (request), -1); if (nwritten < 0) { xfree (request); fd_close (*local_sock); return WRITEFAILED; } xfree (request); /* Get appropriate response. */ err = ftp_response (csock, &respline); if (err != FTPOK) { fd_close (*local_sock); return err; } if (*respline != '2') { xfree (respline); fd_close (*local_sock); return FTPPORTERR; } xfree (respline); return FTPOK; } #ifdef ENABLE_IPV6 static void ip_address_to_lprt_repr (const ip_address *addr, int port, char *buf, size_t buflen) { unsigned char *ptr = IP_INADDR_DATA (addr); /* buf must contain the argument of LPRT (of the form af,n,h1,h2,...,hn,p1,p2). */ assert (buflen >= 21 * 4); /* Construct the argument of LPRT (of the form af,n,h1,h2,...,hn,p1,p2). */ switch (addr->family) { case AF_INET: snprintf (buf, buflen, "%d,%d,%d,%d,%d,%d,%d,%d,%d", 4, 4, ptr[0], ptr[1], ptr[2], ptr[3], 2, (port & 0xff00) >> 8, port & 0xff); break; case AF_INET6: snprintf (buf, buflen, "%d,%d,%d,%d,%d,%d,%d,%d,%d,%d,%d,%d,%d,%d,%d,%d,%d,%d,%d,%d,%d", 6, 16, ptr[0], ptr[1], ptr[2], ptr[3], ptr[4], ptr[5], ptr[6], ptr[7], ptr[8], ptr[9], ptr[10], ptr[11], ptr[12], ptr[13], ptr[14], ptr[15], 2, (port & 0xff00) >> 8, port & 0xff); break; default: abort (); } } /* Bind a port and send the appropriate PORT command to the FTP server. Use acceptport after RETR, to get the socket of data connection. */ uerr_t ftp_lprt (int csock, int *local_sock) { uerr_t err; char *request, *respline; ip_address addr; int nwritten; int port; /* Must contain the argument of LPRT (of the form af,n,h1,h2,...,hn,p1,p2). */ char bytes[21 * 4 + 1]; /* Get the address of this side of the connection. */ if (!socket_ip_address (csock, &addr, ENDPOINT_LOCAL)) return FTPSYSERR; assert (addr.family == AF_INET || addr.family == AF_INET6); /* Setting port to 0 lets the system choose a free port. */ port = 0; /* Bind the port. */ *local_sock = bind_local (&addr, &port); if (*local_sock < 0) return FTPSYSERR; /* Construct the argument of LPRT (of the form af,n,h1,h2,...,hn,p1,p2). */ ip_address_to_lprt_repr (&addr, port, bytes, sizeof (bytes)); /* Send PORT request. */ request = ftp_request ("LPRT", bytes); nwritten = fd_write (csock, request, strlen (request), -1); if (nwritten < 0) { xfree (request); fd_close (*local_sock); return WRITEFAILED; } xfree (request); /* Get appropriate response. */ err = ftp_response (csock, &respline); if (err != FTPOK) { fd_close (*local_sock); return err; } if (*respline != '2') { xfree (respline); fd_close (*local_sock); return FTPPORTERR; } xfree (respline); return FTPOK; } static void ip_address_to_eprt_repr (const ip_address *addr, int port, char *buf, size_t buflen) { int afnum; /* buf must contain the argument of EPRT (of the form |af|addr|port|). * 4 chars for the | separators, INET6_ADDRSTRLEN chars for addr * 1 char for af (1-2) and 5 chars for port (0-65535) */ assert (buflen >= 4 + INET6_ADDRSTRLEN + 1 + 5); /* Construct the argument of EPRT (of the form |af|addr|port|). */ afnum = (addr->family == AF_INET ? 1 : 2); snprintf (buf, buflen, "|%d|%s|%d|", afnum, print_address (addr), port); buf[buflen - 1] = '\0'; } /* Bind a port and send the appropriate PORT command to the FTP server. Use acceptport after RETR, to get the socket of data connection. */ uerr_t ftp_eprt (int csock, int *local_sock) { uerr_t err; char *request, *respline; ip_address addr; int nwritten; int port; /* Must contain the argument of EPRT (of the form |af|addr|port|). * 4 chars for the | separators, INET6_ADDRSTRLEN chars for addr * 1 char for af (1-2) and 5 chars for port (0-65535) */ char bytes[4 + INET6_ADDRSTRLEN + 1 + 5 + 1]; /* Get the address of this side of the connection. */ if (!socket_ip_address (csock, &addr, ENDPOINT_LOCAL)) return FTPSYSERR; /* Setting port to 0 lets the system choose a free port. */ port = 0; /* Bind the port. */ *local_sock = bind_local (&addr, &port); if (*local_sock < 0) return FTPSYSERR; /* Construct the argument of EPRT (of the form |af|addr|port|). */ ip_address_to_eprt_repr (&addr, port, bytes, sizeof (bytes)); /* Send PORT request. */ request = ftp_request ("EPRT", bytes); nwritten = fd_write (csock, request, strlen (request), -1); if (nwritten < 0) { xfree (request); fd_close (*local_sock); return WRITEFAILED; } xfree (request); /* Get appropriate response. */ err = ftp_response (csock, &respline); if (err != FTPOK) { fd_close (*local_sock); return err; } if (*respline != '2') { xfree (respline); fd_close (*local_sock); return FTPPORTERR; } xfree (respline); return FTPOK; } #endif #ifdef HAVE_SSL /* * The following three functions defined into this #ifdef block * wrap the extended FTP commands defined in RFC 2228 (FTP Security Extensions). * Currently, only FTPS is supported, so these functions are only compiled when SSL * support is available, because there's no point in using FTPS when there's no SSL. * Shall someone add new secure FTP protocols in the future, feel free to remove this * #ifdef, or add new constants to it. */ /* * Sends an AUTH command as defined by RFC 2228, * deriving its argument from the scheme. For example, if the provided scheme * is SCHEME_FTPS, the command sent will be "AUTH TLS". Currently, this is the only * scheme supported, so this function will return FTPNOAUTH when supplied a different * one. It will also return FTPNOAUTH if the target server does not support FTPS. */ uerr_t ftp_auth (int csock, enum url_scheme scheme) { uerr_t err = 0; int written = 0; char *request = NULL, *response = NULL; if (scheme == SCHEME_FTPS) { request = ftp_request ("AUTH", "TLS"); written = fd_write (csock, request, strlen (request), -1); if (written < 0) { err = WRITEFAILED; goto bail; } err = ftp_response (csock, &response); if (err != FTPOK) goto bail; if (*response != '2') err = FTPNOAUTH; } else err = FTPNOAUTH; bail: xfree (request); xfree (response); return err; } uerr_t ftp_pbsz (int csock, int pbsz) { uerr_t err = 0; int written = 0; char spbsz[5]; char *request = NULL, *response = NULL; snprintf (spbsz, 5, "%d", pbsz); request = ftp_request ("PBSZ", spbsz); written = fd_write (csock, request, strlen (request), -1); if (written < 0) { err = WRITEFAILED; goto bail; } err = ftp_response (csock, &response); if (err != FTPOK) goto bail; if (*response != '2') err = FTPNOPBSZ; bail: xfree (request); xfree (response); return err; } uerr_t ftp_prot (int csock, enum prot_level prot) { uerr_t err = 0; int written = 0; char *request = NULL, *response = NULL; /* value must be a single character value */ char value[2]; value[0] = prot; value[1] = '\0'; request = ftp_request ("PROT", value); written = fd_write (csock, request, strlen (request), -1); if (written < 0) { err = WRITEFAILED; goto bail; } err = ftp_response (csock, &response); if (err != FTPOK) goto bail; if (*response != '2') err = FTPNOPROT; bail: xfree (request); xfree (response); return err; } #endif /* HAVE_SSL */ /* Similar to ftp_port, but uses `PASV' to initiate the passive FTP transfer. Reads the response from server and parses it. Reads the host and port addresses and returns them. */ uerr_t ftp_pasv (int csock, ip_address *addr, int *port) { char *request, *respline, *s; int nwritten, i; uerr_t err; unsigned char tmp[6]; assert (addr != NULL); assert (port != NULL); xzero (*addr); /* Form the request. */ request = ftp_request ("PASV", NULL); /* And send it. */ nwritten = fd_write (csock, request, strlen (request), -1); if (nwritten < 0) { xfree (request); return WRITEFAILED; } xfree (request); /* Get the server response. */ err = ftp_response (csock, &respline); if (err != FTPOK) return err; if (*respline != '2') { xfree (respline); return FTPNOPASV; } /* Parse the request. */ s = respline; for (s += 4; *s && !c_isdigit (*s); s++) ; if (!*s) { xfree (respline); return FTPINVPASV; } for (i = 0; i < 6; i++) { tmp[i] = 0; for (; c_isdigit (*s); s++) tmp[i] = (*s - '0') + 10 * tmp[i]; if (*s == ',') s++; else if (i < 5) { /* When on the last number, anything can be a terminator. */ xfree (respline); return FTPINVPASV; } } xfree (respline); addr->family = AF_INET; memcpy (IP_INADDR_DATA (addr), tmp, 4); *port = ((tmp[4] << 8) & 0xff00) + tmp[5]; return FTPOK; } #ifdef ENABLE_IPV6 /* Similar to ftp_lprt, but uses `LPSV' to initiate the passive FTP transfer. Reads the response from server and parses it. Reads the host and port addresses and returns them. */ uerr_t ftp_lpsv (int csock, ip_address *addr, int *port) { char *request, *respline, *s; int nwritten, i, af, addrlen, portlen; uerr_t err; unsigned char tmp[16]; unsigned char tmpprt[2]; assert (addr != NULL); assert (port != NULL); xzero (*addr); /* Form the request. */ request = ftp_request ("LPSV", NULL); /* And send it. */ nwritten = fd_write (csock, request, strlen (request), -1); if (nwritten < 0) { xfree (request); return WRITEFAILED; } xfree (request); /* Get the server response. */ err = ftp_response (csock, &respline); if (err != FTPOK) return err; if (*respline != '2') { xfree (respline); return FTPNOPASV; } /* Parse the response. */ s = respline; for (s += 4; *s && !c_isdigit (*s); s++) ; if (!*s) { xfree (respline); return FTPINVPASV; } /* First, get the address family */ af = 0; for (; c_isdigit (*s); s++) af = (*s - '0') + 10 * af; if (af != 4 && af != 6) { xfree (respline); return FTPINVPASV; } if (!*s || *s++ != ',') { xfree (respline); return FTPINVPASV; } /* Then, get the address length */ addrlen = 0; for (; c_isdigit (*s); s++) addrlen = (*s - '0') + 10 * addrlen; if (!*s || *s++ != ',') { xfree (respline); return FTPINVPASV; } if (addrlen > 16) { xfree (respline); return FTPINVPASV; } if ((af == 4 && addrlen != 4) || (af == 6 && addrlen != 16)) { xfree (respline); return FTPINVPASV; } /* Now, we get the actual address */ for (i = 0; i < addrlen; i++) { tmp[i] = 0; for (; c_isdigit (*s); s++) tmp[i] = (*s - '0') + 10 * tmp[i]; if (*s == ',') s++; else { xfree (respline); return FTPINVPASV; } } /* Now, get the port length */ portlen = 0; for (; c_isdigit (*s); s++) portlen = (*s - '0') + 10 * portlen; if (!*s || *s++ != ',') { xfree (respline); return FTPINVPASV; } if (portlen > 2) { xfree (respline); return FTPINVPASV; } /* Finally, we get the port number */ tmpprt[0] = 0; for (; c_isdigit (*s); s++) tmpprt[0] = (*s - '0') + 10 * tmpprt[0]; if (!*s || *s++ != ',') { xfree (respline); return FTPINVPASV; } tmpprt[1] = 0; for (; c_isdigit (*s); s++) tmpprt[1] = (*s - '0') + 10 * tmpprt[1]; assert (s != NULL); if (af == 4) { addr->family = AF_INET; memcpy (IP_INADDR_DATA (addr), tmp, 4); *port = ((tmpprt[0] << 8) & 0xff00) + tmpprt[1]; DEBUGP (("lpsv addr is: %s\n", print_address(addr))); DEBUGP (("tmpprt[0] is: %d\n", tmpprt[0])); DEBUGP (("tmpprt[1] is: %d\n", tmpprt[1])); DEBUGP (("*port is: %d\n", *port)); } else { assert (af == 6); addr->family = AF_INET6; memcpy (IP_INADDR_DATA (addr), tmp, 16); *port = ((tmpprt[0] << 8) & 0xff00) + tmpprt[1]; DEBUGP (("lpsv addr is: %s\n", print_address(addr))); DEBUGP (("tmpprt[0] is: %d\n", tmpprt[0])); DEBUGP (("tmpprt[1] is: %d\n", tmpprt[1])); DEBUGP (("*port is: %d\n", *port)); } xfree (respline); return FTPOK; } /* Similar to ftp_eprt, but uses `EPSV' to initiate the passive FTP transfer. Reads the response from server and parses it. Reads the host and port addresses and returns them. */ uerr_t ftp_epsv (int csock, ip_address *ip, int *port) { char *request, *respline, *start, delim, *s; int nwritten, i; uerr_t err; int tport; assert (ip != NULL); assert (port != NULL); /* IP already contains the IP address of the control connection's peer, so we don't need to call socket_ip_address here. */ /* Form the request. */ /* EPSV 1 means that we ask for IPv4 and EPSV 2 means that we ask for IPv6. */ request = ftp_request ("EPSV", (ip->family == AF_INET ? "1" : "2")); /* And send it. */ nwritten = fd_write (csock, request, strlen (request), -1); if (nwritten < 0) { xfree (request); return WRITEFAILED; } xfree (request); /* Get the server response. */ err = ftp_response (csock, &respline); if (err != FTPOK) return err; if (*respline != '2') { xfree (respline); return FTPNOPASV; } assert (respline != NULL); DEBUGP(("respline is %s\n", respline)); /* Skip the useless stuff and get what's inside the parentheses */ start = strchr (respline, '('); if (start == NULL) { xfree (respline); return FTPINVPASV; } /* Skip the first two void fields */ s = start + 1; delim = *s++; if (delim < 33 || delim > 126) { xfree (respline); return FTPINVPASV; } for (i = 0; i < 2; i++) { if (*s++ != delim) { xfree (respline); return FTPINVPASV; } } /* Finally, get the port number */ for (tport = 0, i = 0; i < 5 && c_isdigit (*s); i++, s++) tport = (*s - '0') + 10 * tport; /* Make sure that the response terminates correctly */ if (*s++ != delim) { xfree (respline); return FTPINVPASV; } if (*s != ')') { xfree (respline); return FTPINVPASV; } *port = tport; xfree (respline); return FTPOK; } #endif /* Sends the TYPE request to the server. */ uerr_t ftp_type (int csock, int type) { char *request, *respline; int nwritten; uerr_t err; char stype[2]; /* Construct argument. */ stype[0] = type; stype[1] = 0; /* Send TYPE request. */ request = ftp_request ("TYPE", stype); nwritten = fd_write (csock, request, strlen (request), -1); if (nwritten < 0) { xfree (request); return WRITEFAILED; } xfree (request); /* Get appropriate response. */ err = ftp_response (csock, &respline); if (err != FTPOK) return err; if (*respline != '2') { xfree (respline); return FTPUNKNOWNTYPE; } xfree (respline); /* All OK. */ return FTPOK; } /* Changes the working directory by issuing a CWD command to the server. */ uerr_t ftp_cwd (int csock, const char *dir) { char *request, *respline; int nwritten; uerr_t err; /* Send CWD request. */ request = ftp_request ("CWD", dir); nwritten = fd_write (csock, request, strlen (request), -1); if (nwritten < 0) { xfree (request); return WRITEFAILED; } xfree (request); /* Get appropriate response. */ err = ftp_response (csock, &respline); if (err != FTPOK) return err; if (*respline == '5') { xfree (respline); return FTPNSFOD; } if (*respline != '2') { xfree (respline); return FTPRERR; } xfree (respline); /* All OK. */ return FTPOK; } /* Sends REST command to the FTP server. */ uerr_t ftp_rest (int csock, wgint offset) { char *request, *respline; int nwritten; uerr_t err; request = ftp_request ("REST", number_to_static_string (offset)); nwritten = fd_write (csock, request, strlen (request), -1); if (nwritten < 0) { xfree (request); return WRITEFAILED; } xfree (request); /* Get appropriate response. */ err = ftp_response (csock, &respline); if (err != FTPOK) return err; if (*respline != '3') { xfree (respline); return FTPRESTFAIL; } xfree (respline); /* All OK. */ return FTPOK; } /* Sends RETR command to the FTP server. */ uerr_t ftp_retr (int csock, const char *file) { char *request, *respline; int nwritten; uerr_t err; /* Send RETR request. */ request = ftp_request ("RETR", file); nwritten = fd_write (csock, request, strlen (request), -1); if (nwritten < 0) { xfree (request); return WRITEFAILED; } xfree (request); /* Get appropriate response. */ err = ftp_response (csock, &respline); if (err != FTPOK) return err; if (*respline == '5') { xfree (respline); return FTPNSFOD; } if (*respline != '1') { xfree (respline); return FTPRERR; } xfree (respline); /* All OK. */ return FTPOK; } /* Sends the LIST command to the server. If FILE is NULL, send just `LIST' (no space). */ uerr_t ftp_list (int csock, const char *file, bool avoid_list_a, bool avoid_list, bool *list_a_used) { char *request, *respline; int nwritten; uerr_t err; bool ok = false; size_t i = 0; /* 2013-10-12 Andrea Urbani (matfanjol) For more information about LIST and "LIST -a" please look at ftp.c, function getftp, text "__LIST_A_EXPLANATION__". If somebody changes the following commands, please, checks also the later "i" variable. */ static const char *list_commands[] = { "LIST -a", "LIST" }; *list_a_used = false; if (avoid_list_a) { i = countof (list_commands)- 1; DEBUGP (("(skipping \"LIST -a\")")); } do { /* Send request. */ request = ftp_request (list_commands[i], file); nwritten = fd_write (csock, request, strlen (request), -1); if (nwritten < 0) { xfree (request); return WRITEFAILED; } xfree (request); /* Get appropriate response. */ err = ftp_response (csock, &respline); if (err == FTPOK) { if (*respline == '5') { err = FTPNSFOD; } else if (*respline == '1') { err = FTPOK; ok = true; /* Which list command was used? */ *list_a_used = (i == 0); } else { err = FTPRERR; } xfree (respline); } ++i; if ((avoid_list) && (i == 1)) { /* I skip LIST */ ++i; DEBUGP (("(skipping \"LIST\")")); } } while (i < countof (list_commands) && !ok); return err; } /* Sends the SYST command to the server. */ uerr_t ftp_syst (int csock, enum stype *server_type, enum ustype *unix_type) { char *request, *respline; int nwritten; uerr_t err; char *ftp_last_respline; /* Send SYST request. */ request = ftp_request ("SYST", NULL); nwritten = fd_write (csock, request, strlen (request), -1); if (nwritten < 0) { xfree (request); return WRITEFAILED; } xfree (request); /* Get appropriate response. */ err = ftp_response (csock, &respline); if (err != FTPOK) return err; if (*respline == '5') { xfree (respline); return FTPSRVERR; } ftp_last_respline = strdup (respline); /* Skip the number (215, but 200 (!!!) in case of VMS) */ strtok (respline, " "); /* Which system type has been reported (we are interested just in the first word of the server response)? */ request = strtok (NULL, " "); *unix_type = UST_OTHER; if (request == NULL) *server_type = ST_OTHER; else if (!c_strcasecmp (request, "VMS")) *server_type = ST_VMS; else if (!c_strcasecmp (request, "UNIX")) { *server_type = ST_UNIX; /* 2013-10-17 Andrea Urbani (matfanjol) I check more in depth the system type */ if (!c_strncasecmp (ftp_last_respline, "215 UNIX Type: L8", 17)) *unix_type = UST_TYPE_L8; else if (!c_strncasecmp (ftp_last_respline, "215 UNIX MultiNet Unix Emulation V5.3(93)", 41)) *unix_type = UST_MULTINET; } else if (!c_strcasecmp (request, "WINDOWS_NT") || !c_strcasecmp (request, "WINDOWS2000")) *server_type = ST_WINNT; else if (!c_strcasecmp (request, "MACOS")) *server_type = ST_MACOS; else if (!c_strcasecmp (request, "OS/400")) *server_type = ST_OS400; else *server_type = ST_OTHER; xfree (ftp_last_respline); xfree (respline); /* All OK. */ return FTPOK; } /* Sends the PWD command to the server. */ uerr_t ftp_pwd (int csock, char **pwd) { char *request, *respline; int nwritten; uerr_t err; /* Send PWD request. */ request = ftp_request ("PWD", NULL); nwritten = fd_write (csock, request, strlen (request), -1); if (nwritten < 0) { xfree (request); return WRITEFAILED; } xfree (request); /* Get appropriate response. */ err = ftp_response (csock, &respline); if (err != FTPOK) return err; if (*respline == '5') { err: xfree (respline); return FTPSRVERR; } /* Skip the number (257), leading citation mark, trailing citation mark and everything following it. */ strtok (respline, "\""); request = strtok (NULL, "\""); if (!request) /* Treat the malformed response as an error, which the caller has to handle gracefully anyway. */ goto err; /* Has the `pwd' been already allocated? Free! */ xfree (*pwd); *pwd = xstrdup (request); xfree (respline); /* All OK. */ return FTPOK; } /* Sends the SIZE command to the server, and returns the value in 'size'. * If an error occurs, size is set to zero. */ uerr_t ftp_size (int csock, const char *file, wgint *size) { char *request, *respline; int nwritten; uerr_t err; /* Send PWD request. */ request = ftp_request ("SIZE", file); nwritten = fd_write (csock, request, strlen (request), -1); if (nwritten < 0) { xfree (request); *size = 0; return WRITEFAILED; } xfree (request); /* Get appropriate response. */ err = ftp_response (csock, &respline); if (err != FTPOK) { *size = 0; return err; } if (*respline == '5') { /* * Probably means SIZE isn't supported on this server. * Error is nonfatal since SIZE isn't in RFC 959 */ xfree (respline); *size = 0; return FTPOK; } errno = 0; *size = str_to_wgint (respline + 4, NULL, 10); if (errno) { /* * Couldn't parse the response for some reason. On the (few) * tests I've done, the response is 213 with nothing else - * maybe something a bit more resilient is necessary. It's not a * fatal error, however. */ xfree (respline); *size = 0; return FTPOK; } xfree (respline); /* All OK. */ return FTPOK; } /* If URL's params are of the form "type=X", return character X. Otherwise, return 'I' (the default type). */ char ftp_process_type (const char *params) { if (params && 0 == strncasecmp (params, "type=", 5) && params[5] != '\0') return c_toupper (params[5]); else return 'I'; } wget-1.21.2/src/init.c0000644000000000000000000017105314115732710011340 00000000000000/* Reading/parsing the initialization file. Copyright (C) 1996-2012, 2014-2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #include "wget.h" #include "exits.h" #include #include #include #include #include #include #include /* not all systems provide PATH_MAX in limits.h */ #ifndef PATH_MAX # include # ifndef PATH_MAX # define PATH_MAX MAXPATHLEN # endif #endif #include #ifdef HAVE_PWD_H # include #endif #include #include "utils.h" #include "init.h" #include "host.h" #include "netrc.h" #include "progress.h" #include "connect.h" /* for connect_cleanup */ #include "ssl.h" /* for ssl_cleanup */ #include "recur.h" /* for INFINITE_RECURSION */ #include "convert.h" /* for convert_cleanup */ #include "res.h" /* for res_cleanup */ #include "http.h" /* for http_cleanup */ #include "retr.h" /* for output_stream */ #include "warc.h" /* for warc_close */ #include "spider.h" /* for spider_cleanup */ #include "html-url.h" /* for cleanup_html_url */ #include "ptimer.h" /* for ptimer_destroy */ #include "c-strcase.h" #ifdef TESTING #include "../tests/unit-tests.h" #endif #define CMD_DECLARE(func) static bool func (const char *, const char *, void *) CMD_DECLARE (cmd_boolean); CMD_DECLARE (cmd_bytes); CMD_DECLARE (cmd_bytes_sum); #ifdef HAVE_SSL CMD_DECLARE (cmd_cert_type); #endif CMD_DECLARE (cmd_directory_vector); CMD_DECLARE (cmd_number); CMD_DECLARE (cmd_number_inf); CMD_DECLARE (cmd_string); CMD_DECLARE (cmd_string_uppercase); CMD_DECLARE (cmd_file); CMD_DECLARE (cmd_file_once); CMD_DECLARE (cmd_directory); CMD_DECLARE (cmd_time); CMD_DECLARE (cmd_vector); CMD_DECLARE (cmd_use_askpass); #ifdef HAVE_LIBZ CMD_DECLARE (cmd_spec_compression); #endif CMD_DECLARE (cmd_spec_dirstruct); CMD_DECLARE (cmd_spec_header); CMD_DECLARE (cmd_spec_warc_header); CMD_DECLARE (cmd_spec_htmlify); CMD_DECLARE (cmd_spec_mirror); CMD_DECLARE (cmd_spec_prefer_family); CMD_DECLARE (cmd_spec_progress); CMD_DECLARE (cmd_spec_progressdisp); CMD_DECLARE (cmd_spec_recursive); CMD_DECLARE (cmd_spec_regex_type); CMD_DECLARE (cmd_spec_restrict_file_names); CMD_DECLARE (cmd_spec_report_speed); #ifdef HAVE_SSL CMD_DECLARE (cmd_spec_secure_protocol); #endif CMD_DECLARE (cmd_spec_timeout); CMD_DECLARE (cmd_spec_useragent); CMD_DECLARE (cmd_spec_verbose); CMD_DECLARE (cmd_check_cert); /* List of recognized commands, each consisting of name, place and function. When adding a new command, simply add it to the list, but be sure to keep the list sorted alphabetically, as command_by_name's binary search depends on it. Also, be sure to add any entries that allocate memory (e.g. cmd_string and cmd_vector) to the cleanup() function below. */ static const struct { const char *name; void *place; bool (*action) (const char *, const char *, void *); } commands[] = { /* KEEP THIS LIST ALPHABETICALLY SORTED */ { "accept", &opt.accepts, cmd_vector }, { "acceptregex", &opt.acceptregex_s, cmd_string }, { "addhostdir", &opt.add_hostdir, cmd_boolean }, { "adjustextension", &opt.adjust_extension, cmd_boolean }, { "alwaysrest", &opt.always_rest, cmd_boolean }, /* deprecated */ { "askpassword", &opt.ask_passwd, cmd_boolean }, { "authnochallenge", &opt.auth_without_challenge, cmd_boolean }, { "background", &opt.background, cmd_boolean }, { "backupconverted", &opt.backup_converted, cmd_boolean }, { "backups", &opt.backups, cmd_number }, { "base", &opt.base_href, cmd_string }, { "bindaddress", &opt.bind_address, cmd_string }, #ifdef HAVE_LIBCARES { "binddnsaddress", &opt.bind_dns_address, cmd_string }, #endif { "bodydata", &opt.body_data, cmd_string }, { "bodyfile", &opt.body_file, cmd_string }, #ifdef HAVE_SSL { "cacertificate", &opt.ca_cert, cmd_file }, #endif { "cache", &opt.allow_cache, cmd_boolean }, #ifdef HAVE_SSL { "cadirectory", &opt.ca_directory, cmd_directory }, { "certificate", &opt.cert_file, cmd_file }, { "certificatetype", &opt.cert_type, cmd_cert_type }, { "checkcertificate", &opt.check_cert, cmd_check_cert }, #endif { "chooseconfig", &opt.choose_config, cmd_file }, #ifdef HAVE_SSL { "ciphers", &opt.tls_ciphers_string, cmd_string }, #endif #ifdef HAVE_LIBZ { "compression", &opt.compression, cmd_spec_compression }, #endif { "connecttimeout", &opt.connect_timeout, cmd_time }, { "contentdisposition", &opt.content_disposition, cmd_boolean }, { "contentonerror", &opt.content_on_error, cmd_boolean }, { "continue", &opt.always_rest, cmd_boolean }, { "convertfileonly", &opt.convert_file_only, cmd_boolean }, { "convertlinks", &opt.convert_links, cmd_boolean }, { "cookies", &opt.cookies, cmd_boolean }, #ifdef HAVE_SSL { "crlfile", &opt.crl_file, cmd_file_once }, #endif { "cutdirs", &opt.cut_dirs, cmd_number }, { "debug", &opt.debug, cmd_boolean }, { "defaultpage", &opt.default_page, cmd_string }, { "deleteafter", &opt.delete_after, cmd_boolean }, { "dirprefix", &opt.dir_prefix, cmd_directory }, { "dirstruct", NULL, cmd_spec_dirstruct }, { "dnscache", &opt.dns_cache, cmd_boolean }, #ifdef HAVE_LIBCARES { "dnsservers", &opt.dns_servers, cmd_string }, #endif { "dnstimeout", &opt.dns_timeout, cmd_time }, { "domains", &opt.domains, cmd_vector }, { "dotbytes", &opt.dot_bytes, cmd_bytes }, { "dotsinline", &opt.dots_in_line, cmd_number }, { "dotspacing", &opt.dot_spacing, cmd_number }, { "dotstyle", &opt.dot_style, cmd_string }, /* deprecated */ #ifdef HAVE_SSL { "egdfile", &opt.egd_file, cmd_file }, #endif { "excludedirectories", &opt.excludes, cmd_directory_vector }, { "excludedomains", &opt.exclude_domains, cmd_vector }, { "followftp", &opt.follow_ftp, cmd_boolean }, { "followtags", &opt.follow_tags, cmd_vector }, { "forcehtml", &opt.force_html, cmd_boolean }, { "ftppasswd", &opt.ftp_passwd, cmd_string }, /* deprecated */ { "ftppassword", &opt.ftp_passwd, cmd_string }, { "ftpproxy", &opt.ftp_proxy, cmd_string }, #ifdef HAVE_SSL { "ftpscleardataconnection", &opt.ftps_clear_data_connection, cmd_boolean }, { "ftpsfallbacktoftp", &opt.ftps_fallback_to_ftp, cmd_boolean }, { "ftpsimplicit", &opt.ftps_implicit, cmd_boolean }, { "ftpsresumessl", &opt.ftps_resume_ssl, cmd_boolean }, #endif #ifdef __VMS { "ftpstmlf", &opt.ftp_stmlf, cmd_boolean }, #endif /* def __VMS */ { "ftpuser", &opt.ftp_user, cmd_string }, { "glob", &opt.ftp_glob, cmd_boolean }, { "header", NULL, cmd_spec_header }, #ifdef HAVE_HSTS { "hsts", &opt.hsts, cmd_boolean }, { "hstsfile", &opt.hsts_file, cmd_file }, #endif { "htmlextension", &opt.adjust_extension, cmd_boolean }, /* deprecated */ { "htmlify", NULL, cmd_spec_htmlify }, { "httpkeepalive", &opt.http_keep_alive, cmd_boolean }, { "httppasswd", &opt.http_passwd, cmd_string }, /* deprecated */ { "httppassword", &opt.http_passwd, cmd_string }, { "httpproxy", &opt.http_proxy, cmd_string }, #ifdef HAVE_SSL { "httpsonly", &opt.https_only, cmd_boolean }, #endif { "httpsproxy", &opt.https_proxy, cmd_string }, { "httpuser", &opt.http_user, cmd_string }, { "ifmodifiedsince", &opt.if_modified_since, cmd_boolean }, { "ignorecase", &opt.ignore_case, cmd_boolean }, { "ignorelength", &opt.ignore_length, cmd_boolean }, { "ignoretags", &opt.ignore_tags, cmd_vector }, { "includedirectories", &opt.includes, cmd_directory_vector }, #ifdef ENABLE_IPV6 { "inet4only", &opt.ipv4_only, cmd_boolean }, { "inet6only", &opt.ipv6_only, cmd_boolean }, #endif { "input", &opt.input_filename, cmd_file }, #ifdef HAVE_METALINK { "inputmetalink", &opt.input_metalink, cmd_file }, #endif { "iri", &opt.enable_iri, cmd_boolean }, { "keepbadhash", &opt.keep_badhash, cmd_boolean }, { "keepsessioncookies", &opt.keep_session_cookies, cmd_boolean }, { "limitrate", &opt.limit_rate, cmd_bytes }, { "loadcookies", &opt.cookies_input, cmd_file }, { "localencoding", &opt.locale, cmd_string }, { "logfile", &opt.lfilename, cmd_file }, { "login", &opt.ftp_user, cmd_string },/* deprecated*/ { "maxredirect", &opt.max_redirect, cmd_number }, #ifdef HAVE_METALINK { "metalinkindex", &opt.metalink_index, cmd_number_inf }, { "metalinkoverhttp", &opt.metalink_over_http, cmd_boolean }, #endif { "method", &opt.method, cmd_string_uppercase }, { "mirror", NULL, cmd_spec_mirror }, { "netrc", &opt.netrc, cmd_boolean }, { "noclobber", &opt.noclobber, cmd_boolean }, { "noconfig", &opt.noconfig, cmd_boolean }, { "noparent", &opt.no_parent, cmd_boolean }, { "noproxy", &opt.no_proxy, cmd_vector }, { "numtries", &opt.ntry, cmd_number_inf },/* deprecated*/ { "outputdocument", &opt.output_document, cmd_file }, { "pagerequisites", &opt.page_requisites, cmd_boolean }, { "passiveftp", &opt.ftp_pasv, cmd_boolean }, { "passwd", &opt.ftp_passwd, cmd_string },/* deprecated*/ { "password", &opt.passwd, cmd_string }, #ifdef HAVE_SSL { "pinnedpubkey", &opt.pinnedpubkey, cmd_string }, #endif { "postdata", &opt.post_data, cmd_string }, { "postfile", &opt.post_file_name, cmd_file }, { "preferfamily", NULL, cmd_spec_prefer_family }, #ifdef HAVE_METALINK { "preferredlocation", &opt.preferred_location, cmd_string }, #endif { "preservepermissions", &opt.preserve_perm, cmd_boolean }, #ifdef HAVE_SSL { "privatekey", &opt.private_key, cmd_file }, { "privatekeytype", &opt.private_key_type, cmd_cert_type }, #endif { "progress", &opt.progress_type, cmd_spec_progress }, { "protocoldirectories", &opt.protocol_directories, cmd_boolean }, { "proxypasswd", &opt.proxy_passwd, cmd_string }, /* deprecated */ { "proxypassword", &opt.proxy_passwd, cmd_string }, { "proxyuser", &opt.proxy_user, cmd_string }, { "quiet", &opt.quiet, cmd_boolean }, { "quota", &opt.quota, cmd_bytes_sum }, #ifdef HAVE_SSL { "randomfile", &opt.random_file, cmd_file }, #endif { "randomwait", &opt.random_wait, cmd_boolean }, { "readtimeout", &opt.read_timeout, cmd_time }, { "reclevel", &opt.reclevel, cmd_number_inf }, { "recursive", NULL, cmd_spec_recursive }, { "referer", &opt.referer, cmd_string }, { "regextype", &opt.regex_type, cmd_spec_regex_type }, { "reject", &opt.rejects, cmd_vector }, { "rejectedlog", &opt.rejected_log, cmd_file }, { "rejectregex", &opt.rejectregex_s, cmd_string }, { "relativeonly", &opt.relative_only, cmd_boolean }, { "remoteencoding", &opt.encoding_remote, cmd_string }, { "removelisting", &opt.remove_listing, cmd_boolean }, { "reportspeed", &opt.report_bps, cmd_spec_report_speed}, { "restrictfilenames", NULL, cmd_spec_restrict_file_names }, { "retrsymlinks", &opt.retr_symlinks, cmd_boolean }, { "retryconnrefused", &opt.retry_connrefused, cmd_boolean }, { "retryonhosterror", &opt.retry_on_host_error, cmd_boolean }, { "retryonhttperror", &opt.retry_on_http_error, cmd_string }, { "robots", &opt.use_robots, cmd_boolean }, { "savecookies", &opt.cookies_output, cmd_file }, { "saveheaders", &opt.save_headers, cmd_boolean }, #ifdef HAVE_SSL { "secureprotocol", &opt.secure_protocol, cmd_spec_secure_protocol }, #endif { "serverresponse", &opt.server_response, cmd_boolean }, { "showalldnsentries", &opt.show_all_dns_entries, cmd_boolean }, { "showprogress", &opt.show_progress, cmd_spec_progressdisp }, { "spanhosts", &opt.spanhost, cmd_boolean }, { "spider", &opt.spider, cmd_boolean }, { "startpos", &opt.start_pos, cmd_bytes }, { "strictcomments", &opt.strict_comments, cmd_boolean }, { "timeout", NULL, cmd_spec_timeout }, { "timestamping", &opt.timestamping, cmd_boolean }, { "tries", &opt.ntry, cmd_number_inf }, { "trustservernames", &opt.trustservernames, cmd_boolean }, { "unlink", &opt.unlink_requested, cmd_boolean }, #ifndef __VMS { "useaskpass" , &opt.use_askpass, cmd_use_askpass }, #endif { "useproxy", &opt.use_proxy, cmd_boolean }, { "user", &opt.user, cmd_string }, { "useragent", NULL, cmd_spec_useragent }, { "useservertimestamps", &opt.useservertimestamps, cmd_boolean }, { "verbose", NULL, cmd_spec_verbose }, { "wait", &opt.wait, cmd_time }, { "waitretry", &opt.waitretry, cmd_time }, { "warccdx", &opt.warc_cdx_enabled, cmd_boolean }, { "warccdxdedup", &opt.warc_cdx_dedup_filename, cmd_file }, #ifdef HAVE_LIBZ { "warccompression", &opt.warc_compression_enabled, cmd_boolean }, #endif { "warcdigests", &opt.warc_digests_enabled, cmd_boolean }, { "warcfile", &opt.warc_filename, cmd_file }, { "warcheader", NULL, cmd_spec_warc_header }, { "warckeeplog", &opt.warc_keep_log, cmd_boolean }, { "warcmaxsize", &opt.warc_maxsize, cmd_bytes }, { "warctempdir", &opt.warc_tempdir, cmd_directory }, #ifdef USE_WATT32 { "wdebug", &opt.wdebug, cmd_boolean }, #endif #ifdef ENABLE_XATTR { "xattr", &opt.enable_xattr, cmd_boolean }, #endif }; /* Look up CMDNAME in the commands[] and return its position in the array. If CMDNAME is not found, return -1. */ static int command_by_name (const char *cmdname) { /* Use binary search for speed. Wget has ~100 commands, which guarantees a worst case performance of 7 string comparisons. */ int lo = 0, hi = countof (commands) - 1; while (lo <= hi) { int mid = (lo + hi) >> 1; int cmp = c_strcasecmp (cmdname, commands[mid].name); if (cmp < 0) hi = mid - 1; else if (cmp > 0) lo = mid + 1; else return mid; } return -1; } /* Reset the variables to default values. */ void defaults (void) { char *tmp; /* Most of the default values are 0 (and 0.0, NULL, and false). Just reset everything, and fill in the non-zero values. Note that initializing pointers to NULL this way is technically illegal, but porting Wget to a machine where NULL is not all-zero bit pattern will be the least of the implementors' worries. */ xzero (opt); #ifdef HAVE_METALINK opt.metalink_index = -1; #endif opt.cookies = true; opt.verbose = -1; opt.ntry = 20; opt.reclevel = 5; opt.add_hostdir = true; opt.netrc = true; opt.ftp_glob = true; opt.htmlify = true; opt.http_keep_alive = true; opt.use_proxy = true; opt.convert_file_only = false; tmp = getenv ("no_proxy"); if (tmp) opt.no_proxy = sepstring (tmp); opt.prefer_family = prefer_none; opt.allow_cache = true; opt.if_modified_since = true; opt.read_timeout = 900; opt.use_robots = true; opt.remove_listing = true; opt.dot_bytes = 1024; opt.dot_spacing = 10; opt.dots_in_line = 50; opt.dns_cache = true; opt.ftp_pasv = true; /* 2014-09-07 Darshit Shah * opt.retr_symlinks is set to true by default. Creating symbolic links on the * local filesystem pose a security threat by malicious FTP Servers that * server a specially crafted .listing file akin to this: * * lrwxrwxrwx 1 root root 33 Dec 25 2012 JoCxl6d8rFU -> / * drwxrwxr-x 15 1024 106 4096 Aug 28 02:02 JoCxl6d8rFU * * A .listing file in this fashion makes Wget susceptiple to a symlink attack * wherein the attacker is able to create arbitrary files, directories and * symbolic links on the target system and even set permissions. * * Hence, by default Wget attempts to retrieve the pointed-to files and does * not create the symbolic links locally. */ opt.retr_symlinks = true; #ifdef HAVE_SSL opt.check_cert = CHECK_CERT_ON; opt.ftps_resume_ssl = true; opt.ftps_fallback_to_ftp = false; opt.ftps_implicit = false; opt.ftps_clear_data_connection = false; #endif #ifdef HAVE_LIBZ opt.compression = compression_none; #endif /* The default for file name restriction defaults to the OS type. */ #if defined(WINDOWS) || defined(MSDOS) || defined(__CYGWIN__) opt.restrict_files_os = restrict_windows; #elif defined(__VMS) opt.restrict_files_os = restrict_vms; #else opt.restrict_files_os = restrict_unix; #endif opt.restrict_files_ctrl = true; opt.restrict_files_nonascii = false; opt.restrict_files_case = restrict_no_case_restriction; opt.regex_type = regex_type_posix; opt.max_redirect = 20; opt.waitretry = 10; #ifdef ENABLE_IRI opt.enable_iri = true; #else opt.enable_iri = false; #endif opt.locale = NULL; opt.encoding_remote = NULL; opt.useservertimestamps = true; opt.show_all_dns_entries = false; opt.warc_maxsize = 0; /* 1024 * 1024 * 1024; */ #ifdef HAVE_LIBZ opt.warc_compression_enabled = true; #else opt.warc_compression_enabled = false; #endif opt.warc_digests_enabled = true; opt.warc_cdx_enabled = false; opt.warc_cdx_dedup_filename = NULL; opt.warc_tempdir = NULL; opt.warc_keep_log = true; /* Use a negative value to mark the absence of --start-pos option */ opt.start_pos = -1; opt.show_progress = -1; opt.noscroll = false; #ifdef HAVE_HSTS /* HSTS is enabled by default */ opt.hsts = true; #endif opt.enable_xattr = false; } /* Return the user's home directory (strdup-ed), or NULL if none is found. */ char * home_dir (void) { static char *buf = NULL; static char *home, *ret; if (!home) { home = getenv ("HOME"); if (!home) { #if defined(MSDOS) int len; /* Under MSDOS, if $HOME isn't defined, use the directory where `wget.exe' resides. */ const char *_w32_get_argv0 (void); /* in libwatt.a/pcconfig.c */ char *p; buff = _w32_get_argv0 (); p = strrchr (buf, '/'); /* djgpp */ if (!p) p = strrchr (buf, '\\'); /* others */ assert (p); len = p - buff + 1; buff = strdup (_w32_get_argv0 ()); home = buf; #elif !defined(WINDOWS) /* If HOME is not defined, try getting it from the password file. */ struct passwd *pwd = getpwuid (getuid ()); if (!pwd || !pwd->pw_dir) return NULL; home = pwd->pw_dir; #else /* !WINDOWS */ /* Under Windows, if $HOME isn't defined, use the directory where `wget.exe' resides. */ home = ws_mypath (); #endif /* WINDOWS */ } } ret = home ? xstrdup (home) : NULL; xfree (buf); return ret; } /* Check the 'WGETRC' environment variable and return the file name if 'WGETRC' is set and is a valid file. If the `WGETRC' variable exists but the file does not exist, the function will exit(). */ char * wgetrc_env_file_name (void) { char *env = getenv ("WGETRC"); if (env && *env) { file_stats_t flstat; if (!file_exists_p (env, &flstat)) { fprintf (stderr, _("%s: WGETRC points to %s, which couldn't be accessed because of error: %s.\n"), exec_name, env, strerror(flstat.access_err)); exit (WGET_EXIT_GENERIC_ERROR); } return xstrdup (env); } return NULL; } /* Append file name to (locally appropriate) directory spec. Return pointer to allocated storage. */ char * ajoin_dir_file (const char *dir, const char *file) { char *dir_file; #ifdef __VMS /* No separator: "dev:[dir]" + "name.type" */ dir_file = aprintf ("%s%s", dir, file); #else /* def __VMS */ /* Slash separator: "/a/b" + "/" + "name.type" */ dir_file = aprintf ("%s/%s", dir, file); #endif /* def __VMS [else] */ return dir_file; } /* Check for the existence of '$HOME/.wgetrc' and return its path if it exists and is set. */ char * wgetrc_user_file_name (void) { char *file = NULL; /* Join opt.homedir ($HOME) and ".wgetrc" */ if (opt.homedir) { file = ajoin_dir_file(opt.homedir, ".wgetrc"); } if (!file) return NULL; #ifndef FUZZING if (!file_exists_p (file, NULL)) { xfree (file); return NULL; } #endif return file; } /* Return the path to the user's .wgetrc. This is either the value of `WGETRC' environment variable, or `$HOME/.wgetrc'. Additionally, for windows, look in the directory where wget.exe resides. */ char * wgetrc_file_name (void) { char *file = wgetrc_env_file_name (); if (file && *file) return file; file = wgetrc_user_file_name (); #ifdef WINDOWS /* Under Windows, if we still haven't found .wgetrc, look for the file `wget.ini' in the directory where `wget.exe' resides; we do this for backward compatibility with previous versions of Wget. SYSTEM_WGETRC should not be defined under WINDOWS. */ if (!file) { const char *home = ws_mypath (); if (home) { file = aprintf ("%s/wget.ini", home); if (!file_exists_p (file, NULL)) { xfree (file); } } } #endif /* WINDOWS */ return file; } /* Return values of parse_line. */ enum parse_line { line_ok, line_empty, line_syntax_error, line_unknown_command }; static enum parse_line parse_line (const char *, char **, char **, int *); static bool setval_internal (int, const char *, const char *); static bool setval_internal_tilde (int, const char *, const char *); /* Initialize variables from a wgetrc file. Returns zero (failure) if there were errors in the file. */ bool run_wgetrc (const char *file, file_stats_t *flstats) { FILE *fp; char *line = NULL; size_t bufsize = 0; int ln; int errcnt = 0; fp = fopen_stat (file, "r", flstats); if (!fp) { fprintf (stderr, _("%s: Cannot read %s (%s).\n"), exec_name, file, strerror (errno)); return true; /* not a fatal error */ } ln = 1; while (getline (&line, &bufsize, fp) > 0) { char *com = NULL, *val = NULL; int comind; /* Parse the line. */ switch (parse_line (line, &com, &val, &comind)) { case line_ok: /* If everything is OK, set the value. */ if (!setval_internal_tilde (comind, com, val)) { fprintf (stderr, _("%s: Error in %s at line %d.\n"), exec_name, file, ln); ++errcnt; } break; case line_syntax_error: fprintf (stderr, _("%s: Syntax error in %s at line %d.\n"), exec_name, file, ln); ++errcnt; break; case line_unknown_command: fprintf (stderr, _("%s: Unknown command %s in %s at line %d.\n"), exec_name, quote (com), file, ln); ++errcnt; break; case line_empty: break; default: abort (); } xfree (com); xfree (val); ++ln; } xfree (line); fclose (fp); return errcnt == 0; } /* Initialize the defaults and run the system wgetrc and user's own wgetrc. */ int initialize (void) { char *env_sysrc; file_stats_t flstats; bool ok = true; memset(&flstats, 0, sizeof(flstats)); /* Run a non-standard system rc file when the according environment variable has been set. For internal testing purposes only! */ env_sysrc = getenv ("SYSTEM_WGETRC"); if (env_sysrc && file_exists_p (env_sysrc, &flstats)) { ok &= run_wgetrc (env_sysrc, &flstats); /* If there are any problems parsing the system wgetrc file, tell the user and exit */ if (! ok) { fprintf (stderr, _("\ Parsing system wgetrc file (env SYSTEM_WGETRC) failed. Please check\n\ '%s',\n\ or specify a different file using --config.\n"), env_sysrc); return WGET_EXIT_PARSE_ERROR; } } /* Otherwise, if SYSTEM_WGETRC is defined, use it. */ #ifdef SYSTEM_WGETRC else if (file_exists_p (SYSTEM_WGETRC, &flstats)) ok &= run_wgetrc (SYSTEM_WGETRC, &flstats); /* If there are any problems parsing the system wgetrc file, tell the user and exit */ if (! ok) { fprintf (stderr, _("\ Parsing system wgetrc file failed. Please check\n\ '%s',\n\ or specify a different file using --config.\n"), SYSTEM_WGETRC); return WGET_EXIT_PARSE_ERROR; } #endif /* Override it with your own, if one exists. */ opt.wgetrcfile = wgetrc_file_name (); if (!opt.wgetrcfile) return 0; /* #### We should canonicalize `file' and SYSTEM_WGETRC with something like realpath() before comparing them with `strcmp' */ #ifdef SYSTEM_WGETRC if (!strcmp (opt.wgetrcfile, SYSTEM_WGETRC)) { fprintf (stderr, _("\ %s: Warning: Both system and user wgetrc point to %s.\n"), exec_name, quote (opt.wgetrcfile)); } else #endif #ifndef FUZZING if (file_exists_p (opt.wgetrcfile, &flstats)) #endif ok &= run_wgetrc (opt.wgetrcfile, &flstats); xfree (opt.wgetrcfile); /* If there were errors processing either `.wgetrc', abort. */ if (!ok) return WGET_EXIT_PARSE_ERROR; return 0; } /* Remove dashes and underscores from S, modifying S in the process. */ static void dehyphen (char *s) { char *t = s; /* t - tortoise */ char *h = s; /* h - hare */ while (*h) if (*h == '_' || *h == '-') ++h; else *t++ = *h++; *t = '\0'; } /* Parse the line pointed by line, with the syntax: * command * = * value * Uses malloc to allocate space for command and value. Returns one of line_ok, line_empty, line_syntax_error, or line_unknown_command. In case of line_ok, *COM and *VAL point to freshly allocated strings, and *COMIND points to com's index. In case of error or empty line, their values are unmodified. */ static enum parse_line parse_line (const char *line, char **com, char **val, int *comind) { const char *p; const char *end = line + strlen (line); const char *cmdstart, *cmdend; const char *valstart, *valend; char buf[1024]; size_t len; char *cmdcopy; int ind; /* Skip leading and trailing whitespace. */ while (*line && c_isspace (*line)) ++line; while (end > line && c_isspace (end[-1])) --end; /* Skip empty lines and comments. */ if (!*line || *line == '#') return line_empty; p = line; cmdstart = p; while (p < end && (c_isalnum (*p) || *p == '_' || *p == '-')) ++p; cmdend = p; /* Skip '=', as well as any space before or after it. */ while (p < end && c_isspace (*p)) ++p; if (p == end || *p != '=') return line_syntax_error; ++p; while (p < end && c_isspace (*p)) ++p; valstart = p; valend = end; /* The syntax is valid (even though the command might not be). Fill in the command name and value. */ *com = strdupdelim (cmdstart, cmdend); *val = strdupdelim (valstart, valend); /* The line now known to be syntactically correct. Check whether the command is valid. */ len = cmdend - cmdstart; if (len < sizeof (buf)) cmdcopy = buf; else cmdcopy = xmalloc (len + 1); memcpy (cmdcopy, cmdstart, len); cmdcopy[len] = 0; dehyphen (cmdcopy); ind = command_by_name (cmdcopy); if (cmdcopy != buf) xfree (cmdcopy); if (ind == -1) return line_unknown_command; /* Report success to the caller. */ *comind = ind; return line_ok; } #if defined(WINDOWS) || defined(MSDOS) # define ISSEP(c) ((c) == '/' || (c) == '\\') # define SEPSTRING "/\\" #else # define ISSEP(c) ((c) == '/') # define SEPSTRING "/" #endif /* Run commands[comind].action. */ static bool setval_internal (int comind, const char *com, const char *val) { assert (0 <= comind && ((size_t) comind) < countof (commands)); if ((unsigned) comind >= countof (commands)) return NULL; DEBUGP (("Setting %s (%s) to %s\n", com, commands[comind].name, val)); return commands[comind].action (com, val, commands[comind].place); } static bool setval_internal_tilde (int comind, const char *com, const char *val) { bool ret; int homelen; char **pstring; ret = setval_internal (comind, com, val); /* We make tilde expansion for cmd_file and cmd_directory */ if (((commands[comind].action == cmd_file) || (commands[comind].action == cmd_directory)) && ret && (*val == '~' && ISSEP (val[1]))) { pstring = commands[comind].place; if (opt.homedir) { char *home = xstrdup(opt.homedir); homelen = strlen (home); while (homelen && ISSEP (home[homelen - 1])) home[--homelen] = '\0'; xfree (*pstring); /* Skip the leading "~/". */ val += strspn(val + 1, SEPSTRING) + 1; *pstring = concat_strings (home, "/", val, (char *)0); xfree (home); } } return ret; } /* Run command COM with value VAL. If running the command produces an error, report the error and exit. This is intended to be called from main to modify Wget's behavior through command-line switches. Since COM is hard-coded in main, it is not canonicalized, and this aborts when COM is not found. If COMIND's are exported to init.h, this function will be changed to accept COMIND directly. */ void setoptval (const char *com, const char *val, const char *optname) { /* Prepend "--" to OPTNAME. */ char dd_optname[2 + MAX_LONGOPTION + 1]; if ((unsigned) snprintf(dd_optname, sizeof (dd_optname), "--%s", optname) > sizeof (dd_optname)) exit (WGET_EXIT_PARSE_ERROR); assert (val != NULL); if (!setval_internal (command_by_name (com), dd_optname, val)) exit (WGET_EXIT_PARSE_ERROR); } /* Parse OPT into command and value and run it. For example, run_command("foo=bar") is equivalent to setoptval("foo", "bar"). This is used by the `--execute' flag in main.c. */ void run_command (const char *cmdopt) { char *com, *val; int comind; switch (parse_line (cmdopt, &com, &val, &comind)) { case line_ok: if (!setval_internal (comind, com, val)) exit (WGET_EXIT_PARSE_ERROR); xfree (com); xfree (val); break; default: fprintf (stderr, _("%s: Invalid --execute command %s\n"), exec_name, quote (cmdopt)); exit (WGET_EXIT_PARSE_ERROR); } } /* Generic helper functions, for use with `commands'. */ /* Forward declarations: */ struct decode_item { const char *name; int code; }; static bool decode_string (const char *, const struct decode_item *, int, int *); static bool simple_atof (const char *, const char *, double *); #define CMP1(p, c0) (c_tolower((p)[0]) == (c0) && (p)[1] == '\0') #define CMP2(p, c0, c1) (c_tolower((p)[0]) == (c0) \ && c_tolower((p)[1]) == (c1) \ && (p)[2] == '\0') #define CMP3(p, c0, c1, c2) (c_tolower((p)[0]) == (c0) \ && c_tolower((p)[1]) == (c1) \ && c_tolower((p)[2]) == (c2) \ && (p)[3] == '\0') static int cmd_boolean_internal (const char *com _GL_UNUSED, const char *val, void *place _GL_UNUSED) { if (CMP2 (val, 'o', 'n') || CMP3 (val, 'y', 'e', 's') || CMP1 (val, '1')) /* "on", "yes" and "1" mean true. */ return 1; else if (CMP3 (val, 'o', 'f', 'f') || CMP2 (val, 'n', 'o') || CMP1 (val, '0')) /* "off", "no" and "0" mean false. */ return 0; return -1; } /* Store the boolean value from VAL to PLACE. COM is ignored, except for error messages. */ static bool cmd_boolean (const char *com, const char *val, void *place) { bool value; switch (cmd_boolean_internal (com, val, place)) { case 0: value = false; break; case 1: value = true; break; default: { fprintf (stderr, _("%s: %s: Invalid boolean %s; use `on' or `off'.\n"), exec_name, com, quote (val)); return false; } } *(bool *) place = value; return true; } /* Store the check_cert value from VAL to PLACE. COM is ignored, except for error messages. */ static bool cmd_check_cert (const char *com, const char *val, void *place) { int value; switch (cmd_boolean_internal (com, val, place)) { case 0: value = CHECK_CERT_OFF; break; case 1: value = CHECK_CERT_ON; break; default: { if (!c_strcasecmp (val, "quiet")) value = CHECK_CERT_QUIET; else { fprintf (stderr, _("%s: %s: Invalid %s; use `on', `off' or `quiet'.\n"), exec_name, com, quote (val)); return false; } } } *(int *) place = value; return true; } /* Set the non-negative integer value from VAL to PLACE. With incorrect specification, the number remains unchanged. */ static bool cmd_number (const char *com, const char *val, void *place) { long l = strtol(val, NULL, 10); if (((l == LONG_MIN || l == LONG_MAX) && errno == ERANGE) || l < 0 || l > INT_MAX) { fprintf (stderr, _("%s: %s: Invalid number %s.\n"), exec_name, com, quote (val)); return false; } *(int *) place = (int) l; return true; } /* Similar to cmd_number(), only accepts `inf' as a synonym for 0. */ static bool cmd_number_inf (const char *com, const char *val, void *place) { if (!c_strcasecmp (val, "inf")) { *(int *) place = 0; return true; } return cmd_number (com, val, place); } /* Copy (strdup) the string at COM to a new location and place a pointer to *PLACE. */ static bool cmd_string (const char *com _GL_UNUSED, const char *val, void *place) { char **pstring = (char **)place; xfree (*pstring); *pstring = xstrdup (val); return true; } /* Like cmd_string but ensure the string is upper case. */ static bool cmd_string_uppercase (const char *com _GL_UNUSED, const char *val, void *place) { char *q, **pstring; pstring = (char **)place; xfree (*pstring); *pstring = xmalloc (strlen (val) + 1); for (q = *pstring; *val; val++, q++) *q = c_toupper (*val); *q = '\0'; return true; } /* Like cmd_string, but handles tilde-expansion when reading a user's `.wgetrc'. In that case, and if VAL begins with `~', the tilde gets expanded to the user's home directory. */ static bool cmd_file (const char *com _GL_UNUSED, const char *val, void *place) { char **pstring = (char **)place; xfree (*pstring); /* #### If VAL is empty, perhaps should set *PLACE to NULL. */ *pstring = xstrdup (val); #if defined(WINDOWS) || defined(MSDOS) /* Convert "\" to "/". */ { char *s; for (s = *pstring; *s; s++) if (*s == '\\') *s = '/'; } #endif return true; } /* like cmd_file, but insist on just a single option usage */ static bool cmd_file_once (const char *com _GL_UNUSED, const char *val, void *place) { if (*(char **)place) { fprintf (stderr, _("%s: %s must only be used once\n"), exec_name, com); return false; } return cmd_file(com, val, place); } /* Like cmd_file, but strips trailing '/' characters. */ static bool cmd_directory (const char *com, const char *val, void *place) { char *s, *t; /* Call cmd_file() for tilde expansion and separator canonicalization (backslash -> slash under Windows). These things should perhaps be in a separate function. */ if (!cmd_file (com, val, place)) return false; s = *(char **)place; t = s + strlen (s); while (t > s && *--t == '/') *t = '\0'; return true; } /* Split VAL by space to a vector of values, and append those values to vector pointed to by the PLACE argument. If VAL is empty, the PLACE vector is cleared instead. */ static bool cmd_vector (const char *com _GL_UNUSED, const char *val, void *place) { char ***pvec = (char ***)place; if (*val) *pvec = merge_vecs (*pvec, sepstring (val)); else { free_vec (*pvec); *pvec = NULL; } return true; } static bool cmd_directory_vector (const char *com _GL_UNUSED, const char *val, void *place) { char ***pvec = (char ***)place; if (*val) { /* Strip the trailing slashes from directories. */ char **t, **seps; seps = sepstring (val); for (t = seps; t && *t; t++) { int len = strlen (*t); /* Skip degenerate case of root directory. */ if (len > 1) { if ((*t)[len - 1] == '/') (*t)[len - 1] = '\0'; } } *pvec = merge_vecs (*pvec, seps); } else { free_vec (*pvec); *pvec = NULL; } return true; } /* Engine for cmd_bytes and cmd_bytes_sum: converts a string such as "100k" or "2.5G" to a floating point number. */ static bool parse_bytes_helper (const char *val, double *result) { double number, mult; const char *end = val + strlen (val); /* Check for "inf". */ if (0 == strcmp (val, "inf")) { *result = 0; return true; } /* Strip trailing whitespace. */ while (val < end && c_isspace (end[-1])) --end; if (val == end) return false; switch (c_tolower (end[-1])) { case 'k': --end, mult = 1024.0; break; case 'm': --end, mult = 1048576.0; break; case 'g': --end, mult = 1073741824.0; break; case 't': --end, mult = 1099511627776.0; break; default: /* Not a recognized suffix: assume it's a digit. (If not, simple_atof will raise an error.) */ mult = 1; } /* Skip leading and trailing whitespace. */ while (val < end && c_isspace (*val)) ++val; while (val < end && c_isspace (end[-1])) --end; if (val == end) return false; if (!simple_atof (val, end, &number) || number < 0) return false; *result = number * mult; return true; } /* Parse VAL as a number and set its value to PLACE (which should point to a wgint). By default, the value is assumed to be in bytes. If "K", "M", or "G" are appended, the value is multiplied with 1<<10, 1<<20, or 1<<30, respectively. Floating point values are allowed and are cast to integer before use. The idea is to be able to use things like 1.5k instead of "1536". The string "inf" is returned as 0. In case of error, false is returned and memory pointed to by PLACE remains unmodified. */ static bool cmd_bytes (const char *com, const char *val, void *place) { double byte_value; if (!parse_bytes_helper (val, &byte_value)) { fprintf (stderr, _("%s: %s: Invalid byte value %s\n"), exec_name, com, quote (val)); return false; } *(wgint *)place = (wgint)byte_value; return true; } /* Like cmd_bytes, but PLACE is interpreted as a pointer to SIZE_SUM. It works by converting the string to double, therefore working with values up to 2^53-1 without loss of precision. This value (8192 TB) is large enough to serve for a while. */ static bool cmd_bytes_sum (const char *com, const char *val, void *place) { double byte_value; if (!parse_bytes_helper (val, &byte_value) || byte_value < WGINT_MIN || byte_value > WGINT_MAX) { fprintf (stderr, _("%s: %s: Invalid byte value %s\n"), exec_name, com, quote (val)); return false; } *(wgint *) place = (wgint) byte_value; return true; } /* Store the value of VAL to *OUT. The value is a time period, by default expressed in seconds, but also accepting suffixes "m", "h", "d", and "w" for minutes, hours, days, and weeks respectively. */ static bool cmd_time (const char *com, const char *val, void *place) { double number, mult; const char *end = val + strlen (val); /* Strip trailing whitespace. */ while (val < end && c_isspace (end[-1])) --end; if (val == end) { err: fprintf (stderr, _("%s: %s: Invalid time period %s\n"), exec_name, com, quote (val)); return false; } switch (c_tolower (end[-1])) { case 's': --end, mult = 1; /* seconds */ break; case 'm': --end, mult = 60; /* minutes */ break; case 'h': --end, mult = 3600; /* hours */ break; case 'd': --end, mult = 86400.0; /* days */ break; case 'w': --end, mult = 604800.0; /* weeks */ break; default: /* Not a recognized suffix: assume it belongs to the number. (If not, simple_atof will raise an error.) */ mult = 1; } /* Skip leading and trailing whitespace. */ while (val < end && c_isspace (*val)) ++val; while (val < end && c_isspace (end[-1])) --end; if (val == end) goto err; if (!simple_atof (val, end, &number)) goto err; if (number < 0) { fprintf (stderr, _("%s: %s: Negative time period %s\n"), exec_name, com, quote (val)); return false; } *(double *)place = number * mult; return true; } static bool cmd_use_askpass (const char *com _GL_UNUSED, const char *val, void *place) { const char *env_name = "WGET_ASKPASS"; const char *env; if (val && *val) return cmd_string (com, val, place); env = getenv (env_name); if (!(env && *env)) { env_name = "SSH_ASKPASS"; env = getenv (env_name); } if (!(env && *env)) { fprintf (stderr, _("use-askpass requires a string or either environment variable WGET_ASKPASS or SSH_ASKPASS to be set.\n")); return false; } return cmd_string (com, env, place); } #ifdef HAVE_SSL static bool cmd_cert_type (const char *com, const char *val, void *place) { static const struct decode_item choices[] = { { "pem", keyfile_pem }, { "der", keyfile_asn1 }, { "asn1", keyfile_asn1 }, }; int ok = decode_string (val, choices, countof (choices), place); if (!ok) fprintf (stderr, _("%s: %s: Invalid value %s.\n"), exec_name, com, quote (val)); return ok; } #endif /* Specialized helper functions, used by `commands' to handle some options specially. */ static bool check_user_specified_header (const char *); #ifdef HAVE_LIBZ static bool cmd_spec_compression (const char *com, const char *val, void *place) { static const struct decode_item choices[] = { { "auto", compression_auto }, { "gzip", compression_gzip }, { "none", compression_none }, }; int ok = decode_string (val, choices, countof (choices), place); if (!ok) { fprintf (stderr, _("%s: %s: Invalid value %s.\n"), exec_name, com, quote (val)); } return ok; } #endif static bool cmd_spec_dirstruct (const char *com, const char *val, void *place_ignored _GL_UNUSED) { if (!cmd_boolean (com, val, &opt.dirstruct)) return false; /* Since dirstruct behaviour is explicitly changed, no_dirstruct must be affected inversely. */ if (opt.dirstruct) opt.no_dirstruct = false; else opt.no_dirstruct = true; return true; } static bool cmd_spec_header (const char *com, const char *val, void *place_ignored _GL_UNUSED) { /* Empty value means reset the list of headers. */ if (*val == '\0') { free_vec (opt.user_headers); opt.user_headers = NULL; return true; } if (!check_user_specified_header (val)) { fprintf (stderr, _("%s: %s: Invalid header %s.\n"), exec_name, com, quote (val)); return false; } opt.user_headers = vec_append (opt.user_headers, val); return true; } static bool cmd_spec_warc_header (const char *com, const char *val, void *place_ignored _GL_UNUSED) { /* Empty value means reset the list of headers. */ if (*val == '\0') { free_vec (opt.warc_user_headers); opt.warc_user_headers = NULL; return true; } if (!check_user_specified_header (val)) { fprintf (stderr, _("%s: %s: Invalid WARC header %s.\n"), exec_name, com, quote (val)); return false; } opt.warc_user_headers = vec_append (opt.warc_user_headers, val); return true; } static bool cmd_spec_htmlify (const char *com, const char *val, void *place_ignored _GL_UNUSED) { int flag = cmd_boolean (com, val, &opt.htmlify); if (flag && !opt.htmlify) opt.remove_listing = false; return flag; } /* Set the "mirror" mode. It means: recursive download, timestamping, no limit on max. recursion depth, and don't remove listings. */ static bool cmd_spec_mirror (const char *com, const char *val, void *place_ignored _GL_UNUSED) { bool mirror; if (!cmd_boolean (com, val, &mirror)) return false; if (mirror) { opt.recursive = true; if (!opt.no_dirstruct) opt.dirstruct = true; opt.timestamping = true; opt.reclevel = INFINITE_RECURSION; opt.remove_listing = false; } return true; } /* Validate --prefer-family and set the choice. Allowed values are "IPv4", "IPv6", and "none". */ static bool cmd_spec_prefer_family (const char *com, const char *val, void *place_ignored _GL_UNUSED) { static const struct decode_item choices[] = { { "IPv4", prefer_ipv4 }, { "IPv6", prefer_ipv6 }, { "none", prefer_none }, }; int prefer_family = prefer_none; int ok = decode_string (val, choices, countof (choices), &prefer_family); if (!ok) fprintf (stderr, _("%s: %s: Invalid value %s.\n"), exec_name, com, quote (val)); opt.prefer_family = prefer_family; return ok; } /* Set progress.type to VAL, but verify that it's a valid progress implementation before that. */ static bool cmd_spec_progress (const char *com, const char *val, void *place_ignored _GL_UNUSED) { if (!valid_progress_implementation_p (val)) { fprintf (stderr, _("%s: %s: Invalid progress type %s.\n"), exec_name, com, quote (val)); return false; } xfree (opt.progress_type); /* Don't call set_progress_implementation here. It will be called in main when it becomes clear what the log output is. */ opt.progress_type = xstrdup (val); return true; } /* Set opt.recursive to VAL as with cmd_boolean. If opt.recursive is set to true, also set opt.dirstruct to true, unless opt.no_dirstruct is specified. */ static bool cmd_spec_recursive (const char *com, const char *val, void *place_ignored _GL_UNUSED) { if (!cmd_boolean (com, val, &opt.recursive)) return false; else { if (opt.recursive && !opt.no_dirstruct) opt.dirstruct = true; } return true; } /* Validate --regex-type and set the choice. */ static bool cmd_spec_regex_type (const char *com, const char *val, void *place_ignored _GL_UNUSED) { static const struct decode_item choices[] = { { "posix", regex_type_posix }, #if defined HAVE_LIBPCRE || defined HAVE_LIBPCRE2 { "pcre", regex_type_pcre }, #endif }; int regex_type = regex_type_posix; int ok = decode_string (val, choices, countof (choices), ®ex_type); if (!ok) fprintf (stderr, _("%s: %s: Invalid value %s.\n"), exec_name, com, quote (val)); opt.regex_type = regex_type; return ok; } static bool cmd_spec_restrict_file_names (const char *com, const char *val, void *place_ignored _GL_UNUSED) { int restrict_os = opt.restrict_files_os; int restrict_ctrl = opt.restrict_files_ctrl; int restrict_case = opt.restrict_files_case; int restrict_nonascii = opt.restrict_files_nonascii; const char *end; #define VAL_IS(string_literal) BOUNDED_EQUAL (val, end, string_literal) do { end = strchr (val, ','); if (!end) end = val + strlen (val); if (VAL_IS ("unix")) restrict_os = restrict_unix; else if (VAL_IS ("vms")) restrict_os = restrict_vms; else if (VAL_IS ("windows")) restrict_os = restrict_windows; else if (VAL_IS ("lowercase")) restrict_case = restrict_lowercase; else if (VAL_IS ("uppercase")) restrict_case = restrict_uppercase; else if (VAL_IS ("nocontrol")) restrict_ctrl = false; else if (VAL_IS ("ascii")) restrict_nonascii = true; else { fprintf (stderr, _("\ %s: %s: Invalid restriction %s,\n\ use [unix|vms|windows],[lowercase|uppercase],[nocontrol],[ascii].\n"), exec_name, com, quote (val)); return false; } if (*end) val = end + 1; } while (*val && *end); #undef VAL_IS opt.restrict_files_os = restrict_os; opt.restrict_files_ctrl = restrict_ctrl; opt.restrict_files_case = restrict_case; opt.restrict_files_nonascii = restrict_nonascii; return true; } static bool cmd_spec_report_speed (const char *com, const char *val, void *place_ignored _GL_UNUSED) { opt.report_bps = c_strcasecmp (val, "bits") == 0; if (!opt.report_bps) fprintf (stderr, _("%s: %s: Invalid value %s.\n"), exec_name, com, quote (val)); return opt.report_bps; } #ifdef HAVE_SSL static bool cmd_spec_secure_protocol (const char *com, const char *val, void *place) { static const struct decode_item choices[] = { { "auto", secure_protocol_auto }, { "sslv2", secure_protocol_sslv2 }, { "sslv3", secure_protocol_sslv3 }, { "tlsv1", secure_protocol_tlsv1 }, { "tlsv1_1", secure_protocol_tlsv1_1 }, { "tlsv1_2", secure_protocol_tlsv1_2 }, { "tlsv1_3", secure_protocol_tlsv1_3 }, { "pfs", secure_protocol_pfs }, }; int ok = decode_string (val, choices, countof (choices), place); if (!ok) fprintf (stderr, _("%s: %s: Invalid value %s.\n"), exec_name, com, quote (val)); return ok; } #endif /* Set all three timeout values. */ static bool cmd_spec_timeout (const char *com, const char *val, void *place_ignored _GL_UNUSED) { double value; if (!cmd_time (com, val, &value)) return false; opt.read_timeout = value; opt.connect_timeout = value; opt.dns_timeout = value; return true; } static bool cmd_spec_useragent (const char *com, const char *val, void *place_ignored _GL_UNUSED) { /* Disallow embedded newlines. */ if (strchr (val, '\n')) { fprintf (stderr, _("%s: %s: Invalid value %s.\n"), exec_name, com, quote (val)); return false; } xfree (opt.useragent); opt.useragent = xstrdup (val); return true; } /* The --show-progress option is not a cmd_boolean since we need to keep track * of whether the user explicitly requested the option or not. -1 means * uninitialized. */ static bool cmd_spec_progressdisp (const char *com, const char *val, void *place _GL_UNUSED) { bool flag; if (cmd_boolean (com, val, &flag)) { opt.show_progress = flag; return true; } return false; } /* The "verbose" option cannot be cmd_boolean because the variable is not bool -- it's of type int (-1 means uninitialized because of some random hackery for disallowing -q -v). */ static bool cmd_spec_verbose (const char *com, const char *val, void *place_ignored _GL_UNUSED) { bool flag; if (cmd_boolean (com, val, &flag)) { opt.verbose = flag; opt.show_progress = -1; return true; } return false; } /* Miscellaneous useful routines. */ /* Trivial atof, with error reporting. Handles "[.]", doesn't handle exponential notation. Returns true on success, false on failure. In case of success, stores its result to *DEST. */ static bool simple_atof (const char *beg, const char *end, double *dest) { double result = 0; bool negative = false; bool seen_dot = false; bool seen_digit = false; double divider = 1; const char *p = beg; while (p < end && c_isspace (*p)) ++p; if (p < end && (*p == '-' || *p == '+')) { negative = (*p == '-'); ++p; } for (; p < end; p++) { char ch = *p; if (c_isdigit (ch)) { if (!seen_dot) result = (10 * result) + (ch - '0'); else result += (ch - '0') / (divider *= 10); seen_digit = true; } else if (ch == '.') { if (!seen_dot) seen_dot = true; else return false; } else return false; } if (!seen_digit) return false; if (negative) result = -result; *dest = result; return true; } /* Verify that the user-specified header in S is valid. It must contain a colon preceded by non-white-space characters and must not contain newlines. */ static bool check_user_specified_header (const char *s) { const char *p; for (p = s; *p && *p != ':' && !c_isspace (*p); p++) ; /* The header MUST contain `:' preceded by at least one non-whitespace character. */ if (*p != ':' || p == s) return false; /* The header MUST NOT contain newlines. */ if (strchr (s, '\n')) return false; return true; } /* Decode VAL into a number, according to ITEMS. */ static bool decode_string (const char *val, const struct decode_item *items, int itemcount, int *place) { int i; for (i = 0; i < itemcount; i++) if (0 == c_strcasecmp (val, items[i].name)) { *place = items[i].code; return true; } return false; } extern struct ptimer *timer; extern int cleaned_up; /* Free the memory allocated by global variables. */ void cleanup (void) { /* Free external resources, close files, etc. */ if (cleaned_up++) return; /* cleanup() must not be called twice */ /* Close WARC file. */ if (opt.warc_filename != 0) warc_close (); log_close (); if (output_stream && output_stream != stderr) { FILE *fp = output_stream; output_stream = NULL; if (fclose (fp) == EOF) inform_exit_status (CLOSEFAILED); } /* No need to check for error because Wget flushes its output (and checks for errors) after any data arrives. */ /* We're exiting anyway so there's no real need to call free() hundreds of times. Skipping the frees will make Wget exit faster. * However, when detecting leaks, it's crucial to free() everything because then you can find the real leaks, i.e. the allocated memory which grows with the size of the program. */ #if defined DEBUG_MALLOC || defined TESTING convert_cleanup (); res_cleanup (); http_cleanup (); cleanup_html_url (); spider_cleanup (); host_cleanup (); log_cleanup (); netrc_cleanup (); #ifdef HAVE_SSL ssl_cleanup (); #endif connect_cleanup (); xfree (opt.choose_config); xfree (opt.lfilename); xfree (opt.dir_prefix); xfree (opt.input_filename); #ifdef HAVE_METALINK xfree (opt.input_metalink); xfree (opt.preferred_location); #endif xfree (opt.output_document); xfree (opt.default_page); if (opt.regex_type == regex_type_posix) { if (opt.acceptregex) regfree (opt.acceptregex); if (opt.rejectregex) regfree (opt.rejectregex); } xfree (opt.acceptregex); xfree (opt.rejectregex); xfree (opt.acceptregex_s); xfree (opt.rejectregex_s); free_vec (opt.accepts); free_vec (opt.rejects); free_vec ((char **)opt.excludes); free_vec ((char **)opt.includes); free_vec (opt.domains); free_vec (opt.exclude_domains); free_vec (opt.follow_tags); free_vec (opt.ignore_tags); xfree (opt.progress_type); xfree (opt.warc_filename); xfree (opt.warc_tempdir); xfree (opt.warc_cdx_dedup_filename); xfree (opt.ftp_user); xfree (opt.ftp_passwd); xfree (opt.ftp_proxy); xfree (opt.https_proxy); xfree (opt.http_proxy); free_vec (opt.no_proxy); xfree (opt.proxy_user); xfree (opt.proxy_passwd); xfree (opt.useragent); xfree (opt.referer); xfree (opt.http_user); xfree (opt.http_passwd); xfree (opt.dot_style); free_vec (opt.user_headers); free_vec (opt.warc_user_headers); # ifdef HAVE_SSL xfree (opt.cert_file); xfree (opt.private_key); xfree (opt.ca_directory); xfree (opt.ca_cert); xfree (opt.crl_file); xfree (opt.pinnedpubkey); xfree (opt.random_file); xfree (opt.egd_file); # endif xfree (opt.bind_address); xfree (opt.cookies_input); xfree (opt.cookies_output); xfree (opt.user); xfree (opt.passwd); xfree (opt.base_href); xfree (opt.method); xfree (opt.post_file_name); xfree (opt.post_data); xfree (opt.body_data); xfree (opt.body_file); xfree (opt.rejected_log); xfree (opt.use_askpass); xfree (opt.retry_on_http_error); xfree (opt.encoding_remote); xfree (opt.locale); #ifdef HAVE_HSTS xfree (opt.hsts_file); #endif xfree (opt.wgetrcfile); xfree (opt.homedir); xfree (exec_name); xfree (program_argstring); ptimer_destroy (timer); timer = NULL; #ifdef HAVE_LIBCARES #include { extern ares_channel ares; xfree (opt.bind_dns_address); xfree (opt.dns_servers); ares_destroy (ares); ares_library_cleanup (); } #endif quotearg_free (); #endif /* DEBUG_MALLOC || TESTING */ } /* Unit testing routines. */ #ifdef TESTING const char * test_commands_sorted(void) { unsigned i; for (i = 1; i < countof(commands); ++i) { if (c_strcasecmp (commands[i - 1].name, commands[i].name) > 0) { mu_assert ("FAILED", false); break; } } return NULL; } const char * test_cmd_spec_restrict_file_names(void) { unsigned i; static const struct { const char *val; int expected_restrict_files_os; bool expected_restrict_files_ctrl; int expected_restrict_files_case; bool result; } test_array[] = { { "windows", restrict_windows, true, restrict_no_case_restriction, true }, { "windows,", restrict_windows, true, restrict_no_case_restriction, true }, { "windows,lowercase", restrict_windows, true, restrict_lowercase, true }, { "unix,nocontrol,lowercase,", restrict_unix, false, restrict_lowercase, true }, }; for (i = 0; i < countof(test_array); ++i) { bool res; defaults(); res = cmd_spec_restrict_file_names ("dummy", test_array[i].val, NULL); /* fprintf (stderr, "test_cmd_spec_restrict_file_names: TEST %d\n", i); fflush (stderr); fprintf (stderr, "opt.restrict_files_os: %d\n", opt.restrict_files_os); fflush (stderr); fprintf (stderr, "opt.restrict_files_ctrl: %d\n", opt.restrict_files_ctrl); fflush (stderr); fprintf (stderr, "opt.restrict_files_case: %d\n", opt.restrict_files_case); fflush (stderr); */ mu_assert ("test_cmd_spec_restrict_file_names: wrong result", res == test_array[i].result && (int) opt.restrict_files_os == test_array[i].expected_restrict_files_os && opt.restrict_files_ctrl == test_array[i].expected_restrict_files_ctrl && (int) opt.restrict_files_case == test_array[i].expected_restrict_files_case); } return NULL; } #endif /* TESTING */ wget-1.21.2/src/recur.h0000644000000000000000000000350214115732710011513 00000000000000/* Declarations for recur.c. Copyright (C) 1996-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #ifndef RECUR_H #define RECUR_H #include "url.h" /* For most options, 0 means no limits, but with -p in the picture, that causes a problem on the maximum recursion depth variable. To retain backwards compatibility we allow users to consider "0" to be synonymous with "inf" for -l, but internally infinite recursion is specified by -1 and 0 means to only retrieve the requisites of a single document. */ #define INFINITE_RECURSION -1 struct urlpos; void recursive_cleanup (void); uerr_t retrieve_tree (struct url *, struct iri *); #endif /* RECUR_H */ wget-1.21.2/src/convert.c0000644000000000000000000011214714115732710012054 00000000000000/* Conversion of links to local files. Copyright (C) 2003-2011, 2014-2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #include "wget.h" #include #include #include #include #include #include #include "convert.h" #include "url.h" #include "recur.h" #include "utils.h" #include "hash.h" #include "ptimer.h" #include "res.h" #include "html-url.h" #include "css-url.h" #include "iri.h" #include "xstrndup.h" static struct hash_table *dl_file_url_map; struct hash_table *dl_url_file_map; /* Set of HTML/CSS files downloaded in this Wget run, used for link conversion after Wget is done. */ struct hash_table *downloaded_html_set; struct hash_table *downloaded_css_set; static void convert_links (const char *, struct urlpos *); static void convert_links_in_hashtable (struct hash_table *downloaded_set, int is_css, int *file_count) { int i, cnt = 0; char *arr[1024], **file_array; if (!downloaded_set || (cnt = hash_table_count (downloaded_set)) == 0) return; if (cnt <= (int) countof (arr)) file_array = arr; else file_array = xmalloc (cnt * sizeof (arr[0])); string_set_to_array (downloaded_set, file_array); for (i = 0; i < cnt; i++) { struct urlpos *urls, *cur_url; char *url; char *file = file_array[i]; /* Determine the URL of the file. get_urls_{html,css} will need it. */ url = hash_table_get (dl_file_url_map, file); if (!url) { DEBUGP (("Apparently %s has been removed.\n", file)); continue; } DEBUGP (("Scanning %s (from %s)\n", file, url)); /* Parse the file... */ urls = is_css ? get_urls_css_file (file, url) : get_urls_html (file, url, NULL, NULL); /* We don't respect meta_disallow_follow here because, even if the file is not followed, we might still want to convert the links that have been followed from other files. */ for (cur_url = urls; cur_url; cur_url = cur_url->next) { char *local_name; struct url *u; struct iri *pi; if (cur_url->link_base_p) { /* Base references have been resolved by our parser, so we turn the base URL into an empty string. (Perhaps we should remove the tag entirely?) */ cur_url->convert = CO_NULLIFY_BASE; continue; } /* We decide the direction of conversion according to whether a URL was downloaded. Downloaded URLs will be converted ABS2REL, whereas non-downloaded will be converted REL2ABS. */ pi = iri_new (); set_uri_encoding (pi, opt.locale, true); u = url_parse (cur_url->url->url, NULL, pi, true); if (!u) continue; local_name = hash_table_get (dl_url_file_map, u->url); /* Decide on the conversion type. */ if (local_name) { /* We've downloaded this URL. Convert it to relative form. We do this even if the URL already is in relative form, because our directory structure may not be identical to that on the server (think `-nd', `--cut-dirs', etc.). If --convert-file-only was passed, we only convert the basename portion of the URL. */ cur_url->convert = (opt.convert_file_only ? CO_CONVERT_BASENAME_ONLY : CO_CONVERT_TO_RELATIVE); cur_url->local_name = xstrdup (local_name); DEBUGP (("will convert url %s to local %s\n", u->url, local_name)); } else { /* We haven't downloaded this URL. If it's not already complete (including a full host name), convert it to that form, so it can be reached while browsing this HTML locally. */ if (!cur_url->link_complete_p) cur_url->convert = CO_CONVERT_TO_COMPLETE; cur_url->local_name = NULL; DEBUGP (("will convert url %s to complete\n", u->url)); } url_free (u); iri_free (pi); } /* Convert the links in the file. */ convert_links (file, urls); ++*file_count; /* Free the data. */ free_urlpos (urls); } if (file_array != arr) xfree (file_array); } /* This function is called when the retrieval is done to convert the links that have been downloaded. It has to be called at the end of the retrieval, because only then does Wget know conclusively which URLs have been downloaded, and which not, so it can tell which direction to convert to. The "direction" means that the URLs to the files that have been downloaded get converted to the relative URL which will point to that file. And the other URLs get converted to the remote URL on the server. All the downloaded HTMLs are kept in downloaded_html_files, and downloaded URLs in urls_downloaded. All the information is extracted from these two lists. */ void convert_all_links (void) { double secs; int file_count = 0; struct ptimer *timer = ptimer_new (); convert_links_in_hashtable (downloaded_html_set, 0, &file_count); convert_links_in_hashtable (downloaded_css_set, 1, &file_count); secs = ptimer_measure (timer); logprintf (LOG_VERBOSE, _("Converted links in %d files in %s seconds.\n"), file_count, print_decimal (secs)); ptimer_destroy (timer); } static void write_backup_file (const char *, downloaded_file_t); static const char *replace_plain (const char*, int, FILE*, const char *); static const char *replace_attr (const char *, int, FILE *, const char *); static const char *replace_attr_refresh_hack (const char *, int, FILE *, const char *, int); static char *local_quote_string (const char *, bool); static char *construct_relative (const char *, const char *); static char *convert_basename (const char *, const struct urlpos *); /* Change the links in one file. LINKS is a list of links in the document, along with their positions and the desired direction of the conversion. */ static void convert_links (const char *file, struct urlpos *links) { struct file_memory *fm; FILE *fp; const char *p; downloaded_file_t downloaded_file_return; struct urlpos *link; int to_url_count = 0, to_file_count = 0; logprintf (LOG_VERBOSE, _("Converting links in %s... "), file); { /* First we do a "dry run": go through the list L and see whether any URL needs to be converted in the first place. If not, just leave the file alone. */ int dry_count = 0; struct urlpos *dry; for (dry = links; dry; dry = dry->next) if (dry->convert != CO_NOCONVERT) ++dry_count; if (!dry_count) { logputs (LOG_VERBOSE, _("nothing to do.\n")); return; } logprintf (LOG_VERBOSE, _("%d.\n"), dry_count); } fm = wget_read_file (file); if (!fm) { logprintf (LOG_NOTQUIET, _("Cannot convert links in %s: %s\n"), file, strerror (errno)); return; } downloaded_file_return = downloaded_file (CHECK_FOR_FILE, file); if (opt.backup_converted && downloaded_file_return) write_backup_file (file, downloaded_file_return); /* Before opening the file for writing, unlink the file. This is important if the data in FM is mapped. In such case, nulling the file, which is what fopen() below does, would make us read all zeroes from the mapped region. */ if (unlink (file) < 0 && errno != ENOENT) { logprintf (LOG_NOTQUIET, _("Unable to delete %s: %s\n"), quote (file), strerror (errno)); wget_read_file_free (fm); return; } /* Now open the file for writing. */ fp = fopen (file, "wb"); if (!fp) { logprintf (LOG_NOTQUIET, _("Cannot convert links in %s: %s\n"), file, strerror (errno)); wget_read_file_free (fm); return; } /* Here we loop through all the URLs in file, replacing those of them that are downloaded with relative references. */ p = fm->content; for (link = links; link; link = link->next) { char *url_start = fm->content + link->pos; if (link->pos >= fm->length) { DEBUGP (("Something strange is going on. Please investigate.")); break; } /* If the URL is not to be converted, skip it. */ if (link->convert == CO_NOCONVERT) { DEBUGP (("Skipping %s at position %d.\n", link->url->url, link->pos)); continue; } /* Echo the file contents, up to the offending URL's opening quote, to the outfile. */ fwrite (p, 1, url_start - p, fp); p = url_start; switch (link->convert) { case CO_CONVERT_TO_RELATIVE: /* Convert absolute URL to relative. */ if (link->local_name) { char *newname = construct_relative (file, link->local_name); char *quoted_newname = local_quote_string (newname, link->link_css_p); if (link->link_css_p || link->link_noquote_html_p) p = replace_plain (p, link->size, fp, quoted_newname); else if (!link->link_refresh_p) p = replace_attr (p, link->size, fp, quoted_newname); else p = replace_attr_refresh_hack (p, link->size, fp, quoted_newname, link->refresh_timeout); DEBUGP (("TO_RELATIVE: %s to %s at position %d in %s.\n", link->url->url, newname, link->pos, file)); xfree (newname); xfree (quoted_newname); ++to_file_count; } break; case CO_CONVERT_BASENAME_ONLY: { char *newname = convert_basename (p, link); char *quoted_newname = local_quote_string (newname, link->link_css_p); if (link->link_css_p || link->link_noquote_html_p) p = replace_plain (p, link->size, fp, quoted_newname); else if (!link->link_refresh_p) p = replace_attr (p, link->size, fp, quoted_newname); else p = replace_attr_refresh_hack (p, link->size, fp, quoted_newname, link->refresh_timeout); DEBUGP (("Converted file part only: %s to %s at position %d in %s.\n", link->url->url, newname, link->pos, file)); xfree (newname); xfree (quoted_newname); ++to_file_count; break; } case CO_CONVERT_TO_COMPLETE: /* Convert the link to absolute URL. */ { char *newlink = link->url->url; char *quoted_newlink = html_quote_string (newlink); if (link->link_css_p || link->link_noquote_html_p) p = replace_plain (p, link->size, fp, newlink); else if (!link->link_refresh_p) p = replace_attr (p, link->size, fp, quoted_newlink); else p = replace_attr_refresh_hack (p, link->size, fp, quoted_newlink, link->refresh_timeout); DEBUGP (("TO_COMPLETE: to %s at position %d in %s.\n", newlink, link->pos, file)); xfree (quoted_newlink); ++to_url_count; break; } case CO_NULLIFY_BASE: /* Change the base href to "". */ p = replace_attr (p, link->size, fp, ""); break; case CO_NOCONVERT: abort (); break; } } /* Output the rest of the file. */ if (p - fm->content < fm->length) fwrite (p, 1, fm->length - (p - fm->content), fp); fclose (fp); wget_read_file_free (fm); logprintf (LOG_VERBOSE, "%d-%d\n", to_file_count, to_url_count); } /* Construct and return a link that points from BASEFILE to LINKFILE. Both files should be local file names, BASEFILE of the referrering file, and LINKFILE of the referred file. Examples: cr("foo", "bar") -> "bar" cr("A/foo", "A/bar") -> "bar" cr("A/foo", "A/B/bar") -> "B/bar" cr("A/X/foo", "A/Y/bar") -> "../Y/bar" cr("X/", "Y/bar") -> "../Y/bar" (trailing slash does matter in BASE) Both files should be absolute or relative, otherwise strange results might ensue. The function makes no special efforts to handle "." and ".." in links, so make sure they're not there (e.g. using path_simplify). */ static char * construct_relative (const char *basefile, const char *linkfile) { char *link; int basedirs; const char *b, *l; int i, start; /* First, skip the initial directory components common to both files. */ start = 0; for (b = basefile, l = linkfile; *b == *l && *b != '\0'; ++b, ++l) { if (*b == '/') start = (b - basefile) + 1; } basefile += start; linkfile += start; /* With common directories out of the way, the situation we have is as follows: b - b1/b2/[...]/bfile l - l1/l2/[...]/lfile The link we're constructing needs to be: lnk - ../../l1/l2/[...]/lfile Where the number of ".."'s equals the number of bN directory components in B. */ /* Count the directory components in B. */ basedirs = 0; for (b = basefile; *b; b++) { if (*b == '/') ++basedirs; } if (!basedirs && (b = strpbrk (linkfile, "/:")) && *b == ':') { link = xmalloc (2 + strlen (linkfile) + 1); memcpy (link, "./", 2); strcpy (link + 2, linkfile); } else { /* Construct LINK as explained above. */ link = xmalloc (3 * basedirs + strlen (linkfile) + 1); for (i = 0; i < basedirs; i++) memcpy (link + 3 * i, "../", 3); strcpy (link + 3 * i, linkfile); } return link; } /* Construct and return a "transparent proxy" URL reflecting changes made by --adjust-extension to the file component (i.e., "basename") of the original URL, but leaving the "dirname" of the URL (protocol://hostname... portion) untouched. Think: populating a squid cache via a recursive wget scrape, where changing URLs to work locally with "file://..." is NOT desirable. Example: if p = "//foo.com/bar.cgi?xyz" and link->local_name = "docroot/foo.com/bar.cgi?xyz.css" then new_construct_func(p, link); will return "//foo.com/bar.cgi?xyz.css" Essentially, we do s/$(basename orig_url)/$(basename link->local_name)/ */ static char * convert_basename (const char *p, const struct urlpos *link) { int len = link->size; char *url = NULL; char *org_basename = NULL, *local_basename; char *result = NULL; if (*p == '"' || *p == '\'') { len -= 2; p++; } url = xstrndup (p, len); org_basename = strrchr (url, '/'); if (org_basename) org_basename++; else org_basename = url; local_basename = link->local_name ? strrchr (link->local_name, '/') : NULL; if (local_basename) local_basename++; else local_basename = url; /* * If the basenames differ, graft the adjusted basename (local_basename) * onto the original URL. */ if (strcmp (org_basename, local_basename) == 0) result = url; else { result = uri_merge (url, local_basename); xfree (url); } return result; } /* Used by write_backup_file to remember which files have been written. */ static struct hash_table *converted_files; static void write_backup_file (const char *file, downloaded_file_t downloaded_file_return) { /* Rather than just writing over the original .html file with the converted version, save the former to *.orig. Note we only do this for files we've _successfully_ downloaded, so we don't clobber .orig files sitting around from previous invocations. On VMS, use "_orig" instead of ".orig". See "wget.h". */ if (!converted_files) converted_files = make_string_hash_table (0); /* We can get called twice on the same URL thanks to the convert_all_links() call in main. If we write the .orig file each time in such a case, it'll end up containing the first-pass conversion, not the original file. So, see if we've already been called on this file. */ if (!string_set_contains (converted_files, file)) { /* Construct the backup filename as the original name plus ".orig". */ char buf[1024]; size_t filename_len = strlen (file); char *filename_plus_orig_suffix; if (filename_len < sizeof (buf) - 5) filename_plus_orig_suffix = buf; else filename_plus_orig_suffix = xmalloc (filename_len + 5 + 1); /* TODO: hack this to work with css files */ if (downloaded_file_return == FILE_DOWNLOADED_AND_HTML_EXTENSION_ADDED) { /* Just write "orig" over "html". We need to do it this way because when we're checking to see if we've downloaded the file before (to see if we can skip downloading it), we don't know if it's a text/html file. Therefore we don't know yet at that stage that -E is going to cause us to tack on ".html", so we need to compare vs. the original URL plus ".orig", not the original URL plus ".html.orig". */ memcpy (filename_plus_orig_suffix, file, filename_len - 4); memcpy (filename_plus_orig_suffix + filename_len - 4, "orig", 5); } else /* downloaded_file_return == FILE_DOWNLOADED_NORMALLY */ { /* Append ".orig" to the name. */ memcpy (filename_plus_orig_suffix, file, filename_len); strcpy (filename_plus_orig_suffix + filename_len, ORIG_SFX); } /* Rename to .orig before former gets written over. */ if (rename (file, filename_plus_orig_suffix) != 0) logprintf (LOG_NOTQUIET, _("Cannot back up %s as %s: %s\n"), file, filename_plus_orig_suffix, strerror (errno)); if (filename_plus_orig_suffix != buf) xfree (filename_plus_orig_suffix); /* Remember that we've already written a .orig backup for this file. Note that we never free this memory since we need it till the convert_all_links() call, which is one of the last things the program does before terminating. BTW, I'm not sure if it would be safe to just set 'converted_file_ptr->string' to 'file' below, rather than making a copy of the string... Another note is that I thought I could just add a field to the urlpos structure saying that we'd written a .orig file for this URL, but that didn't work, so I had to make this separate list. -- Dan Harkless This [adding a field to the urlpos structure] didn't work because convert_file() is called from convert_all_links at the end of the retrieval with a freshly built new urlpos list. -- Hrvoje Niksic */ string_set_add (converted_files, file); } } static bool find_fragment (const char *, int, const char **, const char **); /* Replace a string with NEW_TEXT. Ignore quoting. */ static const char * replace_plain (const char *p, int size, FILE *fp, const char *new_text) { fputs (new_text, fp); p += size; return p; } /* Replace an attribute's original text with NEW_TEXT. */ static const char * replace_attr (const char *p, int size, FILE *fp, const char *new_text) { bool quote_flag = false; char quote_char = '\"'; /* use "..." for quoting, unless the original value is quoted, in which case reuse its quoting char. */ const char *frag_beg, *frag_end; /* Structure of our string is: "...old-contents..." <--- size ---> (with quotes) OR: ...old-contents... <--- size --> (no quotes) */ if (*p == '\"' || *p == '\'') { quote_char = *p; quote_flag = true; ++p; size -= 2; /* disregard opening and closing quote */ } putc (quote_char, fp); fputs (new_text, fp); /* Look for fragment identifier, if any. */ if (find_fragment (p, size, &frag_beg, &frag_end)) fwrite (frag_beg, 1, frag_end - frag_beg, fp); p += size; if (quote_flag) ++p; putc (quote_char, fp); return p; } /* The same as REPLACE_ATTR, but used when replacing because we need to append "timeout_value; URL=" before the next_text. */ static const char * replace_attr_refresh_hack (const char *p, int size, FILE *fp, const char *new_text, int timeout) { /* "0; URL=..." */ char new_with_timeout[1024]; if (((unsigned) snprintf ( new_with_timeout, sizeof (new_with_timeout), "%d; URL=%s", timeout, new_text)) >= sizeof (new_with_timeout)) { // very unlikely fallback using heap memory char *tmp = aprintf("%d; URL=%s", timeout, new_text); const char *res = replace_attr (p, size, fp, tmp); xfree (tmp); return res; } return replace_attr (p, size, fp, new_with_timeout); } /* Find the first occurrence of '#' in [BEG, BEG+SIZE) that is not preceded by '&'. If the character is not found, return zero. If the character is found, return true and set BP and EP to point to the beginning and end of the region. This is used for finding the fragment indentifiers in URLs. */ static bool find_fragment (const char *beg, int size, const char **bp, const char **ep) { const char *end = beg + size; bool saw_amp = false; for (; beg < end; beg++) { switch (*beg) { case '&': saw_amp = true; break; case '#': if (!saw_amp) { *bp = beg; *ep = end; return true; } /* fallthrough */ default: saw_amp = false; } } return false; } /* Quote FILE for use as local reference to an HTML file. We quote ? as %3F to avoid passing part of the file name as the parameter when browsing the converted file through HTTP. However, it is safe to do this only when `--adjust-extension' is turned on. This is because converting "index.html?foo=bar" to "index.html%3Ffoo=bar" would break local browsing, as the latter isn't even recognized as an HTML file! However, converting "index.html?foo=bar.html" to "index.html%3Ffoo=bar.html" should be safe for both local and HTTP-served browsing. We always quote "#" as "%23", "%" as "%25" and ";" as "%3B" because those characters have special meanings in URLs. */ static char * local_quote_string (const char *file, bool no_html_quote) { const char *from; char *newname, *to, *res; char buf[1024]; size_t tolen; char *any = strpbrk (file, "?#%;"); if (!any) return no_html_quote ? strdup (file) : html_quote_string (file); /* Allocate space assuming the worst-case scenario, each character having to be quoted. */ tolen = 3 * strlen (file); if (tolen < sizeof (buf)) to = newname = buf; else to = newname = xmalloc (tolen + 1); for (from = file; *from; from++) switch (*from) { case '%': *to++ = '%'; *to++ = '2'; *to++ = '5'; break; case '#': *to++ = '%'; *to++ = '2'; *to++ = '3'; break; case ';': *to++ = '%'; *to++ = '3'; *to++ = 'B'; break; case '?': if (opt.adjust_extension) { *to++ = '%'; *to++ = '3'; *to++ = 'F'; break; } /* fallthrough */ default: *to++ = *from; } *to = '\0'; if (newname == buf) return no_html_quote ? strdup (newname) : html_quote_string (newname); if (no_html_quote) return newname; res = html_quote_string (newname); xfree (newname); return res; } /* Book-keeping code for dl_file_url_map, dl_url_file_map, downloaded_html_list, and downloaded_html_set. Other code calls these functions to let us know that a file has been downloaded. */ #define ENSURE_TABLES_EXIST do { \ if (!dl_file_url_map) \ dl_file_url_map = make_string_hash_table (0); \ if (!dl_url_file_map) \ dl_url_file_map = make_string_hash_table (0); \ } while (0) /* Return true if S1 and S2 are the same, except for "/index.html". The three cases in which it returns one are (substitute any substring for "foo"): m("foo/index.html", "foo/") ==> 1 m("foo/", "foo/index.html") ==> 1 m("foo", "foo/index.html") ==> 1 m("foo", "foo/" ==> 1 m("foo", "foo") ==> 1 */ static bool match_except_index (const char *s1, const char *s2) { int i; const char *lng; /* Skip common substring. */ for (i = 0; *s1 && *s2 && *s1 == *s2; s1++, s2++, i++) ; if (i == 0) /* Strings differ at the very beginning -- bail out. We need to check this explicitly to avoid `lng - 1' reading outside the array. */ return false; if (!*s1 && !*s2) /* Both strings hit EOF -- strings are equal. */ return true; else if (*s1 && *s2) /* Strings are randomly different, e.g. "/foo/bar" and "/foo/qux". */ return false; else if (*s1) /* S1 is the longer one. */ lng = s1; else /* S2 is the longer one. */ lng = s2; /* foo */ /* foo/ */ /* foo/index.html */ /* or */ /* foo/index.html */ /* ^ */ /* ^ */ if (*lng != '/') /* The right-hand case. */ --lng; if (*lng == '/' && *(lng + 1) == '\0') /* foo */ /* foo/ */ return true; return 0 == strcmp (lng, "/index.html"); } static int dissociate_urls_from_file_mapper (void *key, void *value, void *arg) { char *mapping_url = (char *)key; char *mapping_file = (char *)value; char *file = (char *)arg; if (0 == strcmp (mapping_file, file)) { hash_table_remove (dl_url_file_map, mapping_url); xfree (mapping_url); xfree (mapping_file); } /* Continue mapping. */ return 0; } /* Remove all associations from various URLs to FILE from dl_url_file_map. */ static void dissociate_urls_from_file (const char *file) { /* Can't use hash_table_iter_* because the table mutates while mapping. */ hash_table_for_each (dl_url_file_map, dissociate_urls_from_file_mapper, (char *) file); } /* Register that URL has been successfully downloaded to FILE. This is used by the link conversion code to convert references to URLs to references to local files. It is also being used to check if a URL has already been downloaded. */ void register_download (const char *url, const char *file) { char *old_file, *old_url; ENSURE_TABLES_EXIST; /* With some forms of retrieval, it is possible, although not likely or particularly desirable. If both are downloaded, the second download will override the first one. When that happens, dissociate the old file name from the URL. */ if (hash_table_get_pair (dl_file_url_map, file, &old_file, &old_url)) { if (0 == strcmp (url, old_url)) /* We have somehow managed to download the same URL twice. Nothing to do. */ return; if (match_except_index (url, old_url) && !hash_table_contains (dl_url_file_map, url)) /* The two URLs differ only in the "index.html" ending. For example, one is "http://www.server.com/", and the other is "http://www.server.com/index.html". Don't remove the old one, just add the new one as a non-canonical entry. */ goto url_only; hash_table_remove (dl_file_url_map, file); xfree (old_file); xfree (old_url); /* Remove all the URLs that point to this file. Yes, there can be more than one such URL, because we store redirections as multiple entries in dl_url_file_map. For example, if URL1 redirects to URL2 which gets downloaded to FILE, we map both URL1 and URL2 to FILE in dl_url_file_map. (dl_file_url_map only points to URL2.) When another URL gets loaded to FILE, we want both URL1 and URL2 dissociated from it. This is a relatively expensive operation because it performs a linear search of the whole hash table, but it should be called very rarely, only when two URLs resolve to the same file name, *and* the ".1" extensions are turned off. In other words, almost never. */ dissociate_urls_from_file (file); } hash_table_put (dl_file_url_map, xstrdup (file), xstrdup (url)); url_only: /* A URL->FILE mapping is not possible without a FILE->URL mapping. If the latter were present, it should have been removed by the above `if'. So we could write: assert (!hash_table_contains (dl_url_file_map, url)); The above is correct when running in recursive mode where the same URL always resolves to the same file. But if you do something like: wget URL URL then the first URL will resolve to "FILE", and the other to "FILE.1". In that case, FILE.1 will not be found in dl_file_url_map, but URL will still point to FILE in dl_url_file_map. */ if (hash_table_get_pair (dl_url_file_map, url, &old_url, &old_file)) { hash_table_remove (dl_url_file_map, url); xfree (old_url); xfree (old_file); } hash_table_put (dl_url_file_map, xstrdup (url), xstrdup (file)); } /* Register that FROM has been redirected to "TO". This assumes that TO is successfully downloaded and already registered using register_download() above. */ void register_redirection (const char *from, const char *to) { char *file; ENSURE_TABLES_EXIST; file = hash_table_get (dl_url_file_map, to); assert (file != NULL); if (!hash_table_contains (dl_url_file_map, from)) hash_table_put (dl_url_file_map, xstrdup (from), xstrdup (file)); } /* Register that the file has been deleted. */ void register_delete_file (const char *file) { char *old_url, *old_file; ENSURE_TABLES_EXIST; if (!hash_table_get_pair (dl_file_url_map, file, &old_file, &old_url)) return; hash_table_remove (dl_file_url_map, file); xfree (old_file); xfree (old_url); dissociate_urls_from_file (file); } /* Register that FILE is an HTML file that has been downloaded. */ void register_html (const char *file) { if (!downloaded_html_set) downloaded_html_set = make_string_hash_table (0); string_set_add (downloaded_html_set, file); } /* Register that FILE is a CSS file that has been downloaded. */ void register_css (const char *file) { if (!downloaded_css_set) downloaded_css_set = make_string_hash_table (0); string_set_add (downloaded_css_set, file); } /* Cleanup the data structures associated with this file. */ #if defined DEBUG_MALLOC || defined TESTING static void downloaded_files_free (void); void convert_cleanup (void) { if (dl_file_url_map) { free_keys_and_values (dl_file_url_map); hash_table_destroy (dl_file_url_map); dl_file_url_map = NULL; } if (dl_url_file_map) { free_keys_and_values (dl_url_file_map); hash_table_destroy (dl_url_file_map); dl_url_file_map = NULL; } if (downloaded_html_set) string_set_free (downloaded_html_set); if (downloaded_css_set) string_set_free (downloaded_css_set); downloaded_files_free (); if (converted_files) string_set_free (converted_files); } #endif /* Book-keeping code for downloaded files that enables extension hacks. */ /* This table should really be merged with dl_file_url_map and downloaded_html_files. This was originally a list, but I changed it to a hash table because it was actually taking a lot of time to find things in it. */ static struct hash_table *downloaded_files_hash; /* We're storing "modes" of type downloaded_file_t in the hash table. However, our hash tables only accept pointers for keys and values. So when we need a pointer, we use the address of a downloaded_file_t variable of static storage. */ static downloaded_file_t * downloaded_mode_to_ptr (downloaded_file_t mode) { static downloaded_file_t v1 = FILE_NOT_ALREADY_DOWNLOADED, v2 = FILE_DOWNLOADED_NORMALLY, v3 = FILE_DOWNLOADED_AND_HTML_EXTENSION_ADDED, v4 = CHECK_FOR_FILE; switch (mode) { case FILE_NOT_ALREADY_DOWNLOADED: return &v1; case FILE_DOWNLOADED_NORMALLY: return &v2; case FILE_DOWNLOADED_AND_HTML_EXTENSION_ADDED: return &v3; case CHECK_FOR_FILE: return &v4; } return NULL; } /* Remembers which files have been downloaded. In the standard case, should be called with mode == FILE_DOWNLOADED_NORMALLY for each file we actually download successfully (i.e. not for ones we have failures on or that we skip due to -N). When we've downloaded a file and tacked on a ".html" extension due to -E, call this function with FILE_DOWNLOADED_AND_HTML_EXTENSION_ADDED rather than FILE_DOWNLOADED_NORMALLY. If you just want to check if a file has been previously added without adding it, call with mode == CHECK_FOR_FILE. Please be sure to call this function with local filenames, not remote URLs. */ downloaded_file_t downloaded_file (downloaded_file_t mode, const char *file) { downloaded_file_t *ptr; if (mode == CHECK_FOR_FILE) { if (!downloaded_files_hash) return FILE_NOT_ALREADY_DOWNLOADED; ptr = hash_table_get (downloaded_files_hash, file); if (!ptr) return FILE_NOT_ALREADY_DOWNLOADED; return *ptr; } if (!downloaded_files_hash) downloaded_files_hash = make_string_hash_table (0); ptr = hash_table_get (downloaded_files_hash, file); if (ptr) return *ptr; ptr = downloaded_mode_to_ptr (mode); hash_table_put (downloaded_files_hash, xstrdup (file), ptr); return FILE_NOT_ALREADY_DOWNLOADED; } #if defined DEBUG_MALLOC || defined TESTING static void downloaded_files_free (void) { if (downloaded_files_hash) { hash_table_iterator iter; for (hash_table_iterate (downloaded_files_hash, &iter); hash_table_iter_next (&iter); ) xfree (iter.key); hash_table_destroy (downloaded_files_hash); downloaded_files_hash = NULL; } } #endif /* The function returns the pointer to the malloc-ed quoted version of string s. It will recognize and quote numeric and special graphic entities, as per RFC1866: `&' -> `&' `<' -> `<' `>' -> `>' `"' -> `"' SP -> ` ' No other entities are recognized or replaced. */ char * html_quote_string (const char *s) { const char *b = s; char *p, *res; int i; /* Pass through the string, and count the new size. */ for (i = 0; *s; s++, i++) { if (*s == '&') i += 4; /* `amp;' */ else if (*s == '<' || *s == '>') i += 3; /* `lt;' and `gt;' */ else if (*s == '\"') i += 5; /* `quot;' */ else if (*s == ' ') i += 4; /* #32; */ } res = xmalloc (i + 1); s = b; for (p = res; *s; s++) { switch (*s) { case '&': *p++ = '&'; *p++ = 'a'; *p++ = 'm'; *p++ = 'p'; *p++ = ';'; break; case '<': case '>': *p++ = '&'; *p++ = (*s == '<' ? 'l' : 'g'); *p++ = 't'; *p++ = ';'; break; case '\"': *p++ = '&'; *p++ = 'q'; *p++ = 'u'; *p++ = 'o'; *p++ = 't'; *p++ = ';'; break; case ' ': *p++ = '&'; *p++ = '#'; *p++ = '3'; *p++ = '2'; *p++ = ';'; break; default: *p++ = *s; } } *p = '\0'; return res; } /* * vim: et ts=2 sw=2 */ wget-1.21.2/src/version.h0000644000000000000000000000313714115732710012064 00000000000000/* Extern declarations for printing version information Copyright (C) 2013, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #ifndef WGET_VERSION_H #define WGET_VERSION_H /* Extern declarations for strings in version.c */ extern const char *version_string; extern const char *compilation_string; extern const char *link_string; /* Extern declaration for string in build_info.c */ extern const char *compiled_features[]; #endif /* WGET_VERSION_H */ wget-1.21.2/src/css-tokens.h0000644000000000000000000000340114115732710012462 00000000000000/* Declarations for css.lex Copyright (C) 2006, 2009-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #ifndef CSS_TOKENS_H #define CSS_TOKENS_H enum { CSSEOF = 0, S = 1, CDO = 2, CDC = 3, INCLUDES = 4, DASHMATCH = 5, STRING = 6, BAD_STRING = 7, IDENT = 8, HASH = 9, IMPORT_SYM = 10, PAGE_SYM = 11, MEDIA_SYM = 12, CHARSET_SYM = 13, IMPORTANT_SYM = 14, EMS = 15, EXS = 16, LENGTH = 17, ANGLE = 18, TIME = 19, FREQ = 20, DIMENSION = 21, PERCENTAGE = 22, NUMBER = 23, URI = 24, BAD_URI = 25, FUNCTION = 26, COMMENT = 27 }; #endif /* CSS_TOKENS_H */ wget-1.21.2/src/css.c0000644000000000000000000050124214115733377011174 00000000000000#line 1 "css.c" /* config.h must precede flex's inclusion of in order for its _GNU_SOURCE definition to take effect. */ #include #line 6 "css.c" #define YY_INT_ALIGNED short int /* A lexical scanner generated by flex */ #define FLEX_SCANNER #define YY_FLEX_MAJOR_VERSION 2 #define YY_FLEX_MINOR_VERSION 6 #define YY_FLEX_SUBMINOR_VERSION 4 #if YY_FLEX_SUBMINOR_VERSION > 0 #define FLEX_BETA #endif /* First, we deal with platform-specific or compiler-specific issues. */ /* begin standard C headers. */ #include #include #include #include /* end standard C headers. */ /* flex integer type definitions */ #ifndef FLEXINT_H #define FLEXINT_H /* C99 systems have . Non-C99 systems may or may not. */ #if defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L /* C99 says to define __STDC_LIMIT_MACROS before including stdint.h, * if you want the limit (max/min) macros for int types. */ #ifndef __STDC_LIMIT_MACROS #define __STDC_LIMIT_MACROS 1 #endif #include typedef int8_t flex_int8_t; typedef uint8_t flex_uint8_t; typedef int16_t flex_int16_t; typedef uint16_t flex_uint16_t; typedef int32_t flex_int32_t; typedef uint32_t flex_uint32_t; #else typedef signed char flex_int8_t; typedef short int flex_int16_t; typedef int flex_int32_t; typedef unsigned char flex_uint8_t; typedef unsigned short int flex_uint16_t; typedef unsigned int flex_uint32_t; /* Limits of integral types. */ #ifndef INT8_MIN #define INT8_MIN (-128) #endif #ifndef INT16_MIN #define INT16_MIN (-32767-1) #endif #ifndef INT32_MIN #define INT32_MIN (-2147483647-1) #endif #ifndef INT8_MAX #define INT8_MAX (127) #endif #ifndef INT16_MAX #define INT16_MAX (32767) #endif #ifndef INT32_MAX #define INT32_MAX (2147483647) #endif #ifndef UINT8_MAX #define UINT8_MAX (255U) #endif #ifndef UINT16_MAX #define UINT16_MAX (65535U) #endif #ifndef UINT32_MAX #define UINT32_MAX (4294967295U) #endif #ifndef SIZE_MAX #define SIZE_MAX (~(size_t)0) #endif #endif /* ! C99 */ #endif /* ! FLEXINT_H */ /* begin standard C++ headers. */ /* TODO: this is always defined, so inline it */ #define yyconst const #if defined(__GNUC__) && __GNUC__ >= 3 #define yynoreturn __attribute__((__noreturn__)) #else #define yynoreturn #endif /* Returned upon end-of-file. */ #define YY_NULL 0 /* Promotes a possibly negative, possibly signed char to an * integer in range [0..255] for use as an array index. */ #define YY_SC_TO_UI(c) ((YY_CHAR) (c)) /* Enter a start condition. This macro really ought to take a parameter, * but we do it the disgusting crufty way forced on us by the ()-less * definition of BEGIN. */ #define BEGIN (yy_start) = 1 + 2 * /* Translate the current start state into a value that can be later handed * to BEGIN to return to the state. The YYSTATE alias is for lex * compatibility. */ #define YY_START (((yy_start) - 1) / 2) #define YYSTATE YY_START /* Action number for EOF rule of a given start state. */ #define YY_STATE_EOF(state) (YY_END_OF_BUFFER + state + 1) /* Special action meaning "start processing a new file". */ #define YY_NEW_FILE yyrestart( yyin ) #define YY_END_OF_BUFFER_CHAR 0 /* Size of default input buffer. */ #ifndef YY_BUF_SIZE #ifdef __ia64__ /* On IA-64, the buffer size is 16k, not 8k. * Moreover, YY_BUF_SIZE is 2*YY_READ_BUF_SIZE in the general case. * Ditto for the __ia64__ case accordingly. */ #define YY_BUF_SIZE 32768 #else #define YY_BUF_SIZE 16384 #endif /* __ia64__ */ #endif /* The state buf must be large enough to hold one state per character in the main buffer. */ #define YY_STATE_BUF_SIZE ((YY_BUF_SIZE + 2) * sizeof(yy_state_type)) #ifndef YY_TYPEDEF_YY_BUFFER_STATE #define YY_TYPEDEF_YY_BUFFER_STATE typedef struct yy_buffer_state *YY_BUFFER_STATE; #endif #ifndef YY_TYPEDEF_YY_SIZE_T #define YY_TYPEDEF_YY_SIZE_T typedef size_t yy_size_t; #endif extern int yyleng; extern FILE *yyin, *yyout; #define EOB_ACT_CONTINUE_SCAN 0 #define EOB_ACT_END_OF_FILE 1 #define EOB_ACT_LAST_MATCH 2 #define YY_LESS_LINENO(n) #define YY_LINENO_REWIND_TO(ptr) /* Return all but the first "n" matched characters back to the input stream. */ #define yyless(n) \ do \ { \ /* Undo effects of setting up yytext. */ \ int yyless_macro_arg = (n); \ YY_LESS_LINENO(yyless_macro_arg);\ *yy_cp = (yy_hold_char); \ YY_RESTORE_YY_MORE_OFFSET \ (yy_c_buf_p) = yy_cp = yy_bp + yyless_macro_arg - YY_MORE_ADJ; \ YY_DO_BEFORE_ACTION; /* set up yytext again */ \ } \ while ( 0 ) #define unput(c) yyunput( c, (yytext_ptr) ) #ifndef YY_STRUCT_YY_BUFFER_STATE #define YY_STRUCT_YY_BUFFER_STATE struct yy_buffer_state { FILE *yy_input_file; char *yy_ch_buf; /* input buffer */ char *yy_buf_pos; /* current position in input buffer */ /* Size of input buffer in bytes, not including room for EOB * characters. */ int yy_buf_size; /* Number of characters read into yy_ch_buf, not including EOB * characters. */ int yy_n_chars; /* Whether we "own" the buffer - i.e., we know we created it, * and can realloc() it to grow it, and should free() it to * delete it. */ int yy_is_our_buffer; /* Whether this is an "interactive" input source; if so, and * if we're using stdio for input, then we want to use getc() * instead of fread(), to make sure we stop fetching input after * each newline. */ int yy_is_interactive; /* Whether we're considered to be at the beginning of a line. * If so, '^' rules will be active on the next match, otherwise * not. */ int yy_at_bol; int yy_bs_lineno; /**< The line count. */ int yy_bs_column; /**< The column count. */ /* Whether to try to fill the input buffer when we reach the * end of it. */ int yy_fill_buffer; int yy_buffer_status; #define YY_BUFFER_NEW 0 #define YY_BUFFER_NORMAL 1 /* When an EOF's been seen but there's still some text to process * then we mark the buffer as YY_EOF_PENDING, to indicate that we * shouldn't try reading from the input source any more. We might * still have a bunch of tokens to match, though, because of * possible backing-up. * * When we actually see the EOF, we change the status to "new" * (via yyrestart()), so that the user can continue scanning by * just pointing yyin at a new input file. */ #define YY_BUFFER_EOF_PENDING 2 }; #endif /* !YY_STRUCT_YY_BUFFER_STATE */ /* Stack of input buffers. */ static size_t yy_buffer_stack_top = 0; /**< index of top of stack. */ static size_t yy_buffer_stack_max = 0; /**< capacity of stack. */ static YY_BUFFER_STATE * yy_buffer_stack = NULL; /**< Stack as an array. */ /* We provide macros for accessing buffer states in case in the * future we want to put the buffer states in a more general * "scanner state". * * Returns the top of the stack, or NULL. */ #define YY_CURRENT_BUFFER ( (yy_buffer_stack) \ ? (yy_buffer_stack)[(yy_buffer_stack_top)] \ : NULL) /* Same as previous macro, but useful when we know that the buffer stack is not * NULL or when we need an lvalue. For internal use only. */ #define YY_CURRENT_BUFFER_LVALUE (yy_buffer_stack)[(yy_buffer_stack_top)] /* yy_hold_char holds the character lost when yytext is formed. */ static char yy_hold_char; static int yy_n_chars; /* number of characters read into yy_ch_buf */ int yyleng; /* Points to current character in buffer. */ static char *yy_c_buf_p = NULL; static int yy_init = 0; /* whether we need to initialize */ static int yy_start = 0; /* start state number */ /* Flag which is used to allow yywrap()'s to do buffer switches * instead of setting up a fresh yyin. A bit of a hack ... */ static int yy_did_buffer_switch_on_eof; void yyrestart ( FILE *input_file ); void yy_switch_to_buffer ( YY_BUFFER_STATE new_buffer ); YY_BUFFER_STATE yy_create_buffer ( FILE *file, int size ); void yy_delete_buffer ( YY_BUFFER_STATE b ); void yy_flush_buffer ( YY_BUFFER_STATE b ); void yypush_buffer_state ( YY_BUFFER_STATE new_buffer ); void yypop_buffer_state ( void ); static void yyensure_buffer_stack ( void ); static void yy_load_buffer_state ( void ); static void yy_init_buffer ( YY_BUFFER_STATE b, FILE *file ); #define YY_FLUSH_BUFFER yy_flush_buffer( YY_CURRENT_BUFFER ) YY_BUFFER_STATE yy_scan_buffer ( char *base, yy_size_t size ); YY_BUFFER_STATE yy_scan_string ( const char *yy_str ); YY_BUFFER_STATE yy_scan_bytes ( const char *bytes, int len ); void *yyalloc ( yy_size_t ); void *yyrealloc ( void *, yy_size_t ); void yyfree ( void * ); #define yy_new_buffer yy_create_buffer #define yy_set_interactive(is_interactive) \ { \ if ( ! YY_CURRENT_BUFFER ){ \ yyensure_buffer_stack (); \ YY_CURRENT_BUFFER_LVALUE = \ yy_create_buffer( yyin, YY_BUF_SIZE ); \ } \ YY_CURRENT_BUFFER_LVALUE->yy_is_interactive = is_interactive; \ } #define yy_set_bol(at_bol) \ { \ if ( ! YY_CURRENT_BUFFER ){\ yyensure_buffer_stack (); \ YY_CURRENT_BUFFER_LVALUE = \ yy_create_buffer( yyin, YY_BUF_SIZE ); \ } \ YY_CURRENT_BUFFER_LVALUE->yy_at_bol = at_bol; \ } #define YY_AT_BOL() (YY_CURRENT_BUFFER_LVALUE->yy_at_bol) /* Begin user sect3 */ #define yywrap() (/*CONSTCOND*/1) #define YY_SKIP_YYWRAP typedef flex_uint8_t YY_CHAR; FILE *yyin = NULL, *yyout = NULL; typedef int yy_state_type; extern int yylineno; int yylineno = 1; extern char *yytext; #ifdef yytext_ptr #undef yytext_ptr #endif #define yytext_ptr yytext static yy_state_type yy_get_previous_state ( void ); static yy_state_type yy_try_NUL_trans ( yy_state_type current_state ); static int yy_get_next_buffer ( void ); static void yynoreturn yy_fatal_error ( const char* msg ); /* Done after the current pattern has been matched and before the * corresponding action - sets up yytext. */ #define YY_DO_BEFORE_ACTION \ (yytext_ptr) = yy_bp; \ yyleng = (int) (yy_cp - yy_bp); \ (yy_hold_char) = *yy_cp; \ *yy_cp = '\0'; \ (yy_c_buf_p) = yy_cp; #define YY_NUM_RULES 41 #define YY_END_OF_BUFFER 42 /* This struct is not used in this scanner, but its presence is necessary. */ struct yy_trans_info { flex_int32_t yy_verify; flex_int32_t yy_nxt; }; static const flex_int16_t yy_accept[1103] = { 0, 0, 0, 42, 40, 1, 1, 40, 10, 40, 10, 40, 40, 40, 35, 40, 40, 11, 11, 40, 40, 40, 1, 0, 0, 0, 0, 10, 9, 10, 12, 0, 0, 10, 10, 0, 11, 0, 35, 4, 34, 0, 0, 35, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 30, 0, 0, 0, 0, 0, 0, 0, 39, 11, 0, 11, 11, 11, 8, 7, 0, 0, 0, 0, 0, 0, 0, 10, 10, 10, 0, 12, 12, 10, 10, 10, 6, 4, 4, 0, 33, 0, 21, 0, 33, 0, 18, 19, 0, 33, 0, 31, 0, 23, 0, 33, 0, 22, 29, 0, 25, 24, 20, 0, 33, 0, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 30, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 11, 11, 11, 11, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 10, 10, 0, 0, 12, 12, 10, 10, 10, 4, 2, 33, 33, 33, 33, 33, 21, 26, 0, 33, 33, 33, 33, 33, 33, 33, 33, 18, 19, 33, 0, 33, 33, 33, 33, 33, 33, 33, 31, 33, 33, 33, 23, 32, 0, 33, 33, 33, 33, 33, 33, 33, 33, 33, 22, 29, 33, 33, 33, 33, 33, 24, 20, 27, 0, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 30, 33, 33, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 11, 11, 38, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 3, 12, 10, 4, 4, 33, 33, 33, 33, 33, 21, 21, 33, 33, 33, 26, 33, 33, 33, 33, 33, 33, 33, 33, 33, 18, 19, 18, 28, 0, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 31, 31, 33, 33, 33, 23, 23, 33, 33, 33, 32, 33, 33, 33, 33, 33, 33, 33, 33, 33, 22, 29, 22, 33, 33, 33, 33, 33, 25, 24, 20, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 30, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 25, 33, 33, 33, 30, 30, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 14, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 11, 38, 38, 38, 38, 37, 0, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 12, 10, 33, 33, 33, 33, 21, 21, 21, 21, 33, 33, 33, 26, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 18, 19, 18, 18, 18, 19, 19, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 31, 31, 31, 31, 33, 33, 33, 23, 23, 23, 23, 33, 33, 33, 32, 32, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 22, 29, 22, 22, 22, 29, 29, 33, 33, 33, 33, 33, 25, 24, 20, 25, 25, 24, 24, 20, 20, 33, 33, 33, 27, 33, 33, 33, 33, 33, 33, 27, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 30, 33, 33, 33, 25, 33, 27, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 15, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 11, 38, 38, 38, 38, 38, 38, 38, 38, 0, 38, 37, 38, 38, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 12, 10, 33, 33, 33, 33, 21, 21, 33, 33, 33, 26, 26, 26, 33, 33, 33, 33, 33, 33, 33, 33, 33, 18, 19, 18, 33, 33, 33, 28, 33, 33, 33, 33, 33, 33, 28, 33, 33, 33, 33, 33, 28, 33, 33, 33, 31, 31, 33, 33, 33, 23, 23, 33, 33, 33, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33, 33, 33, 22, 29, 22, 33, 33, 33, 33, 33, 25, 24, 20, 33, 33, 33, 27, 27, 27, 33, 33, 33, 33, 27, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 30, 33, 33, 33, 25, 33, 27, 0, 13, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 14, 0, 0, 0, 0, 11, 38, 36, 38, 38, 38, 38, 38, 38, 38, 38, 38, 0, 38, 38, 37, 38, 0, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 12, 10, 33, 33, 33, 21, 21, 33, 33, 33, 26, 33, 33, 33, 33, 33, 33, 33, 18, 19, 18, 33, 33, 33, 28, 28, 28, 33, 33, 33, 33, 33, 33, 33, 33, 28, 33, 33, 31, 31, 33, 33, 23, 23, 33, 33, 33, 32, 32, 33, 33, 33, 33, 33, 33, 33, 22, 29, 22, 33, 33, 33, 33, 25, 24, 20, 33, 33, 33, 27, 33, 33, 33, 27, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 30, 33, 33, 33, 25, 33, 27, 0, 0, 0, 0, 13, 0, 0, 0, 0, 0, 0, 0, 15, 0, 0, 0, 0, 0, 14, 14, 0, 11, 38, 38, 38, 38, 38, 38, 38, 0, 0, 38, 38, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 33, 21, 21, 33, 33, 26, 33, 18, 19, 18, 33, 33, 33, 28, 33, 33, 33, 33, 33, 28, 31, 31, 23, 23, 33, 33, 32, 32, 33, 22, 29, 22, 25, 24, 20, 33, 33, 27, 33, 27, 16, 0, 13, 0, 0, 0, 0, 0, 15, 15, 0, 0, 38, 38, 38, 0, 0, 0, 0, 38, 17, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 26, 33, 33, 28, 33, 32, 32, 27, 0, 13, 13, 0, 0, 38, 38, 38, 0, 0, 38, 0, 0, 0, 17, 0, 0, 0, 0, 0, 0, 0, 28, 0, 38, 38, 38, 0, 38, 0, 17, 0, 0, 0, 0, 38, 38, 0, 38, 0, 17, 17, 0, 0, 0, 0, 0 } ; static const YY_CHAR yy_ec[256] = { 0, 1, 1, 1, 1, 1, 1, 1, 1, 2, 3, 1, 4, 5, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 6, 7, 8, 9, 10, 11, 10, 12, 13, 14, 15, 10, 10, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 10, 10, 29, 30, 31, 10, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 42, 49, 50, 51, 52, 42, 42, 53, 42, 54, 10, 55, 10, 10, 42, 10, 56, 57, 58, 59, 60, 61, 62, 63, 64, 42, 65, 66, 67, 68, 69, 70, 42, 71, 72, 73, 74, 42, 42, 75, 42, 76, 10, 77, 10, 78, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79, 79 } ; static const YY_CHAR yy_meta[80] = { 0, 1, 2, 3, 3, 3, 2, 2, 4, 2, 2, 2, 4, 5, 2, 2, 6, 2, 7, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 2, 2, 2, 2, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 2, 2, 9 } ; static const flex_int16_t yy_base[1137] = { 0, 0, 0, 4195, 7110, 78, 83, 88, 87, 78, 85, 82, 88, 4161, 142, 4151, 90, 86, 206, 259, 4118, 4083, 98, 234, 4083, 72, 109, 116, 7110, 318, 100, 4082, 361, 208, 420, 4017, 92, 463, 205, 4024, 7110, 3977, 222, 0, 3974, 209, 89, 257, 202, 180, 245, 248, 259, 355, 272, 3955, 524, 3987, 83, 280, 311, 274, 585, 7110, 117, 637, 348, 210, 697, 7110, 7110, 3963, 302, 382, 243, 3940, 3933, 371, 251, 356, 757, 3945, 101, 817, 246, 357, 877, 7110, 3938, 252, 920, 3894, 974, 3886, 1017, 397, 447, 3861, 3860, 661, 376, 904, 3850, 721, 3841, 448, 451, 1040, 3840, 3825, 1045, 3813, 3796, 3787, 1063, 569, 600, 3784, 1137, 484, 1180, 613, 1221, 748, 380, 566, 347, 591, 907, 654, 462, 3767, 3789, 118, 559, 779, 633, 593, 588, 628, 782, 654, 3776, 775, 3775, 802, 772, 682, 336, 1279, 345, 447, 1322, 3754, 254, 695, 837, 658, 581, 632, 656, 806, 1012, 456, 862, 1365, 3744, 256, 903, 1408, 278, 964, 1451, 3730, 7110, 3689, 1511, 839, 751, 790, 3655, 3633, 1086, 865, 3648, 3645, 868, 812, 3627, 850, 3599, 3559, 3554, 895, 897, 344, 3587, 3580, 885, 900, 892, 922, 3526, 1001, 929, 943, 3484, 3483, 1165, 1002, 3500, 3499, 1037, 1074, 945, 3485, 969, 3476, 3439, 3424, 1096, 3450, 410, 3442, 438, 3403, 3397, 3396, 1109, 1104, 3416, 3396, 745, 1554, 1126, 1597, 1199, 1638, 1237, 1696, 1591, 1451, 1224, 1636, 1495, 1673, 1680, 1762, 1836, 1318, 1740, 1703, 7110, 573, 1049, 1161, 1158, 970, 987, 1131, 1017, 1192, 1245, 3385, 3371, 1046, 1269, 1252, 3369, 3358, 1342, 1600, 1403, 1552, 1788, 1263, 1891, 1796, 1934, 3362, 1868, 1238, 1307, 1248, 3350, 3304, 1055, 1352, 1779, 1931, 1391, 1392, 1393, 1983, 3284, 7110, 2026, 2069, 3253, 347, 892, 2112, 1392, 1108, 1110, 1248, 1488, 1500, 3239, 3204, 3174, 1541, 3194, 3177, 2112, 1624, 1119, 3145, 1176, 3142, 1861, 1873, 1878, 3104, 1704, 1607, 3071, 2999, 1394, 2951, 2939, 1979, 1503, 1180, 1209, 1896, 1901, 1669, 1259, 1273, 1936, 1971, 1707, 1278, 1296, 2904, 1691, 2927, 2900, 2022, 1731, 1336, 2867, 1403, 2856, 1994, 2027, 2063, 1753, 2851, 716, 2848, 950, 2068, 2106, 2117, 1786, 2836, 2831, 1793, 2833, 2795, 2150, 2191, 1807, 2234, 2138, 2275, 2156, 2333, 2228, 2237, 2273, 2344, 2358, 2367, 2378, 2449, 2523, 2189, 2433, 2413, 1498, 2121, 2020, 2269, 2451, 1769, 2278, 1797, 2271, 1426, 1684, 1625, 2322, 1465, 2311, 2342, 2464, 2148, 2457, 2261, 1528, 2388, 2423, 2759, 1179, 1347, 2149, 2296, 2245, 2683, 2676, 1875, 1831, 2535, 2548, 1910, 2480, 2177, 2645, 2637, 2383, 2555, 7110, 2446, 2468, 2600, 2595, 2508, 2546, 2570, 2561, 2456, 2552, 2533, 2540, 1981, 2561, 2614, 2651, 2674, 741, 457, 7110, 2729, 2807, 2597, 661, 2134, 2596, 2579, 1548, 1591, 2330, 2405, 2814, 2604, 1589, 2641, 97, 2882, 2584, 771, 2942, 3003, 3046, 2605, 1637, 1675, 2688, 2695, 2537, 1235, 2639, 2556, 2555, 2710, 2662, 2543, 2538, 2819, 1925, 2402, 2684, 1849, 2508, 1979, 2507, 2715, 2720, 2844, 2463, 1448, 2462, 1598, 2832, 2475, 2463, 2837, 2420, 2412, 2894, 2582, 2374, 2365, 3042, 2023, 2628, 2816, 2853, 2025, 2109, 2889, 2919, 2312, 1614, 2857, 2229, 2270, 2929, 3047, 2262, 1615, 2931, 2423, 2439, 3052, 3083, 3071, 2289, 2269, 3106, 2300, 2407, 3094, 2461, 2233, 2530, 2232, 3119, 3124, 3129, 2191, 1714, 2183, 1734, 3117, 2186, 1033, 2175, 1110, 3142, 3147, 3152, 2135, 1803, 2132, 2181, 2103, 2338, 3076, 2090, 2077, 3160, 3148, 2035, 2003, 3172, 3132, 3134, 0, 3213, 1864, 3230, 3173, 3254, 3181, 3312, 3378, 3437, 3511, 3581, 3656, 3723, 3785, 3859, 3933, 3219, 3988, 4050, 3236, 2813, 3269, 2888, 2606, 2901, 3277, 3160, 2608, 2666, 3251, 3162, 3338, 3190, 2881, 3264, 7110, 3191, 3285, 1994, 1985, 2987, 3302, 3350, 3326, 3303, 3358, 3340, 1965, 1964, 3341, 3369, 3367, 3038, 3373, 3009, 4107, 3394, 1039, 3416, 4166, 696, 4225, 3442, 3463, 3481, 3497, 3517, 4285, 4346, 4407, 3059, 3408, 3193, 1957, 1942, 3295, 3430, 3539, 3544, 3283, 2258, 3485, 311, 4467, 4510, 4553, 4596, 590, 2669, 3080, 3327, 3553, 3451, 1920, 1919, 3559, 1876, 2372, 805, 1897, 1896, 3572, 3260, 3261, 1801, 3311, 1794, 3565, 3614, 3619, 3505, 1764, 1751, 3626, 3572, 1712, 1704, 3635, 3537, 3597, 0, 842, 1683, 1668, 3640, 3543, 0, 962, 3354, 3355, 3645, 3661, 1143, 3420, 3436, 3672, 3681, 3673, 3477, 3547, 3701, 3710, 1613, 2478, 1302, 1608, 1607, 3716, 3707, 3553, 1605, 3554, 1537, 3732, 3737, 3748, 3732, 1529, 1796, 1502, 1830, 3762, 3770, 3777, 3774, 1498, 1459, 3798, 1394, 3135, 1339, 1384, 1374, 3811, 0, 4639, 3784, 1740, 3822, 2122, 3822, 3869, 3895, 3916, 3941, 3970, 3978, 3995, 4699, 4002, 3880, 4083, 4105, 3722, 0, 3634, 0, 1410, 7110, 3904, 3810, 1349, 1342, 3782, 3868, 4066, 4635, 3883, 3654, 3856, 1427, 3908, 1333, 1308, 3944, 4176, 3996, 3768, 3982, 1575, 4001, 4071, 4011, 3853, 4024, 1763, 4756, 4088, 7110, 1487, 3423, 4816, 896, 3208, 4876, 4262, 4444, 4919, 4504, 4547, 4591, 4677, 4979, 5040, 5101, 3896, 4040, 4023, 1274, 1266, 3798, 4080, 4509, 4041, 4065, 4639, 1821, 4709, 5144, 5187, 5230, 3632, 3726, 4208, 4213, 2076, 1260, 1207, 4231, 1176, 1123, 4793, 3760, 1108, 3762, 1099, 4236, 4322, 4327, 4093, 1043, 1038, 4332, 994, 3434, 2212, 1028, 1002, 4913, 992, 951, 4552, 3833, 0, 3946, 3953, 4474, 4479, 4041, 4059, 4645, 4716, 2284, 4091, 4106, 4721, 4733, 929, 840, 4919, 4115, 838, 4116, 837, 4738, 4743, 4756, 811, 2240, 777, 3157, 4763, 4798, 4803, 2371, 745, 718, 4823, 658, 646, 5138, 0, 4956, 5225, 5181, 5269, 5279, 5291, 5296, 5303, 5315, 5386, 5143, 5350, 5369, 5375, 5394, 5408, 5448, 5413, 642, 4130, 618, 583, 7110, 4253, 5467, 4134, 4243, 5276, 2587, 4250, 4854, 4277, 4128, 4675, 2903, 4432, 7110, 526, 2948, 4859, 1544, 4963, 5522, 1357, 3705, 5565, 5608, 5472, 5669, 5477, 5730, 4466, 5171, 4581, 508, 493, 4435, 5496, 4865, 4634, 5275, 3053, 5506, 5511, 5565, 478, 449, 4864, 5767, 5602, 5607, 5772, 3203, 451, 445, 4924, 443, 442, 5777, 5790, 5795, 5805, 5812, 5835, 5853, 5859, 4140, 4164, 4984, 5017, 5866, 5871, 5876, 5881, 5889, 5894, 5899, 411, 381, 5022, 5907, 5913, 7110, 4866, 5027, 5214, 4678, 5521, 3400, 4944, 7110, 370, 3468, 3483, 5950, 5993, 6036, 5987, 6030, 6096, 0, 6139, 7110, 5502, 5131, 4161, 4218, 4832, 5304, 6035, 5313, 4441, 5307, 3527, 5992, 344, 285, 5490, 6133, 6073, 6138, 6176, 5382, 7110, 296, 3629, 3675, 6213, 6256, 6299, 6199, 6342, 6385, 5403, 237, 230, 7110, 5415, 6212, 6255, 5439, 3976, 5380, 3852, 6293, 3928, 6428, 6471, 6514, 6557, 6600, 5441, 5531, 5500, 5100, 5886, 4057, 6643, 6686, 6729, 5612, 5628, 7110, 133, 4058, 6772, 4181, 6336, 7110, 6833, 6837, 6846, 6850, 6855, 6864, 6873, 6882, 6891, 6900, 112, 6904, 6913, 6922, 6931, 6940, 6949, 6958, 6967, 6976, 6984, 6993, 7002, 7011, 7020, 7029, 7038, 7047, 7056, 7065, 7074, 7083, 7092, 7100 } ; static const flex_int16_t yy_def[1137] = { 0, 1102, 1, 1102, 1102, 1102, 1102, 1102, 1103, 1104, 1105, 1106, 1102, 1102, 1102, 1102, 1102, 1107, 1107, 1108, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1103, 1102, 1109, 1104, 1102, 1110, 1105, 1111, 1102, 1107, 1108, 14, 1112, 1102, 1113, 1102, 14, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1107, 1115, 1107, 1107, 1107, 1102, 1102, 1116, 1102, 1102, 1102, 1102, 1102, 1102, 1103, 1103, 1103, 1117, 1104, 1104, 1105, 1105, 1105, 1102, 1112, 1118, 56, 1114, 1119, 1114, 1119, 1114, 94, 1114, 1114, 94, 1114, 94, 1114, 94, 1114, 94, 1114, 94, 1114, 1114, 94, 1114, 1114, 1114, 94, 1114, 94, 1114, 1114, 118, 118, 118, 118, 118, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1107, 68, 1107, 1107, 68, 1116, 1120, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1103, 1103, 80, 1117, 1121, 1104, 83, 1105, 1105, 86, 1122, 1102, 1114, 118, 176, 176, 176, 1114, 1114, 94, 176, 176, 176, 176, 176, 176, 176, 176, 1114, 1114, 1114, 94, 176, 176, 176, 1114, 176, 176, 176, 1114, 176, 176, 176, 1114, 1114, 94, 176, 176, 176, 1114, 176, 176, 176, 176, 176, 1114, 1114, 176, 176, 176, 176, 176, 1114, 1114, 1114, 94, 176, 176, 176, 1114, 118, 233, 233, 233, 233, 233, 233, 239, 239, 239, 239, 239, 239, 239, 239, 233, 248, 248, 239, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1107, 68, 1123, 68, 1124, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 80, 1125, 1102, 83, 86, 1122, 1126, 1114, 176, 301, 301, 301, 301, 301, 176, 176, 176, 1114, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 1114, 94, 176, 176, 176, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 176, 176, 176, 1114, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 176, 176, 176, 301, 301, 301, 301, 233, 373, 373, 373, 373, 373, 373, 379, 379, 379, 379, 379, 379, 379, 379, 373, 388, 388, 379, 1114, 1114, 1114, 1114, 373, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 373, 1114, 1114, 373, 1114, 1114, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 276, 1123, 1123, 1127, 1128, 1102, 1102, 276, 1129, 1130, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1131, 1132, 1133, 1102, 86, 301, 476, 476, 476, 476, 476, 1114, 1114, 301, 301, 301, 301, 476, 476, 476, 476, 1114, 1114, 476, 476, 476, 476, 476, 476, 476, 476, 1114, 1114, 1114, 1114, 176, 176, 176, 301, 301, 301, 301, 476, 476, 476, 476, 1114, 1114, 476, 476, 476, 476, 476, 476, 1114, 1114, 476, 476, 476, 476, 476, 1114, 1114, 301, 301, 301, 301, 301, 476, 476, 476, 476, 1114, 1114, 476, 476, 476, 476, 476, 476, 476, 476, 1114, 1114, 1114, 1114, 476, 476, 476, 476, 476, 476, 476, 476, 1114, 1114, 1114, 1114, 1114, 1114, 301, 301, 301, 301, 476, 476, 476, 476, 1114, 1114, 476, 373, 582, 582, 582, 582, 582, 582, 582, 582, 582, 582, 582, 582, 590, 590, 582, 582, 582, 590, 582, 582, 582, 582, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 276, 1102, 1127, 1102, 1134, 1128, 1135, 1123, 1123, 1102, 1123, 1123, 1123, 1102, 456, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1131, 474, 475, 476, 668, 668, 668, 668, 668, 476, 476, 476, 476, 1114, 1114, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 301, 301, 301, 301, 476, 476, 476, 476, 1114, 1114, 476, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 476, 476, 476, 476, 476, 1114, 1114, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 476, 476, 476, 476, 1114, 1114, 668, 668, 668, 668, 668, 582, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 276, 1102, 1102, 1127, 1127, 1127, 1128, 1128, 1128, 1123, 1123, 649, 1136, 1136, 1123, 1136, 649, 1102, 651, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1131, 474, 475, 668, 842, 842, 842, 842, 668, 668, 668, 668, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 476, 476, 476, 476, 1114, 1114, 668, 668, 668, 668, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 668, 668, 668, 668, 668, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 668, 668, 668, 668, 842, 842, 842, 842, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 826, 1127, 1127, 813, 1128, 1128, 816, 649, 1136, 1102, 1136, 824, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1114, 1114, 1114, 842, 842, 842, 1114, 1114, 1114, 1114, 668, 668, 668, 668, 842, 842, 842, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 842, 842, 842, 842, 1114, 1114, 1114, 1114, 1114, 1114, 1114, 842, 842, 842, 1114, 1114, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 813, 816, 649, 1136, 1136, 1136, 962, 824, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1114, 842, 842, 842, 1114, 1114, 1114, 1114, 1102, 1102, 1102, 1102, 1102, 813, 816, 649, 1136, 1033, 824, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1114, 1102, 813, 816, 649, 1033, 824, 1102, 1102, 1102, 1102, 1102, 1102, 813, 816, 1033, 1082, 1102, 1102, 1102, 1102, 1033, 1102, 1136, 0, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102 } ; static const flex_int16_t yy_nxt[7190] = { 0, 4, 5, 6, 5, 5, 5, 7, 8, 9, 4, 4, 10, 4, 4, 4, 11, 12, 13, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 15, 4, 4, 16, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 18, 17, 17, 19, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 18, 17, 17, 20, 21, 17, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 23, 23, 23, 23, 23, 28, 31, 28, 35, 63, 22, 22, 22, 22, 22, 63, 24, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 72, 1102, 1102, 75, 44, 76, 133, 28, 58, 95, 73, 74, 25, 63, 59, 75, 32, 76, 60, 1096, 37, 61, 72, 34, 65, 29, 26, 96, 62, 133, 65, 58, 95, 77, 253, 25, 40, 59, 32, 32, 60, 41, 42, 61, 43, 43, 43, 43, 43, 43, 43, 43, 43, 43, 29, 65, 77, 253, 44, 44, 45, 46, 47, 44, 48, 49, 50, 44, 51, 44, 52, 44, 44, 53, 54, 55, 44, 44, 44, 44, 56, 44, 44, 45, 46, 47, 44, 48, 49, 50, 51, 44, 52, 44, 44, 53, 54, 55, 44, 44, 44, 44, 63, 28, 44, 1102, 63, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 102, 103, 23, 23, 23, 23, 23, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 100, 24, 1086, 93, 66, 102, 101, 28, 28, 1086, 65, 161, 34, 94, 65, 75, 89, 76, 154, 174, 167, 280, 100, 295, 25, 93, 66, 68, 68, 68, 68, 68, 68, 68, 68, 68, 68, 106, 26, 28, 104, 68, 68, 68, 68, 68, 68, 25, 1057, 105, 34, 97, 107, 108, 115, 29, 138, 1078, 109, 98, 106, 99, 104, 110, 68, 68, 68, 68, 68, 68, 27, 27, 79, 97, 134, 108, 116, 115, 139, 138, 109, 98, 34, 158, 135, 159, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 134, 136, 63, 155, 80, 80, 80, 80, 80, 80, 156, 277, 27, 33, 63, 299, 328, 28, 174, 137, 1078, 329, 28, 330, 136, 155, 1024, 80, 80, 80, 80, 80, 80, 83, 83, 83, 83, 83, 83, 83, 83, 83, 83, 111, 65, 150, 104, 83, 83, 83, 83, 83, 83, 65, 157, 105, 65, 1055, 158, 112, 159, 113, 193, 114, 29, 34, 111, 150, 104, 72, 83, 83, 83, 83, 83, 83, 33, 33, 85, 73, 160, 112, 100, 113, 194, 193, 364, 1055, 101, 181, 365, 72, 86, 86, 86, 86, 86, 86, 86, 86, 86, 86, 160, 67, 100, 182, 86, 86, 86, 86, 86, 86, 181, 63, 364, 1052, 1052, 28, 365, 183, 203, 1051, 640, 184, 204, 185, 205, 1051, 1048, 86, 86, 86, 86, 86, 86, 68, 68, 68, 68, 68, 68, 68, 68, 68, 68, 175, 175, 206, 115, 68, 68, 68, 68, 68, 68, 65, 234, 1048, 207, 208, 234, 234, 234, 234, 29, 643, 1043, 175, 175, 206, 116, 115, 68, 68, 68, 68, 68, 68, 117, 117, 207, 1043, 950, 117, 117, 117, 117, 117, 117, 117, 117, 117, 117, 117, 117, 117, 118, 119, 119, 119, 120, 121, 122, 123, 119, 119, 117, 117, 117, 117, 119, 119, 119, 119, 119, 119, 124, 125, 126, 117, 127, 117, 128, 117, 117, 129, 130, 131, 117, 117, 117, 117, 117, 119, 119, 119, 119, 119, 119, 124, 125, 126, 127, 117, 128, 117, 117, 129, 130, 131, 117, 117, 117, 117, 117, 117, 117, 140, 227, 1018, 254, 141, 142, 143, 144, 262, 843, 255, 844, 263, 288, 264, 229, 102, 103, 417, 230, 228, 231, 145, 265, 227, 254, 146, 106, 247, 147, 248, 249, 234, 234, 234, 234, 288, 1018, 102, 266, 417, 175, 107, 267, 1016, 145, 265, 268, 146, 269, 106, 147, 149, 149, 149, 149, 149, 149, 149, 149, 149, 149, 1014, 175, 289, 260, 149, 149, 149, 149, 149, 149, 458, 287, 1014, 280, 186, 158, 271, 159, 187, 188, 189, 190, 261, 111, 272, 289, 260, 149, 149, 149, 149, 149, 149, 67, 67, 67, 151, 67, 155, 112, 191, 113, 640, 114, 63, 156, 111, 272, 192, 138, 152, 152, 152, 152, 152, 152, 152, 152, 152, 152, 155, 112, 191, 113, 152, 152, 152, 152, 152, 152, 192, 139, 138, 563, 199, 1013, 281, 564, 176, 200, 176, 201, 117, 640, 282, 643, 65, 152, 152, 152, 152, 152, 152, 163, 78, 78, 164, 163, 281, 28, 175, 247, 1013, 248, 249, 234, 234, 234, 234, 202, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 473, 305, 175, 295, 165, 165, 165, 165, 165, 165, 641, 202, 256, 1008, 92, 270, 257, 271, 258, 141, 142, 143, 144, 136, 305, 274, 29, 165, 165, 165, 165, 165, 165, 82, 82, 82, 168, 82, 259, 290, 306, 137, 851, 75, 852, 76, 136, 1008, 274, 1102, 169, 169, 169, 169, 169, 169, 169, 169, 169, 169, 259, 134, 320, 306, 169, 169, 169, 169, 169, 169, 283, 135, 302, 1006, 1006, 284, 303, 285, 304, 78, 871, 1004, 872, 134, 28, 320, 32, 169, 169, 169, 169, 169, 169, 170, 84, 84, 171, 170, 311, 286, 322, 315, 312, 28, 313, 316, 317, 318, 319, 175, 172, 172, 172, 172, 172, 172, 172, 172, 172, 172, 82, 286, 640, 322, 172, 172, 172, 172, 172, 172, 325, 29, 193, 332, 326, 1102, 327, 195, 333, 335, 334, 176, 196, 176, 197, 323, 34, 172, 172, 172, 172, 172, 172, 119, 194, 193, 175, 119, 119, 119, 119, 92, 335, 175, 324, 643, 108, 198, 323, 336, 1004, 109, 32, 117, 117, 117, 110, 117, 175, 117, 340, 84, 117, 117, 117, 175, 993, 563, 108, 198, 28, 564, 336, 109, 341, 355, 117, 117, 117, 117, 876, 117, 877, 340, 117, 117, 117, 176, 176, 176, 176, 176, 176, 176, 176, 176, 176, 341, 355, 357, 425, 176, 176, 176, 176, 176, 176, 993, 291, 291, 291, 292, 291, 34, 337, 346, 992, 426, 338, 347, 339, 348, 357, 425, 176, 176, 176, 176, 176, 176, 177, 176, 176, 176, 178, 176, 179, 176, 176, 176, 426, 640, 992, 92, 176, 176, 176, 176, 176, 176, 744, 72, 427, 209, 745, 989, 180, 210, 213, 211, 989, 73, 214, 215, 216, 217, 428, 176, 176, 176, 176, 176, 176, 72, 212, 427, 220, 434, 180, 175, 221, 222, 223, 224, 218, 207, 208, 350, 641, 219, 418, 351, 352, 353, 354, 435, 281, 212, 419, 307, 434, 175, 175, 308, 282, 309, 218, 207, 225, 358, 226, 219, 418, 359, 360, 361, 362, 369, 281, 310, 984, 370, 366, 371, 175, 175, 367, 744, 368, 984, 225, 745, 226, 117, 117, 117, 232, 117, 480, 374, 481, 982, 310, 374, 374, 374, 374, 175, 175, 499, 233, 234, 234, 234, 235, 236, 237, 238, 234, 234, 880, 480, 881, 481, 234, 234, 234, 234, 234, 234, 175, 424, 499, 254, 420, 257, 415, 258, 342, 421, 255, 422, 176, 343, 176, 344, 92, 234, 234, 234, 234, 234, 234, 234, 982, 254, 239, 240, 241, 234, 242, 243, 244, 423, 175, 429, 501, 523, 245, 430, 246, 431, 387, 345, 388, 389, 374, 374, 374, 374, 399, 399, 399, 400, 399, 423, 175, 981, 92, 501, 523, 245, 482, 246, 234, 345, 524, 239, 240, 241, 234, 242, 243, 244, 482, 482, 482, 483, 482, 250, 387, 251, 388, 389, 374, 374, 374, 374, 432, 524, 148, 465, 263, 91, 264, 440, 284, 100, 285, 268, 63, 269, 250, 101, 251, 148, 148, 148, 275, 148, 981, 459, 436, 971, 92, 91, 437, 460, 438, 100, 530, 971, 276, 276, 276, 276, 276, 276, 276, 276, 276, 276, 439, 459, 531, 537, 276, 276, 276, 276, 276, 276, 65, 530, 415, 415, 415, 416, 415, 889, 461, 890, 944, 538, 462, 439, 463, 531, 537, 276, 276, 276, 276, 276, 276, 278, 278, 278, 278, 278, 278, 278, 278, 278, 278, 374, 538, 944, 464, 278, 278, 278, 278, 278, 278, 442, 910, 938, 911, 141, 142, 143, 144, 640, 938, 467, 550, 92, 374, 158, 464, 159, 278, 278, 278, 278, 278, 278, 293, 293, 293, 293, 293, 293, 293, 293, 293, 293, 912, 550, 291, 605, 293, 293, 293, 293, 293, 293, 912, 445, 445, 445, 446, 445, 470, 477, 643, 513, 75, 478, 76, 479, 514, 605, 515, 293, 293, 293, 293, 293, 293, 296, 296, 296, 296, 296, 296, 296, 296, 296, 296, 72, 72, 552, 136, 296, 296, 296, 296, 296, 296, 73, 73, 92, 257, 502, 258, 397, 397, 397, 398, 397, 137, 72, 72, 932, 552, 136, 296, 296, 296, 296, 296, 296, 297, 297, 297, 297, 297, 297, 297, 297, 297, 297, 102, 103, 909, 932, 297, 297, 297, 297, 297, 297, 482, 482, 482, 483, 482, 640, 97, 403, 403, 403, 404, 403, 102, 92, 98, 106, 99, 297, 297, 297, 297, 297, 297, 175, 175, 175, 300, 175, 97, 484, 107, 909, 520, 485, 903, 486, 98, 521, 106, 522, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 91, 104, 641, 93, 301, 301, 301, 301, 301, 301, 105, 903, 640, 94, 447, 447, 447, 448, 447, 897, 488, 115, 91, 104, 489, 93, 490, 301, 301, 301, 301, 301, 301, 373, 374, 374, 374, 375, 376, 377, 378, 374, 374, 116, 115, 138, 659, 374, 374, 374, 374, 374, 374, 394, 394, 394, 395, 394, 263, 641, 264, 504, 443, 443, 443, 444, 443, 139, 138, 659, 374, 374, 374, 374, 374, 374, 374, 525, 532, 379, 380, 381, 374, 382, 383, 384, 509, 897, 396, 660, 510, 385, 511, 386, 891, 891, 91, 155, 401, 401, 401, 402, 401, 494, 156, 134, 96, 495, 496, 497, 498, 396, 660, 92, 385, 135, 386, 374, 91, 155, 379, 380, 381, 374, 382, 383, 384, 134, 92, 92, 92, 104, 390, 672, 391, 405, 405, 405, 406, 405, 105, 91, 407, 407, 407, 408, 407, 401, 527, 873, 102, 103, 528, 104, 529, 390, 672, 391, 392, 392, 392, 393, 392, 91, 873, 407, 407, 407, 408, 407, 539, 673, 102, 106, 540, 374, 541, 553, 91, 374, 374, 374, 374, 506, 870, 108, 534, 507, 107, 508, 109, 535, 870, 536, 673, 110, 106, 555, 102, 103, 91, 93, 405, 405, 405, 406, 405, 108, 108, 175, 545, 94, 109, 109, 546, 547, 548, 549, 110, 923, 102, 924, 925, 93, 409, 409, 409, 410, 409, 92, 108, 175, 557, 374, 864, 109, 558, 559, 560, 561, 106, 468, 468, 468, 469, 468, 268, 864, 269, 92, 445, 445, 445, 446, 445, 107, 374, 411, 450, 450, 450, 450, 450, 106, 452, 571, 565, 91, 453, 572, 454, 573, 575, 112, 97, 113, 576, 114, 577, 904, 411, 859, 98, 905, 99, 136, 583, 155, 859, 91, 583, 583, 583, 583, 156, 112, 97, 113, 412, 412, 412, 413, 412, 137, 98, 284, 100, 285, 136, 155, 614, 455, 101, 904, 257, 374, 258, 905, 92, 374, 374, 374, 374, 502, 502, 502, 503, 502, 100, 414, 23, 23, 23, 23, 23, 504, 504, 504, 505, 504, 502, 502, 502, 503, 502, 689, 24, 757, 757, 757, 757, 116, 414, 148, 148, 148, 275, 148, 525, 525, 525, 526, 525, 525, 525, 525, 526, 525, 689, 25, 449, 449, 449, 449, 449, 449, 449, 449, 449, 449, 853, 853, 418, 26, 449, 449, 449, 449, 449, 449, 419, 92, 25, 468, 468, 468, 469, 468, 532, 532, 532, 533, 532, 617, 418, 850, 850, 449, 449, 449, 449, 449, 449, 456, 456, 456, 456, 456, 456, 456, 456, 456, 456, 834, 181, 618, 617, 456, 456, 456, 456, 456, 456, 532, 532, 532, 533, 532, 834, 155, 182, 517, 517, 517, 518, 517, 156, 181, 803, 803, 456, 456, 456, 456, 456, 456, 553, 553, 553, 554, 553, 155, 471, 471, 471, 471, 471, 471, 471, 471, 471, 471, 519, 797, 138, 691, 471, 471, 471, 471, 471, 471, 797, 755, 543, 543, 543, 544, 543, 555, 555, 555, 556, 555, 194, 519, 139, 138, 691, 471, 471, 471, 471, 471, 471, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 755, 193, 95, 712, 474, 474, 474, 474, 474, 474, 553, 553, 553, 554, 553, 565, 565, 565, 566, 565, 96, 207, 208, 194, 193, 95, 712, 474, 474, 474, 474, 474, 474, 475, 475, 475, 475, 475, 475, 475, 475, 475, 475, 207, 979, 749, 980, 475, 475, 475, 475, 475, 475, 567, 567, 567, 568, 567, 749, 492, 492, 492, 493, 492, 569, 569, 569, 570, 569, 392, 475, 475, 475, 475, 475, 475, 476, 476, 476, 476, 476, 476, 476, 476, 476, 476, 923, 713, 924, 925, 476, 476, 476, 476, 476, 476, 181, 579, 579, 579, 580, 579, 596, 92, 597, 598, 583, 583, 583, 583, 713, 93, 182, 476, 476, 476, 476, 476, 476, 181, 596, 94, 597, 598, 583, 583, 583, 583, 111, 567, 652, 581, 92, 93, 653, 92, 415, 415, 415, 416, 415, 623, 743, 606, 112, 430, 113, 431, 114, 607, 228, 111, 652, 743, 581, 582, 583, 583, 583, 584, 585, 586, 587, 583, 583, 606, 112, 583, 113, 583, 583, 583, 583, 583, 583, 394, 394, 394, 395, 394, 990, 92, 991, 92, 397, 397, 397, 398, 397, 92, 583, 92, 583, 583, 583, 583, 583, 583, 583, 736, 736, 588, 589, 590, 583, 591, 592, 593, 1009, 612, 601, 717, 1010, 594, 421, 595, 422, 394, 91, 399, 399, 399, 399, 400, 399, 583, 397, 97, 96, 583, 583, 583, 583, 601, 717, 98, 594, 99, 595, 583, 91, 729, 588, 589, 590, 583, 591, 592, 593, 97, 281, 95, 718, 1000, 599, 1001, 600, 98, 282, 405, 608, 729, 92, 91, 609, 100, 610, 100, 97, 96, 403, 101, 281, 101, 95, 718, 98, 599, 99, 600, 392, 392, 392, 393, 392, 91, 569, 100, 611, 100, 97, 401, 401, 401, 402, 401, 106, 583, 98, 207, 208, 583, 583, 583, 583, 403, 403, 403, 404, 403, 611, 107, 92, 104, 405, 405, 405, 406, 405, 106, 678, 207, 105, 93, 459, 407, 407, 407, 408, 407, 460, 706, 108, 94, 91, 104, 412, 109, 92, 1011, 706, 1012, 110, 102, 103, 93, 459, 625, 91, 104, 492, 263, 106, 264, 108, 543, 91, 91, 105, 109, 407, 407, 407, 408, 407, 102, 115, 107, 108, 661, 91, 104, 92, 109, 284, 106, 285, 699, 110, 91, 405, 405, 405, 406, 405, 699, 181, 583, 116, 115, 108, 583, 583, 583, 583, 109, 409, 409, 409, 410, 409, 722, 182, 108, 604, 409, 207, 208, 109, 181, 628, 583, 407, 110, 629, 583, 630, 723, 106, 583, 583, 583, 583, 228, 722, 108, 724, 604, 207, 602, 109, 695, 631, 107, 583, 181, 437, 111, 438, 91, 723, 106, 735, 695, 619, 112, 134, 113, 620, 114, 621, 182, 602, 112, 108, 113, 135, 114, 181, 109, 111, 91, 92, 92, 110, 735, 622, 112, 134, 113, 412, 412, 412, 413, 412, 112, 108, 113, 92, 690, 690, 109, 615, 615, 615, 616, 615, 583, 445, 622, 434, 583, 583, 583, 583, 615, 615, 615, 616, 615, 443, 603, 626, 626, 626, 627, 626, 683, 435, 447, 633, 737, 683, 434, 268, 136, 269, 634, 634, 634, 635, 634, 136, 116, 603, 636, 677, 677, 254, 141, 142, 143, 144, 137, 737, 255, 260, 92, 136, 138, 137, 254, 134, 658, 473, 136, 703, 462, 255, 463, 254, 704, 135, 705, 265, 261, 421, 458, 422, 260, 654, 139, 138, 254, 134, 655, 632, 656, 664, 669, 266, 632, 158, 670, 159, 671, 517, 265, 637, 637, 637, 637, 637, 637, 637, 637, 637, 637, 779, 468, 657, 787, 637, 637, 637, 637, 637, 637, 450, 450, 450, 450, 450, 674, 452, 624, 193, 675, 453, 676, 454, 779, 657, 624, 787, 637, 637, 637, 637, 637, 637, 638, 638, 638, 638, 638, 680, 1102, 194, 193, 681, 1102, 682, 454, 155, 482, 482, 482, 483, 482, 613, 156, 482, 482, 482, 483, 482, 613, 684, 788, 845, 455, 685, 686, 687, 688, 155, 678, 678, 678, 679, 678, 502, 502, 502, 503, 502, 504, 504, 504, 505, 504, 788, 845, 455, 644, 645, 646, 646, 646, 645, 647, 644, 647, 647, 647, 644, 644, 648, 647, 647, 647, 647, 649, 649, 649, 649, 649, 649, 649, 649, 649, 649, 647, 647, 647, 647, 649, 649, 649, 649, 649, 649, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 650, 649, 649, 649, 649, 649, 649, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 67, 67, 67, 151, 67, 92, 578, 662, 662, 662, 663, 662, 492, 492, 492, 493, 492, 651, 651, 651, 651, 651, 651, 651, 651, 651, 651, 757, 757, 757, 757, 651, 651, 651, 651, 651, 651, 502, 502, 502, 503, 502, 692, 708, 578, 574, 693, 696, 694, 181, 574, 697, 281, 698, 651, 651, 651, 651, 651, 651, 282, 562, 324, 709, 562, 182, 708, 714, 710, 551, 711, 715, 181, 716, 281, 163, 78, 78, 164, 163, 551, 28, 525, 525, 525, 526, 525, 700, 700, 700, 701, 700, 665, 665, 665, 665, 665, 665, 665, 665, 665, 665, 757, 757, 757, 757, 665, 665, 665, 665, 665, 665, 525, 525, 525, 526, 525, 430, 542, 431, 254, 702, 532, 532, 532, 533, 532, 255, 29, 665, 665, 665, 665, 665, 665, 82, 82, 82, 168, 82, 324, 719, 254, 780, 702, 542, 720, 781, 721, 30, 92, 516, 666, 666, 666, 666, 666, 666, 666, 666, 666, 666, 437, 516, 438, 780, 666, 666, 666, 666, 666, 666, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 32, 666, 666, 666, 666, 666, 666, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 512, 617, 30, 667, 667, 667, 667, 667, 667, 667, 667, 667, 667, 141, 142, 143, 144, 667, 667, 667, 667, 667, 667, 618, 617, 517, 517, 517, 518, 517, 532, 532, 532, 533, 532, 724, 724, 724, 725, 724, 667, 667, 667, 667, 667, 667, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 707, 462, 265, 463, 668, 668, 668, 668, 668, 668, 724, 724, 724, 725, 724, 726, 512, 827, 266, 727, 746, 728, 194, 707, 747, 265, 748, 668, 668, 668, 668, 668, 668, 543, 543, 543, 544, 543, 730, 828, 827, 846, 731, 732, 733, 734, 553, 553, 553, 554, 553, 555, 555, 555, 556, 555, 553, 553, 553, 554, 553, 738, 579, 750, 846, 739, 740, 741, 742, 565, 565, 565, 566, 565, 567, 567, 567, 568, 567, 569, 569, 569, 570, 569, 92, 207, 208, 750, 750, 750, 751, 750, 752, 227, 500, 227, 753, 500, 754, 579, 579, 579, 580, 579, 786, 1009, 789, 207, 609, 1010, 610, 421, 228, 422, 228, 92, 227, 770, 227, 771, 772, 757, 757, 757, 757, 770, 491, 771, 772, 757, 757, 757, 757, 756, 792, 793, 642, 833, 257, 794, 258, 795, 655, 491, 656, 640, 415, 415, 415, 416, 415, 1049, 228, 1050, 92, 487, 756, 757, 757, 757, 757, 758, 759, 760, 761, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 762, 763, 764, 757, 765, 766, 767, 757, 757, 757, 757, 643, 768, 487, 769, 615, 299, 757, 757, 757, 757, 757, 757, 181, 762, 763, 764, 757, 765, 766, 767, 854, 855, 856, 857, 768, 773, 769, 774, 182, 757, 757, 757, 757, 782, 858, 181, 473, 606, 783, 838, 784, 796, 778, 607, 284, 620, 285, 621, 773, 254, 774, 392, 392, 392, 393, 392, 255, 858, 798, 606, 466, 228, 430, 785, 431, 778, 482, 482, 482, 483, 482, 254, 757, 757, 757, 757, 260, 790, 790, 790, 791, 790, 801, 652, 860, 785, 263, 653, 264, 799, 799, 799, 800, 799, 93, 261, 802, 804, 626, 260, 629, 437, 630, 438, 94, 652, 466, 860, 805, 805, 805, 806, 805, 634, 458, 441, 93, 394, 394, 394, 395, 394, 418, 807, 878, 879, 441, 268, 427, 269, 419, 260, 433, 638, 638, 638, 638, 638, 757, 757, 757, 757, 428, 434, 418, 454, 433, 878, 879, 265, 261, 427, 775, 372, 260, 809, 809, 809, 809, 809, 609, 435, 610, 639, 829, 266, 434, 810, 640, 830, 96, 831, 265, 372, 865, 775, 397, 397, 397, 398, 397, 638, 638, 638, 638, 638, 835, 1102, 92, 92, 462, 1102, 463, 454, 882, 92, 832, 757, 757, 757, 757, 363, 638, 638, 638, 638, 638, 847, 1102, 363, 883, 848, 1102, 849, 454, 641, 92, 882, 832, 97, 646, 646, 646, 646, 646, 662, 92, 98, 620, 99, 621, 92, 454, 883, 455, 356, 638, 638, 638, 638, 638, 97, 1102, 629, 356, 630, 1102, 887, 454, 98, 399, 399, 399, 400, 399, 455, 638, 638, 638, 638, 638, 861, 1102, 349, 349, 862, 1102, 863, 454, 281, 887, 757, 757, 757, 757, 92, 92, 282, 836, 836, 836, 837, 836, 836, 836, 836, 837, 836, 655, 455, 656, 281, 482, 482, 482, 483, 482, 100, 678, 678, 678, 679, 678, 101, 502, 502, 502, 503, 502, 455, 323, 492, 492, 492, 493, 492, 875, 888, 92, 100, 401, 401, 401, 402, 401, 459, 896, 898, 867, 324, 459, 460, 868, 323, 869, 324, 460, 700, 331, 875, 888, 757, 757, 757, 757, 331, 92, 459, 181, 896, 898, 92, 459, 504, 504, 504, 505, 504, 502, 502, 502, 503, 502, 321, 182, 865, 865, 865, 866, 865, 323, 181, 102, 103, 700, 700, 700, 701, 700, 517, 517, 517, 518, 517, 525, 525, 525, 526, 525, 324, 783, 321, 784, 323, 102, 403, 403, 403, 404, 403, 525, 525, 525, 526, 525, 977, 314, 931, 708, 314, 874, 532, 532, 532, 533, 532, 757, 757, 757, 757, 532, 532, 532, 533, 532, 92, 228, 324, 977, 884, 931, 708, 194, 874, 885, 794, 886, 795, 418, 104, 724, 724, 724, 725, 724, 814, 419, 92, 105, 724, 724, 724, 725, 724, 640, 543, 543, 543, 544, 543, 418, 104, 405, 405, 405, 406, 405, 892, 893, 894, 895, 553, 553, 553, 554, 553, 555, 555, 555, 556, 555, 92, 299, 757, 757, 757, 757, 553, 553, 553, 554, 553, 899, 900, 901, 902, 167, 643, 181, 978, 106, 565, 565, 565, 566, 565, 154, 207, 208, 567, 567, 567, 568, 567, 182, 107, 569, 569, 569, 570, 569, 181, 978, 106, 407, 407, 407, 408, 407, 207, 906, 273, 273, 983, 907, 985, 908, 750, 750, 750, 751, 750, 252, 915, 916, 917, 427, 918, 919, 920, 579, 579, 579, 580, 579, 921, 983, 922, 985, 92, 428, 392, 392, 392, 393, 392, 937, 108, 827, 427, 780, 783, 109, 784, 781, 91, 92, 110, 921, 92, 922, 915, 916, 917, 913, 918, 919, 920, 92, 108, 828, 827, 780, 926, 109, 927, 790, 91, 409, 409, 409, 410, 409, 228, 93, 92, 995, 913, 394, 394, 394, 395, 394, 830, 94, 831, 926, 92, 927, 415, 415, 415, 416, 415, 939, 324, 93, 434, 609, 995, 610, 776, 92, 92, 397, 397, 397, 398, 397, 942, 418, 91, 92, 928, 421, 435, 422, 112, 419, 113, 434, 114, 92, 92, 776, 399, 399, 399, 400, 399, 933, 96, 418, 91, 943, 934, 928, 935, 794, 112, 795, 113, 412, 412, 412, 413, 412, 97, 92, 965, 401, 401, 401, 402, 401, 98, 92, 99, 966, 934, 89, 935, 936, 757, 757, 757, 757, 167, 162, 97, 945, 965, 100, 777, 620, 162, 621, 98, 101, 403, 403, 403, 404, 403, 936, 154, 996, 405, 405, 405, 406, 405, 799, 997, 100, 116, 777, 405, 405, 405, 406, 405, 102, 103, 407, 407, 407, 408, 407, 996, 132, 412, 412, 412, 413, 412, 997, 92, 757, 757, 757, 757, 948, 104, 102, 106, 430, 949, 431, 965, 427, 629, 105, 630, 805, 106, 92, 952, 966, 90, 107, 437, 930, 438, 428, 104, 89, 108, 106, 970, 107, 965, 109, 427, 830, 87, 831, 110, 106, 407, 407, 407, 408, 407, 116, 930, 967, 975, 434, 108, 968, 462, 969, 463, 109, 940, 940, 940, 941, 940, 950, 950, 950, 951, 950, 998, 435, 968, 1039, 969, 1040, 434, 405, 405, 405, 406, 405, 809, 809, 809, 809, 809, 108, 999, 81, 71, 972, 109, 998, 810, 91, 655, 110, 656, 407, 407, 407, 408, 407, 986, 70, 459, 606, 987, 108, 988, 999, 460, 607, 109, 106, 1002, 91, 808, 808, 808, 808, 808, 808, 808, 808, 808, 808, 459, 606, 107, 1003, 808, 808, 808, 808, 808, 808, 106, 1002, 69, 1017, 108, 1005, 1007, 1022, 934, 109, 935, 609, 57, 610, 110, 617, 1003, 808, 808, 808, 808, 808, 808, 639, 639, 812, 108, 1053, 1005, 1007, 39, 109, 946, 946, 946, 947, 946, 618, 617, 813, 813, 813, 813, 813, 813, 813, 813, 813, 813, 1102, 1053, 1054, 1072, 813, 813, 813, 813, 813, 813, 1068, 1102, 1069, 1102, 617, 482, 482, 482, 483, 482, 482, 482, 482, 483, 482, 1054, 1072, 813, 813, 813, 813, 813, 813, 642, 642, 815, 618, 617, 678, 678, 678, 679, 678, 502, 502, 502, 503, 502, 1102, 816, 816, 816, 816, 816, 816, 816, 816, 816, 816, 1102, 1073, 1102, 1102, 816, 816, 816, 816, 816, 816, 638, 638, 638, 638, 638, 1023, 1102, 1102, 1019, 794, 1102, 795, 454, 783, 1073, 784, 1102, 816, 816, 816, 816, 816, 816, 817, 817, 817, 818, 817, 606, 1102, 1102, 1102, 1026, 1102, 607, 454, 620, 1102, 621, 1102, 819, 819, 819, 819, 819, 819, 819, 819, 819, 819, 606, 1102, 1102, 455, 819, 819, 819, 819, 819, 819, 504, 504, 504, 505, 504, 502, 502, 502, 503, 502, 865, 865, 865, 866, 865, 1102, 455, 819, 819, 819, 819, 819, 819, 820, 821, 638, 638, 638, 821, 822, 820, 822, 822, 822, 820, 820, 823, 822, 822, 822, 822, 824, 824, 824, 824, 824, 824, 824, 824, 824, 824, 822, 822, 822, 822, 824, 824, 824, 824, 824, 824, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 825, 824, 824, 824, 824, 824, 824, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 822, 826, 826, 826, 826, 826, 826, 826, 826, 826, 826, 1102, 1102, 1102, 1102, 826, 826, 826, 826, 826, 826, 638, 817, 638, 638, 638, 1027, 1102, 1102, 1044, 629, 1102, 630, 454, 830, 1102, 831, 1102, 826, 826, 826, 826, 826, 826, 163, 78, 78, 164, 163, 827, 28, 525, 525, 525, 526, 525, 525, 525, 525, 526, 525, 839, 839, 839, 839, 839, 839, 839, 839, 839, 839, 828, 827, 1102, 455, 839, 839, 839, 839, 839, 839, 646, 646, 646, 646, 646, 973, 973, 973, 974, 973, 1102, 1036, 454, 1102, 1102, 1037, 29, 839, 839, 839, 839, 839, 839, 840, 840, 840, 840, 840, 840, 840, 840, 840, 840, 1036, 1102, 1102, 1102, 840, 840, 840, 840, 840, 840, 638, 638, 638, 638, 638, 517, 517, 517, 518, 517, 962, 652, 454, 1102, 1102, 653, 1102, 840, 840, 840, 840, 840, 840, 841, 841, 841, 841, 841, 841, 841, 841, 841, 841, 652, 1102, 1102, 994, 841, 841, 841, 841, 841, 841, 1102, 638, 638, 638, 638, 638, 1102, 1102, 1042, 1102, 962, 1102, 968, 454, 969, 194, 994, 841, 841, 841, 841, 841, 841, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 1102, 1102, 1102, 1102, 842, 842, 842, 842, 842, 842, 1102, 1102, 940, 940, 940, 941, 940, 836, 1102, 1102, 1102, 455, 532, 532, 532, 533, 532, 842, 842, 842, 842, 842, 842, 914, 914, 914, 914, 914, 914, 914, 914, 914, 914, 1102, 1102, 1102, 1102, 914, 914, 914, 914, 914, 914, 946, 646, 646, 646, 646, 646, 606, 652, 1102, 1102, 459, 653, 607, 454, 1102, 1102, 460, 914, 914, 914, 914, 914, 914, 409, 409, 409, 410, 409, 606, 652, 617, 1102, 459, 163, 78, 78, 164, 163, 1102, 28, 532, 532, 532, 533, 532, 724, 724, 724, 725, 724, 1102, 780, 618, 617, 962, 781, 929, 724, 724, 724, 725, 724, 553, 553, 553, 554, 553, 555, 555, 555, 556, 555, 112, 780, 113, 1102, 114, 1102, 1102, 929, 553, 553, 553, 554, 553, 1102, 29, 565, 565, 565, 566, 565, 1102, 1102, 112, 1102, 113, 953, 953, 953, 953, 953, 953, 953, 953, 953, 953, 1102, 1102, 1102, 1102, 953, 953, 953, 953, 953, 953, 492, 492, 492, 493, 492, 567, 567, 567, 568, 567, 569, 569, 569, 570, 569, 1102, 1102, 953, 953, 953, 953, 953, 953, 954, 811, 811, 955, 954, 1102, 640, 750, 750, 750, 751, 750, 1102, 1102, 181, 1102, 1102, 956, 956, 956, 956, 956, 956, 956, 956, 956, 956, 1102, 1102, 1102, 182, 956, 956, 956, 956, 956, 956, 181, 1024, 1024, 1024, 1025, 1024, 148, 148, 148, 275, 148, 678, 678, 678, 679, 678, 641, 956, 956, 956, 956, 956, 956, 957, 814, 814, 958, 957, 1036, 1047, 1056, 1102, 1037, 640, 655, 934, 656, 935, 1102, 1102, 959, 959, 959, 959, 959, 959, 959, 959, 959, 959, 1036, 1102, 1102, 1102, 959, 959, 959, 959, 959, 959, 700, 700, 700, 701, 700, 1102, 543, 543, 543, 544, 543, 865, 865, 865, 866, 865, 643, 959, 959, 959, 959, 959, 959, 960, 960, 960, 960, 960, 960, 960, 960, 960, 960, 1102, 875, 1102, 1102, 960, 960, 960, 960, 960, 960, 117, 117, 117, 232, 117, 1060, 1102, 1102, 811, 794, 324, 795, 1102, 640, 875, 207, 208, 960, 960, 960, 960, 960, 960, 821, 821, 821, 963, 821, 724, 724, 724, 725, 724, 1102, 1102, 1102, 1102, 207, 1102, 1102, 964, 964, 964, 964, 964, 964, 964, 964, 964, 964, 1102, 1102, 1102, 92, 964, 964, 964, 964, 964, 964, 641, 724, 724, 724, 725, 724, 750, 750, 750, 751, 750, 1057, 1057, 1057, 1058, 1057, 1102, 964, 964, 964, 964, 964, 964, 644, 645, 646, 646, 646, 645, 647, 644, 647, 647, 647, 644, 644, 648, 647, 647, 647, 647, 649, 649, 649, 649, 649, 649, 649, 649, 649, 649, 647, 647, 647, 647, 649, 649, 649, 649, 649, 649, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 650, 649, 649, 649, 649, 649, 649, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 647, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 1102, 1102, 1102, 1102, 64, 64, 64, 64, 64, 64, 579, 579, 579, 580, 579, 412, 412, 412, 413, 412, 1071, 1036, 1102, 1102, 1039, 1037, 1040, 64, 64, 64, 64, 64, 64, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 1036, 1015, 1102, 115, 30, 30, 30, 30, 30, 30, 394, 394, 394, 395, 394, 1102, 1102, 1038, 1102, 1102, 228, 1039, 1102, 1040, 1015, 116, 115, 30, 30, 30, 30, 30, 30, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 1102, 1041, 95, 1102, 33, 33, 33, 33, 33, 33, 1102, 392, 392, 392, 393, 392, 1102, 1059, 1102, 1102, 96, 1102, 783, 1041, 784, 95, 1102, 33, 33, 33, 33, 33, 33, 976, 976, 976, 976, 976, 976, 976, 976, 976, 976, 1102, 1102, 1102, 1102, 976, 976, 976, 976, 976, 976, 1102, 93, 397, 397, 397, 398, 397, 1102, 1102, 973, 940, 94, 399, 399, 399, 400, 399, 976, 976, 976, 976, 976, 976, 93, 401, 401, 401, 402, 401, 403, 403, 403, 404, 403, 1102, 1102, 405, 405, 405, 406, 405, 1045, 1102, 1102, 1102, 97, 1102, 1102, 407, 407, 407, 408, 407, 98, 1074, 99, 606, 652, 968, 100, 969, 653, 607, 1077, 1102, 101, 1102, 97, 830, 1102, 831, 827, 1102, 104, 106, 98, 102, 103, 606, 652, 1102, 100, 105, 415, 415, 415, 416, 415, 1102, 107, 1102, 108, 1102, 828, 827, 104, 109, 106, 102, 1102, 1102, 110, 405, 405, 405, 406, 405, 1102, 407, 407, 407, 408, 407, 108, 1075, 1102, 1102, 1102, 109, 409, 409, 409, 410, 409, 1102, 1102, 1102, 117, 117, 117, 232, 117, 1079, 1102, 1102, 1102, 92, 934, 1102, 935, 106, 117, 117, 117, 232, 117, 117, 117, 117, 232, 117, 108, 111, 1085, 1102, 107, 109, 965, 1068, 1102, 1069, 110, 1102, 106, 181, 1087, 966, 1102, 112, 1039, 113, 1040, 114, 108, 1102, 111, 1102, 1102, 109, 965, 182, 117, 117, 117, 232, 117, 1102, 181, 1102, 1090, 112, 1095, 113, 968, 92, 969, 1068, 1102, 1069, 92, 1020, 1020, 1020, 1021, 1020, 646, 646, 646, 646, 646, 638, 821, 638, 638, 638, 227, 1102, 454, 1102, 1102, 1102, 1102, 454, 865, 865, 865, 866, 865, 1102, 1045, 1045, 1045, 1046, 1045, 228, 1102, 1102, 1102, 227, 175, 175, 175, 300, 175, 482, 482, 482, 483, 482, 780, 1098, 1102, 1067, 781, 1039, 1020, 1040, 1068, 962, 1069, 827, 1102, 1102, 962, 1096, 1096, 1096, 1097, 1096, 1102, 1102, 780, 1028, 1028, 1028, 1028, 1028, 1028, 1028, 1028, 1028, 1028, 828, 827, 1070, 1102, 1028, 1028, 1028, 1028, 1028, 1028, 92, 1102, 1102, 1102, 1102, 92, 482, 482, 482, 483, 482, 780, 1102, 1102, 1070, 781, 1102, 1028, 1028, 1028, 1028, 1028, 1028, 1029, 1029, 1029, 1029, 1029, 1029, 1029, 1029, 1029, 1029, 780, 1102, 1102, 1102, 1029, 1029, 1029, 1029, 1029, 1029, 502, 502, 502, 503, 502, 504, 504, 504, 505, 504, 821, 821, 821, 963, 821, 1102, 92, 1029, 1029, 1029, 1029, 1029, 1029, 1030, 1030, 1030, 1030, 1030, 1030, 1030, 1030, 1030, 1030, 1102, 1102, 1102, 1102, 1030, 1030, 1030, 1030, 1030, 1030, 1100, 1102, 1102, 1102, 1102, 1068, 1102, 1069, 1102, 1102, 92, 1102, 1102, 1102, 1102, 92, 1102, 1030, 1030, 1030, 1030, 1030, 1030, 820, 1031, 646, 646, 646, 1031, 1032, 820, 1032, 1032, 1032, 820, 820, 823, 1032, 1032, 1032, 1032, 1033, 1033, 1033, 1033, 1033, 1033, 1033, 1033, 1033, 1033, 1032, 1032, 1032, 1032, 1033, 1033, 1033, 1033, 1033, 1033, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1034, 1033, 1033, 1033, 1033, 1033, 1033, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1032, 1035, 1035, 1035, 1035, 1035, 1035, 1035, 1035, 1035, 1035, 1102, 1102, 1102, 1102, 1035, 1035, 1035, 1035, 1035, 1035, 492, 492, 492, 493, 492, 502, 502, 502, 503, 502, 700, 700, 700, 701, 700, 1102, 1102, 1035, 1035, 1035, 1035, 1035, 1035, 517, 517, 517, 518, 517, 175, 175, 175, 300, 175, 1102, 1102, 1102, 1102, 181, 175, 175, 175, 300, 175, 1102, 995, 525, 525, 525, 526, 525, 1102, 1102, 1102, 182, 193, 1102, 1102, 1102, 92, 1102, 181, 1102, 323, 324, 1102, 1102, 1102, 995, 525, 525, 525, 526, 525, 1102, 1102, 1102, 194, 193, 1102, 1102, 1102, 324, 1102, 1102, 1102, 323, 532, 532, 532, 533, 532, 92, 532, 532, 532, 533, 532, 1102, 92, 543, 543, 543, 544, 543, 553, 553, 553, 554, 553, 555, 555, 555, 556, 555, 553, 553, 553, 554, 553, 1102, 1088, 92, 565, 565, 565, 566, 565, 567, 567, 567, 568, 567, 569, 569, 569, 570, 569, 1102, 1102, 92, 579, 579, 579, 580, 579, 92, 175, 175, 175, 300, 175, 207, 208, 1102, 1102, 1102, 1102, 92, 1102, 1102, 1102, 1102, 92, 1102, 1102, 1102, 1102, 92, 1036, 1102, 1102, 1102, 1037, 207, 227, 92, 1102, 1102, 1102, 1102, 92, 1102, 1102, 1102, 1102, 92, 1102, 1102, 1102, 1102, 1036, 1102, 1102, 228, 1102, 1102, 1102, 227, 1102, 92, 1061, 1061, 1061, 1061, 1061, 1061, 1061, 1061, 1061, 1061, 1102, 1102, 1102, 1102, 1061, 1061, 1061, 1061, 1061, 1061, 646, 646, 646, 646, 646, 678, 678, 678, 679, 678, 1102, 1102, 454, 1102, 1102, 1102, 1102, 1061, 1061, 1061, 1061, 1061, 1061, 1062, 1062, 1062, 1062, 1062, 1062, 1062, 1062, 1062, 1062, 1102, 1102, 1102, 1102, 1062, 1062, 1062, 1062, 1062, 1062, 646, 646, 646, 646, 646, 1075, 1075, 1075, 1076, 1075, 962, 1102, 454, 1102, 1102, 92, 1102, 1062, 1062, 1062, 1062, 1062, 1062, 1063, 1063, 1063, 1063, 1063, 1063, 1063, 1063, 1063, 1063, 1102, 1102, 1102, 1102, 1063, 1063, 1063, 1063, 1063, 1063, 724, 724, 724, 725, 724, 1102, 965, 1102, 1102, 1102, 962, 1102, 1102, 1102, 1102, 966, 1102, 1063, 1063, 1063, 1063, 1063, 1063, 1031, 1031, 1031, 1064, 1031, 965, 1102, 1102, 1102, 1102, 1102, 1102, 454, 1102, 1102, 1102, 1102, 1065, 1065, 1065, 1065, 1065, 1065, 1065, 1065, 1065, 1065, 1102, 1102, 1102, 92, 1065, 1065, 1065, 1065, 1065, 1065, 700, 700, 700, 701, 700, 724, 724, 724, 725, 724, 1102, 1102, 1102, 1102, 1102, 1102, 962, 1065, 1065, 1065, 1065, 1065, 1065, 1066, 1066, 1066, 1066, 1066, 1066, 1066, 1066, 1066, 1066, 1102, 323, 1102, 1102, 1066, 1066, 1066, 1066, 1066, 1066, 750, 750, 750, 751, 750, 1102, 1102, 1102, 1102, 1102, 324, 1102, 1102, 1102, 323, 92, 1102, 1066, 1066, 1066, 1066, 1066, 1066, 646, 1031, 646, 646, 646, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 454, 1088, 1088, 1088, 1089, 1088, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 92, 1080, 1080, 1080, 1080, 1080, 1080, 1080, 1080, 1080, 1080, 1102, 1102, 1102, 1102, 1080, 1080, 1080, 1080, 1080, 1080, 1102, 1102, 962, 1102, 1102, 1088, 1088, 1088, 1089, 1088, 1102, 1036, 1102, 1102, 1102, 1037, 1102, 1080, 1080, 1080, 1080, 1080, 1080, 1081, 1081, 1081, 1081, 1081, 1081, 1081, 1081, 1081, 1081, 1036, 1102, 1102, 1102, 1081, 1081, 1081, 1081, 1081, 1081, 865, 865, 865, 866, 865, 1102, 1102, 1102, 1102, 1102, 1102, 1036, 1102, 1102, 1102, 1037, 1102, 1081, 1081, 1081, 1081, 1081, 1081, 1082, 1082, 1082, 1082, 1082, 1082, 1082, 1082, 1082, 1082, 1036, 1102, 1102, 1102, 1082, 1082, 1082, 1082, 1082, 1082, 1031, 1031, 1031, 1064, 1031, 1102, 1102, 1102, 1102, 1102, 92, 1102, 454, 1102, 1102, 1102, 1102, 1082, 1082, 1082, 1082, 1082, 1082, 1083, 1083, 1083, 1083, 1083, 1083, 1083, 1083, 1083, 1083, 1102, 1102, 1102, 1102, 1083, 1083, 1083, 1083, 1083, 1083, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 962, 1102, 1102, 1102, 1102, 1102, 1102, 1083, 1083, 1083, 1083, 1083, 1083, 1084, 1084, 1084, 1084, 1084, 1084, 1084, 1084, 1084, 1084, 1102, 1102, 1102, 1102, 1084, 1084, 1084, 1084, 1084, 1084, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1084, 1084, 1084, 1084, 1084, 1084, 1091, 1091, 1091, 1091, 1091, 1091, 1091, 1091, 1091, 1091, 1102, 1102, 1102, 1102, 1091, 1091, 1091, 1091, 1091, 1091, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1091, 1091, 1091, 1091, 1091, 1091, 1092, 1092, 1092, 1092, 1092, 1092, 1092, 1092, 1092, 1092, 1102, 1102, 1102, 1102, 1092, 1092, 1092, 1092, 1092, 1092, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1092, 1092, 1092, 1092, 1092, 1092, 451, 451, 451, 451, 451, 451, 451, 451, 451, 451, 1102, 1102, 1102, 1102, 451, 451, 451, 451, 451, 451, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 451, 451, 451, 451, 451, 451, 1093, 1093, 1093, 1093, 1093, 1093, 1093, 1093, 1093, 1093, 1102, 1102, 1102, 1102, 1093, 1093, 1093, 1093, 1093, 1093, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1093, 1093, 1093, 1093, 1093, 1093, 1094, 1094, 1094, 1094, 1094, 1094, 1094, 1094, 1094, 1094, 1102, 1102, 1102, 1102, 1094, 1094, 1094, 1094, 1094, 1094, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1094, 1094, 1094, 1094, 1094, 1094, 639, 639, 639, 639, 639, 639, 639, 639, 639, 639, 1102, 1102, 1102, 1102, 639, 639, 639, 639, 639, 639, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 639, 639, 639, 639, 639, 639, 642, 642, 642, 642, 642, 642, 642, 642, 642, 642, 1102, 1102, 1102, 1102, 642, 642, 642, 642, 642, 642, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 642, 642, 642, 642, 642, 642, 1099, 1099, 1099, 1099, 1099, 1099, 1099, 1099, 1099, 1099, 1102, 1102, 1102, 1102, 1099, 1099, 1099, 1099, 1099, 1099, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1099, 1099, 1099, 1099, 1099, 1099, 1101, 1101, 1101, 1101, 1101, 1101, 1101, 1101, 1101, 1101, 1102, 1102, 1102, 1102, 1101, 1101, 1101, 1101, 1101, 1101, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1101, 1101, 1101, 1101, 1101, 1101, 27, 27, 1102, 27, 27, 27, 27, 27, 27, 30, 30, 30, 30, 33, 33, 1102, 33, 33, 33, 33, 33, 33, 36, 1102, 1102, 36, 64, 64, 1102, 64, 64, 67, 67, 1102, 67, 67, 67, 67, 67, 67, 78, 78, 78, 78, 78, 78, 78, 78, 78, 82, 82, 1102, 82, 82, 82, 82, 82, 82, 84, 84, 84, 84, 84, 84, 84, 84, 84, 88, 88, 88, 88, 88, 88, 88, 88, 88, 91, 1102, 91, 91, 148, 148, 1102, 148, 148, 148, 148, 148, 148, 153, 153, 153, 153, 153, 153, 153, 153, 153, 166, 166, 166, 166, 166, 166, 166, 166, 166, 173, 173, 173, 173, 173, 173, 173, 173, 173, 175, 175, 1102, 175, 175, 175, 175, 175, 175, 279, 279, 279, 279, 279, 279, 279, 279, 279, 294, 294, 294, 294, 294, 294, 294, 294, 294, 298, 298, 298, 298, 298, 298, 298, 298, 298, 451, 451, 451, 1102, 451, 451, 451, 451, 457, 457, 457, 457, 457, 457, 457, 457, 457, 472, 472, 472, 472, 472, 472, 472, 472, 472, 173, 173, 173, 173, 173, 173, 173, 173, 173, 639, 639, 1102, 639, 639, 639, 639, 639, 639, 642, 642, 1102, 642, 642, 642, 642, 642, 642, 457, 457, 457, 457, 457, 457, 457, 457, 457, 279, 279, 279, 279, 279, 279, 279, 279, 279, 27, 27, 27, 27, 27, 27, 27, 27, 27, 472, 472, 472, 472, 472, 472, 472, 472, 472, 294, 294, 294, 294, 294, 294, 294, 294, 294, 811, 811, 811, 811, 811, 811, 811, 811, 811, 814, 814, 814, 814, 814, 814, 814, 814, 814, 961, 961, 1102, 1102, 961, 961, 961, 961, 3, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102 } ; static const flex_int16_t yy_chk[7190] = { 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 7, 7, 7, 7, 7, 8, 9, 10, 11, 17, 22, 22, 22, 22, 22, 36, 7, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 25, 30, 82, 470, 1113, 470, 58, 27, 16, 46, 25, 26, 7, 64, 16, 26, 9, 26, 16, 1097, 11, 16, 25, 10, 17, 8, 7, 46, 16, 58, 36, 16, 46, 26, 133, 7, 14, 16, 30, 82, 16, 14, 14, 16, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 27, 64, 26, 133, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 18, 33, 14, 38, 67, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 49, 49, 23, 23, 23, 23, 23, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 48, 23, 1069, 45, 18, 49, 48, 84, 78, 1068, 18, 74, 33, 45, 67, 74, 89, 74, 154, 89, 167, 154, 48, 167, 23, 45, 18, 19, 19, 19, 19, 19, 19, 19, 19, 19, 19, 51, 23, 170, 50, 19, 19, 19, 19, 19, 19, 23, 1058, 50, 84, 47, 51, 52, 54, 78, 61, 1050, 52, 47, 51, 47, 50, 52, 19, 19, 19, 19, 19, 19, 29, 29, 29, 47, 59, 52, 54, 54, 61, 61, 52, 47, 170, 664, 59, 664, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 59, 60, 148, 72, 29, 29, 29, 29, 29, 29, 72, 150, 79, 85, 66, 299, 195, 79, 299, 60, 1049, 195, 85, 195, 60, 72, 1025, 29, 29, 29, 29, 29, 29, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 53, 148, 66, 126, 32, 32, 32, 32, 32, 32, 150, 73, 126, 66, 1012, 73, 53, 73, 53, 100, 53, 79, 85, 53, 66, 126, 77, 32, 32, 32, 32, 32, 32, 34, 34, 34, 77, 73, 53, 124, 53, 100, 100, 222, 1011, 124, 95, 222, 77, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 73, 151, 124, 95, 34, 34, 34, 34, 34, 34, 95, 151, 224, 991, 990, 163, 224, 96, 105, 988, 453, 96, 105, 96, 105, 987, 980, 34, 34, 34, 34, 34, 34, 37, 37, 37, 37, 37, 37, 37, 37, 37, 37, 96, 105, 105, 130, 37, 37, 37, 37, 37, 37, 151, 119, 979, 106, 106, 119, 119, 119, 119, 163, 453, 969, 96, 105, 105, 130, 130, 37, 37, 37, 37, 37, 37, 56, 56, 106, 968, 951, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 62, 115, 935, 134, 62, 62, 62, 62, 137, 669, 134, 669, 137, 158, 137, 116, 125, 125, 253, 116, 115, 116, 62, 138, 115, 134, 62, 127, 121, 62, 121, 121, 121, 121, 121, 121, 158, 934, 125, 138, 253, 116, 127, 139, 932, 62, 138, 139, 62, 139, 127, 62, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 911, 116, 159, 136, 65, 65, 65, 65, 65, 65, 458, 157, 910, 458, 99, 157, 141, 157, 99, 99, 99, 99, 136, 129, 141, 159, 136, 65, 65, 65, 65, 65, 65, 68, 68, 68, 68, 68, 160, 129, 99, 129, 642, 129, 68, 160, 129, 141, 99, 147, 68, 68, 68, 68, 68, 68, 68, 68, 68, 68, 160, 129, 99, 129, 68, 68, 68, 68, 68, 68, 99, 147, 147, 360, 103, 908, 155, 360, 103, 103, 103, 103, 232, 452, 155, 642, 68, 68, 68, 68, 68, 68, 68, 80, 80, 80, 80, 80, 155, 80, 103, 123, 907, 123, 123, 123, 123, 123, 123, 103, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 473, 178, 103, 473, 80, 80, 80, 80, 80, 80, 452, 103, 135, 901, 232, 140, 135, 143, 135, 140, 140, 140, 140, 146, 178, 143, 80, 80, 80, 80, 80, 80, 80, 83, 83, 83, 83, 83, 135, 161, 179, 146, 680, 161, 680, 161, 146, 899, 143, 83, 83, 83, 83, 83, 83, 83, 83, 83, 83, 83, 135, 145, 187, 179, 83, 83, 83, 83, 83, 83, 156, 145, 177, 895, 893, 156, 177, 156, 177, 164, 703, 890, 703, 145, 164, 187, 83, 83, 83, 83, 83, 83, 83, 86, 86, 86, 86, 86, 183, 156, 189, 186, 183, 86, 183, 186, 186, 186, 186, 300, 86, 86, 86, 86, 86, 86, 86, 86, 86, 86, 168, 156, 814, 189, 86, 86, 86, 86, 86, 86, 194, 164, 198, 199, 194, 168, 194, 101, 199, 200, 199, 101, 101, 101, 101, 193, 86, 86, 86, 86, 86, 86, 86, 90, 198, 198, 194, 90, 90, 90, 90, 300, 200, 101, 193, 814, 128, 101, 193, 201, 889, 128, 168, 90, 90, 90, 128, 90, 194, 90, 204, 171, 90, 90, 90, 101, 872, 362, 128, 101, 171, 362, 201, 128, 205, 214, 90, 90, 90, 90, 709, 90, 709, 204, 90, 90, 90, 92, 92, 92, 92, 92, 92, 92, 92, 92, 92, 205, 214, 216, 257, 92, 92, 92, 92, 92, 92, 871, 162, 162, 162, 162, 162, 171, 203, 209, 869, 258, 203, 209, 203, 209, 216, 257, 92, 92, 92, 92, 92, 92, 94, 94, 94, 94, 94, 94, 94, 94, 94, 94, 258, 639, 868, 865, 94, 94, 94, 94, 94, 94, 559, 162, 260, 107, 559, 863, 94, 107, 110, 107, 862, 162, 110, 110, 110, 110, 260, 94, 94, 94, 94, 94, 94, 162, 107, 260, 114, 265, 94, 107, 114, 114, 114, 114, 110, 212, 212, 213, 639, 110, 254, 213, 213, 213, 213, 265, 286, 107, 254, 182, 265, 107, 114, 182, 286, 182, 110, 212, 114, 220, 114, 110, 254, 220, 220, 220, 220, 229, 286, 182, 857, 229, 228, 229, 114, 182, 228, 561, 228, 855, 114, 561, 114, 118, 118, 118, 118, 118, 303, 234, 304, 852, 182, 234, 234, 234, 234, 182, 228, 316, 118, 118, 118, 118, 118, 118, 118, 118, 118, 118, 714, 303, 714, 304, 118, 118, 118, 118, 118, 118, 228, 256, 316, 259, 255, 256, 416, 256, 208, 255, 259, 255, 208, 208, 208, 208, 118, 118, 118, 118, 118, 118, 118, 120, 851, 259, 120, 120, 120, 120, 120, 120, 120, 255, 208, 261, 318, 333, 120, 261, 120, 261, 236, 208, 236, 236, 236, 236, 236, 236, 242, 242, 242, 242, 242, 255, 208, 849, 416, 318, 333, 120, 483, 120, 122, 208, 334, 122, 122, 122, 122, 122, 122, 122, 305, 305, 305, 305, 305, 122, 238, 122, 238, 238, 238, 238, 238, 238, 262, 334, 275, 283, 262, 242, 262, 267, 283, 242, 283, 267, 275, 267, 122, 242, 122, 149, 149, 149, 149, 149, 848, 281, 266, 831, 483, 242, 266, 281, 266, 242, 338, 830, 149, 149, 149, 149, 149, 149, 149, 149, 149, 149, 266, 281, 339, 343, 149, 149, 149, 149, 149, 149, 275, 338, 249, 249, 249, 249, 249, 726, 282, 726, 795, 344, 282, 266, 282, 339, 343, 149, 149, 149, 149, 149, 149, 152, 152, 152, 152, 152, 152, 152, 152, 152, 152, 249, 344, 794, 282, 152, 152, 152, 152, 152, 152, 270, 752, 784, 752, 270, 270, 270, 270, 957, 783, 287, 351, 249, 249, 287, 282, 287, 152, 152, 152, 152, 152, 152, 165, 165, 165, 165, 165, 165, 165, 165, 165, 165, 754, 351, 292, 417, 165, 165, 165, 165, 165, 165, 753, 272, 272, 272, 272, 272, 290, 302, 957, 328, 290, 302, 290, 302, 328, 417, 328, 165, 165, 165, 165, 165, 165, 169, 169, 169, 169, 169, 169, 169, 169, 169, 169, 291, 292, 353, 272, 169, 169, 169, 169, 169, 169, 291, 292, 750, 792, 503, 792, 241, 241, 241, 241, 241, 272, 291, 292, 779, 353, 272, 169, 169, 169, 169, 169, 169, 172, 172, 172, 172, 172, 172, 172, 172, 172, 172, 401, 401, 748, 779, 172, 172, 172, 172, 172, 172, 306, 306, 306, 306, 306, 811, 241, 244, 244, 244, 244, 244, 401, 503, 241, 405, 241, 172, 172, 172, 172, 172, 172, 176, 176, 176, 176, 176, 241, 307, 405, 747, 332, 307, 741, 307, 241, 332, 405, 332, 176, 176, 176, 176, 176, 176, 176, 176, 176, 176, 244, 244, 811, 392, 176, 176, 176, 176, 176, 176, 244, 739, 954, 392, 273, 273, 273, 273, 273, 734, 311, 412, 244, 244, 311, 392, 311, 176, 176, 176, 176, 176, 176, 233, 233, 233, 233, 233, 233, 233, 233, 233, 233, 412, 412, 273, 462, 233, 233, 233, 233, 233, 233, 240, 240, 240, 240, 240, 801, 954, 801, 505, 271, 271, 271, 271, 271, 273, 273, 462, 233, 233, 233, 233, 233, 233, 235, 526, 533, 235, 235, 235, 235, 235, 235, 235, 325, 732, 240, 463, 325, 235, 325, 235, 728, 727, 240, 468, 243, 243, 243, 243, 243, 315, 468, 271, 240, 315, 315, 315, 315, 240, 463, 505, 235, 271, 235, 237, 240, 468, 237, 237, 237, 237, 237, 237, 237, 271, 724, 526, 533, 403, 237, 478, 237, 245, 245, 245, 245, 245, 403, 243, 246, 246, 246, 246, 246, 402, 337, 705, 243, 243, 337, 403, 337, 237, 478, 237, 239, 239, 239, 239, 239, 243, 704, 251, 251, 251, 251, 251, 346, 479, 243, 245, 346, 239, 346, 554, 245, 239, 239, 239, 239, 324, 698, 246, 342, 324, 245, 324, 246, 342, 697, 342, 479, 246, 245, 556, 402, 402, 245, 239, 250, 250, 250, 250, 250, 246, 251, 324, 350, 239, 246, 251, 350, 350, 350, 350, 251, 759, 402, 759, 759, 239, 247, 247, 247, 247, 247, 554, 251, 324, 358, 250, 694, 251, 358, 358, 358, 358, 250, 288, 288, 288, 288, 288, 807, 693, 807, 556, 274, 274, 274, 274, 274, 250, 250, 247, 277, 277, 277, 277, 277, 250, 277, 366, 566, 247, 277, 366, 277, 366, 369, 247, 397, 247, 369, 247, 369, 740, 247, 688, 397, 740, 397, 274, 374, 288, 686, 247, 374, 374, 374, 374, 288, 247, 397, 247, 248, 248, 248, 248, 248, 274, 397, 838, 399, 838, 274, 288, 424, 277, 399, 742, 424, 248, 424, 742, 566, 248, 248, 248, 248, 320, 320, 320, 320, 320, 399, 248, 280, 280, 280, 280, 280, 321, 321, 321, 321, 321, 322, 322, 322, 322, 322, 495, 280, 583, 583, 583, 583, 248, 248, 276, 276, 276, 276, 276, 335, 335, 335, 335, 335, 336, 336, 336, 336, 336, 495, 280, 276, 276, 276, 276, 276, 276, 276, 276, 276, 276, 682, 681, 423, 280, 276, 276, 276, 276, 276, 276, 423, 678, 280, 289, 289, 289, 289, 289, 340, 340, 340, 340, 340, 427, 423, 676, 675, 276, 276, 276, 276, 276, 276, 278, 278, 278, 278, 278, 278, 278, 278, 278, 278, 656, 492, 427, 427, 278, 278, 278, 278, 278, 278, 341, 341, 341, 341, 341, 655, 289, 492, 331, 331, 331, 331, 331, 289, 492, 630, 629, 278, 278, 278, 278, 278, 278, 355, 355, 355, 355, 355, 289, 293, 293, 293, 293, 293, 293, 293, 293, 293, 293, 331, 621, 447, 497, 293, 293, 293, 293, 293, 293, 620, 577, 349, 349, 349, 349, 349, 356, 356, 356, 356, 356, 331, 331, 447, 447, 497, 293, 293, 293, 293, 293, 293, 296, 296, 296, 296, 296, 296, 296, 296, 296, 296, 576, 517, 394, 521, 296, 296, 296, 296, 296, 296, 357, 357, 357, 357, 357, 363, 363, 363, 363, 363, 394, 349, 349, 517, 517, 394, 521, 296, 296, 296, 296, 296, 296, 297, 297, 297, 297, 297, 297, 297, 297, 297, 297, 349, 847, 573, 847, 297, 297, 297, 297, 297, 297, 364, 364, 364, 364, 364, 572, 314, 314, 314, 314, 314, 365, 365, 365, 365, 365, 393, 297, 297, 297, 297, 297, 297, 301, 301, 301, 301, 301, 301, 301, 301, 301, 301, 761, 522, 761, 761, 301, 301, 301, 301, 301, 301, 314, 372, 372, 372, 372, 372, 376, 569, 376, 376, 376, 376, 376, 376, 522, 393, 314, 301, 301, 301, 301, 301, 301, 314, 378, 393, 378, 378, 378, 378, 378, 378, 409, 568, 459, 372, 567, 393, 459, 565, 389, 389, 389, 389, 389, 429, 560, 418, 409, 429, 409, 429, 409, 418, 372, 409, 459, 558, 372, 373, 373, 373, 373, 373, 373, 373, 373, 373, 373, 418, 409, 389, 409, 373, 373, 373, 373, 373, 373, 380, 380, 380, 380, 380, 867, 568, 867, 555, 381, 381, 381, 381, 381, 389, 389, 553, 373, 373, 373, 373, 373, 373, 375, 549, 547, 375, 375, 375, 375, 375, 375, 375, 900, 420, 380, 528, 900, 375, 420, 375, 420, 395, 380, 400, 382, 382, 382, 382, 382, 411, 398, 381, 380, 411, 411, 411, 411, 380, 528, 381, 375, 381, 375, 377, 380, 541, 377, 377, 377, 377, 377, 377, 377, 381, 662, 395, 529, 884, 377, 884, 377, 381, 662, 406, 419, 540, 532, 382, 419, 400, 419, 382, 398, 395, 404, 400, 662, 382, 395, 529, 398, 377, 398, 377, 379, 379, 379, 379, 379, 382, 570, 400, 419, 382, 398, 383, 383, 383, 383, 383, 406, 379, 398, 543, 543, 379, 379, 379, 379, 384, 384, 384, 384, 384, 419, 406, 525, 404, 385, 385, 385, 385, 385, 406, 679, 543, 404, 379, 464, 386, 386, 386, 386, 386, 464, 515, 407, 379, 383, 404, 413, 407, 570, 906, 514, 906, 407, 383, 383, 379, 464, 432, 384, 384, 493, 432, 385, 432, 407, 544, 383, 385, 384, 407, 391, 391, 391, 391, 391, 383, 413, 385, 386, 465, 384, 384, 679, 386, 465, 385, 465, 511, 386, 385, 390, 390, 390, 390, 390, 510, 493, 414, 413, 413, 386, 414, 414, 414, 414, 386, 387, 387, 387, 387, 387, 535, 493, 391, 414, 410, 544, 544, 391, 493, 435, 390, 408, 391, 435, 396, 435, 536, 390, 396, 396, 396, 396, 414, 535, 391, 725, 414, 544, 387, 391, 508, 436, 390, 390, 396, 436, 410, 436, 387, 536, 390, 546, 507, 428, 387, 443, 387, 428, 387, 428, 396, 387, 410, 408, 410, 443, 410, 396, 408, 410, 387, 504, 502, 408, 546, 428, 387, 443, 387, 388, 388, 388, 388, 388, 410, 408, 410, 725, 498, 496, 408, 425, 425, 425, 425, 425, 388, 446, 428, 439, 388, 388, 388, 388, 426, 426, 426, 426, 426, 444, 388, 433, 433, 433, 433, 433, 490, 439, 448, 440, 548, 489, 439, 440, 445, 440, 441, 441, 441, 441, 441, 446, 388, 388, 442, 486, 485, 425, 442, 442, 442, 442, 445, 548, 425, 433, 482, 445, 448, 446, 426, 444, 461, 472, 446, 513, 461, 426, 461, 425, 513, 444, 513, 441, 433, 942, 457, 942, 433, 460, 448, 448, 426, 444, 460, 438, 460, 467, 477, 441, 437, 467, 477, 467, 477, 518, 441, 449, 449, 449, 449, 449, 449, 449, 449, 449, 449, 605, 469, 460, 609, 449, 449, 449, 449, 449, 449, 450, 450, 450, 450, 450, 484, 450, 431, 518, 484, 450, 484, 450, 605, 460, 430, 609, 449, 449, 449, 449, 449, 449, 451, 451, 451, 451, 451, 488, 451, 518, 518, 488, 451, 488, 451, 469, 480, 480, 480, 480, 480, 422, 469, 481, 481, 481, 481, 481, 421, 494, 610, 670, 450, 494, 494, 494, 494, 469, 487, 487, 487, 487, 487, 499, 499, 499, 499, 499, 500, 500, 500, 500, 500, 610, 670, 451, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 455, 456, 456, 456, 456, 456, 415, 371, 466, 466, 466, 466, 466, 491, 491, 491, 491, 491, 456, 456, 456, 456, 456, 456, 456, 456, 456, 456, 602, 602, 602, 602, 456, 456, 456, 456, 456, 456, 501, 501, 501, 501, 501, 506, 519, 370, 368, 506, 509, 506, 491, 367, 509, 466, 509, 456, 456, 456, 456, 456, 456, 466, 361, 519, 520, 359, 491, 519, 527, 520, 354, 520, 527, 491, 527, 466, 471, 471, 471, 471, 471, 352, 471, 523, 523, 523, 523, 523, 512, 512, 512, 512, 512, 471, 471, 471, 471, 471, 471, 471, 471, 471, 471, 604, 604, 604, 604, 471, 471, 471, 471, 471, 471, 524, 524, 524, 524, 524, 948, 348, 948, 615, 512, 530, 530, 530, 530, 530, 615, 471, 471, 471, 471, 471, 471, 471, 474, 474, 474, 474, 474, 512, 534, 615, 606, 512, 347, 534, 606, 534, 474, 345, 330, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 952, 329, 952, 606, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 474, 327, 622, 474, 475, 475, 475, 475, 475, 475, 475, 475, 475, 475, 636, 636, 636, 636, 475, 475, 475, 475, 475, 475, 622, 622, 516, 516, 516, 516, 516, 531, 531, 531, 531, 531, 537, 537, 537, 537, 537, 475, 475, 475, 475, 475, 475, 476, 476, 476, 476, 476, 476, 476, 476, 476, 476, 516, 975, 634, 975, 476, 476, 476, 476, 476, 476, 538, 538, 538, 538, 538, 539, 326, 652, 634, 539, 571, 539, 516, 516, 571, 634, 571, 476, 476, 476, 476, 476, 476, 542, 542, 542, 542, 542, 545, 652, 652, 671, 545, 545, 545, 545, 550, 550, 550, 550, 550, 551, 551, 551, 551, 551, 552, 552, 552, 552, 552, 557, 580, 751, 671, 557, 557, 557, 557, 562, 562, 562, 562, 562, 563, 563, 563, 563, 563, 564, 564, 564, 564, 564, 323, 542, 542, 574, 574, 574, 574, 574, 575, 579, 319, 580, 575, 317, 575, 578, 578, 578, 578, 578, 608, 902, 612, 542, 608, 902, 608, 612, 579, 612, 580, 751, 579, 585, 580, 585, 585, 585, 585, 585, 585, 587, 313, 587, 587, 587, 587, 587, 587, 578, 614, 618, 815, 654, 614, 618, 614, 618, 654, 312, 654, 815, 598, 598, 598, 598, 598, 986, 578, 986, 310, 309, 578, 582, 582, 582, 582, 582, 582, 582, 582, 582, 582, 598, 598, 598, 598, 582, 582, 582, 582, 582, 582, 584, 584, 584, 584, 584, 584, 584, 601, 601, 601, 601, 815, 584, 308, 584, 616, 298, 582, 582, 582, 582, 582, 582, 601, 586, 586, 586, 586, 586, 586, 586, 684, 684, 684, 684, 584, 586, 584, 586, 601, 603, 603, 603, 603, 607, 685, 601, 294, 611, 607, 661, 607, 619, 603, 611, 661, 619, 661, 619, 586, 616, 586, 588, 588, 588, 588, 588, 616, 685, 623, 611, 285, 603, 623, 607, 623, 603, 672, 672, 672, 672, 672, 616, 588, 588, 588, 588, 626, 613, 613, 613, 613, 613, 625, 657, 687, 607, 625, 657, 625, 624, 624, 624, 624, 624, 588, 626, 628, 631, 627, 626, 628, 631, 628, 631, 588, 657, 284, 687, 632, 632, 632, 632, 632, 635, 279, 269, 588, 589, 589, 589, 589, 589, 613, 633, 710, 711, 268, 633, 624, 633, 613, 627, 264, 638, 638, 638, 638, 638, 589, 589, 589, 589, 624, 632, 613, 638, 263, 710, 711, 635, 627, 624, 589, 231, 627, 640, 640, 640, 640, 640, 1022, 632, 1022, 812, 653, 635, 632, 640, 812, 653, 589, 653, 635, 230, 866, 589, 590, 590, 590, 590, 590, 644, 644, 644, 644, 644, 658, 644, 227, 226, 658, 644, 658, 644, 715, 225, 653, 590, 590, 590, 590, 223, 645, 645, 645, 645, 645, 674, 645, 221, 716, 674, 645, 674, 645, 812, 219, 715, 653, 590, 646, 646, 646, 646, 646, 663, 866, 590, 1026, 590, 1026, 218, 646, 716, 644, 217, 647, 647, 647, 647, 647, 590, 647, 1027, 215, 1027, 647, 720, 647, 590, 591, 591, 591, 591, 591, 645, 648, 648, 648, 648, 648, 692, 648, 211, 210, 692, 648, 692, 648, 663, 720, 591, 591, 591, 591, 207, 206, 663, 659, 659, 659, 659, 659, 660, 660, 660, 660, 660, 1047, 647, 1047, 663, 673, 673, 673, 673, 673, 591, 677, 677, 677, 677, 677, 591, 689, 689, 689, 689, 689, 648, 700, 683, 683, 683, 683, 683, 707, 721, 202, 591, 592, 592, 592, 592, 592, 659, 731, 733, 696, 700, 660, 659, 696, 700, 696, 707, 660, 701, 197, 707, 721, 592, 592, 592, 592, 196, 192, 659, 683, 731, 733, 191, 660, 690, 690, 690, 690, 690, 691, 691, 691, 691, 691, 190, 683, 695, 695, 695, 695, 695, 701, 683, 592, 592, 699, 699, 699, 699, 699, 706, 706, 706, 706, 706, 712, 712, 712, 712, 712, 701, 1059, 188, 1059, 701, 592, 593, 593, 593, 593, 593, 713, 713, 713, 713, 713, 843, 185, 777, 699, 184, 706, 717, 717, 717, 717, 717, 593, 593, 593, 593, 718, 718, 718, 718, 718, 181, 777, 699, 843, 719, 777, 699, 706, 706, 719, 1060, 719, 1060, 790, 593, 722, 722, 722, 722, 722, 958, 790, 180, 593, 723, 723, 723, 723, 723, 958, 729, 729, 729, 729, 729, 790, 593, 594, 594, 594, 594, 594, 730, 730, 730, 730, 735, 735, 735, 735, 735, 736, 736, 736, 736, 736, 175, 173, 594, 594, 594, 594, 737, 737, 737, 737, 737, 738, 738, 738, 738, 166, 958, 775, 844, 594, 743, 743, 743, 743, 743, 153, 729, 729, 744, 744, 744, 744, 744, 775, 594, 745, 745, 745, 745, 745, 775, 844, 594, 595, 595, 595, 595, 595, 729, 746, 144, 142, 854, 746, 856, 746, 749, 749, 749, 749, 749, 132, 758, 758, 758, 799, 758, 758, 758, 755, 755, 755, 755, 755, 758, 854, 758, 856, 131, 799, 762, 762, 762, 762, 762, 782, 595, 832, 799, 785, 782, 595, 782, 785, 595, 117, 595, 758, 113, 758, 760, 760, 760, 755, 760, 760, 760, 112, 595, 832, 832, 785, 760, 595, 760, 791, 595, 596, 596, 596, 596, 596, 755, 762, 111, 874, 755, 763, 763, 763, 763, 763, 1077, 762, 1077, 760, 109, 760, 772, 772, 772, 772, 772, 786, 874, 762, 805, 786, 874, 786, 596, 108, 104, 764, 764, 764, 764, 764, 789, 791, 596, 102, 763, 789, 805, 789, 596, 791, 596, 805, 596, 98, 97, 596, 765, 765, 765, 765, 765, 781, 763, 791, 596, 793, 781, 763, 781, 793, 596, 793, 596, 597, 597, 597, 597, 597, 764, 93, 827, 766, 766, 766, 766, 766, 764, 91, 764, 827, 1079, 88, 1079, 781, 597, 597, 597, 597, 81, 76, 764, 796, 827, 765, 597, 796, 75, 796, 764, 765, 767, 767, 767, 767, 767, 781, 71, 876, 768, 768, 768, 768, 768, 800, 877, 765, 597, 597, 599, 599, 599, 599, 599, 766, 766, 769, 769, 769, 769, 769, 876, 57, 771, 771, 771, 771, 771, 877, 55, 599, 599, 599, 599, 798, 767, 766, 768, 798, 802, 798, 1075, 800, 802, 767, 802, 806, 599, 44, 804, 1075, 41, 768, 804, 771, 804, 800, 767, 39, 769, 768, 829, 599, 1075, 769, 800, 829, 35, 829, 769, 599, 600, 600, 600, 600, 600, 771, 771, 828, 835, 806, 769, 828, 835, 828, 835, 769, 787, 787, 787, 787, 787, 803, 803, 803, 803, 803, 880, 806, 1090, 1098, 1090, 1098, 806, 773, 773, 773, 773, 773, 809, 809, 809, 809, 809, 600, 881, 31, 24, 833, 600, 880, 809, 600, 833, 600, 833, 774, 774, 774, 774, 774, 861, 21, 836, 787, 861, 600, 861, 881, 836, 787, 600, 773, 885, 600, 637, 637, 637, 637, 637, 637, 637, 637, 637, 637, 836, 787, 773, 886, 637, 637, 637, 637, 637, 637, 773, 885, 20, 933, 774, 892, 894, 939, 933, 774, 933, 939, 15, 939, 774, 946, 886, 637, 637, 637, 637, 637, 637, 641, 641, 641, 774, 1000, 892, 894, 13, 774, 797, 797, 797, 797, 797, 946, 946, 641, 641, 641, 641, 641, 641, 641, 641, 641, 641, 3, 1000, 1001, 1039, 641, 641, 641, 641, 641, 641, 1100, 0, 1100, 0, 797, 845, 845, 845, 845, 845, 846, 846, 846, 846, 846, 1001, 1039, 641, 641, 641, 641, 641, 641, 643, 643, 643, 797, 797, 850, 850, 850, 850, 850, 858, 858, 858, 858, 858, 0, 643, 643, 643, 643, 643, 643, 643, 643, 643, 643, 0, 1040, 0, 0, 643, 643, 643, 643, 643, 643, 817, 817, 817, 817, 817, 943, 817, 0, 937, 943, 817, 943, 817, 937, 1040, 937, 0, 643, 643, 643, 643, 643, 643, 649, 649, 649, 649, 649, 940, 649, 0, 0, 945, 649, 940, 649, 945, 0, 945, 0, 649, 649, 649, 649, 649, 649, 649, 649, 649, 649, 940, 0, 0, 817, 649, 649, 649, 649, 649, 649, 859, 859, 859, 859, 859, 860, 860, 860, 860, 860, 864, 864, 864, 864, 864, 0, 649, 649, 649, 649, 649, 649, 649, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 650, 651, 651, 651, 651, 651, 651, 651, 651, 651, 651, 0, 0, 0, 0, 651, 651, 651, 651, 651, 651, 818, 818, 818, 818, 818, 949, 818, 0, 970, 949, 818, 949, 818, 970, 0, 970, 0, 651, 651, 651, 651, 651, 651, 665, 665, 665, 665, 665, 1045, 665, 878, 878, 878, 878, 878, 879, 879, 879, 879, 879, 665, 665, 665, 665, 665, 665, 665, 665, 665, 665, 1045, 1045, 0, 818, 665, 665, 665, 665, 665, 665, 820, 820, 820, 820, 820, 834, 834, 834, 834, 834, 0, 965, 820, 0, 0, 965, 665, 665, 665, 665, 665, 665, 665, 666, 666, 666, 666, 666, 666, 666, 666, 666, 666, 965, 0, 0, 0, 666, 666, 666, 666, 666, 666, 821, 821, 821, 821, 821, 873, 873, 873, 873, 873, 820, 834, 821, 0, 0, 834, 0, 666, 666, 666, 666, 666, 666, 667, 667, 667, 667, 667, 667, 667, 667, 667, 667, 834, 0, 0, 873, 667, 667, 667, 667, 667, 667, 0, 822, 822, 822, 822, 822, 0, 822, 967, 0, 821, 822, 967, 822, 967, 873, 873, 667, 667, 667, 667, 667, 667, 668, 668, 668, 668, 668, 668, 668, 668, 668, 668, 0, 0, 0, 0, 668, 668, 668, 668, 668, 668, 0, 0, 788, 788, 788, 788, 788, 837, 0, 0, 0, 822, 882, 882, 882, 882, 882, 668, 668, 668, 668, 668, 668, 757, 757, 757, 757, 757, 757, 757, 757, 757, 757, 0, 0, 0, 0, 757, 757, 757, 757, 757, 757, 947, 823, 823, 823, 823, 823, 788, 973, 0, 0, 837, 973, 788, 823, 0, 0, 837, 757, 757, 757, 757, 757, 757, 770, 770, 770, 770, 770, 788, 973, 947, 0, 837, 839, 839, 839, 839, 839, 0, 839, 883, 883, 883, 883, 883, 887, 887, 887, 887, 887, 0, 1020, 947, 947, 823, 1020, 770, 888, 888, 888, 888, 888, 896, 896, 896, 896, 896, 897, 897, 897, 897, 897, 770, 1020, 770, 0, 770, 0, 0, 770, 898, 898, 898, 898, 898, 0, 839, 903, 903, 903, 903, 903, 0, 0, 770, 0, 770, 808, 808, 808, 808, 808, 808, 808, 808, 808, 808, 0, 0, 0, 0, 808, 808, 808, 808, 808, 808, 853, 853, 853, 853, 853, 904, 904, 904, 904, 904, 905, 905, 905, 905, 905, 0, 0, 808, 808, 808, 808, 808, 808, 813, 813, 813, 813, 813, 0, 813, 909, 909, 909, 909, 909, 0, 0, 853, 0, 0, 813, 813, 813, 813, 813, 813, 813, 813, 813, 813, 0, 0, 0, 853, 813, 813, 813, 813, 813, 813, 853, 944, 944, 944, 944, 944, 953, 953, 953, 953, 953, 981, 981, 981, 981, 981, 813, 813, 813, 813, 813, 813, 813, 816, 816, 816, 816, 816, 1041, 972, 1017, 0, 1041, 816, 972, 1017, 972, 1017, 0, 0, 816, 816, 816, 816, 816, 816, 816, 816, 816, 816, 1041, 0, 0, 0, 816, 816, 816, 816, 816, 816, 870, 870, 870, 870, 870, 0, 891, 891, 891, 891, 891, 989, 989, 989, 989, 989, 816, 816, 816, 816, 816, 816, 816, 819, 819, 819, 819, 819, 819, 819, 819, 819, 819, 0, 870, 0, 0, 819, 819, 819, 819, 819, 819, 914, 914, 914, 914, 914, 1023, 0, 0, 955, 1023, 870, 1023, 0, 955, 870, 891, 891, 819, 819, 819, 819, 819, 819, 824, 824, 824, 824, 824, 1002, 1002, 1002, 1002, 1002, 0, 0, 0, 0, 891, 0, 0, 824, 824, 824, 824, 824, 824, 824, 824, 824, 824, 0, 0, 0, 914, 824, 824, 824, 824, 824, 824, 955, 1003, 1003, 1003, 1003, 1003, 1013, 1013, 1013, 1013, 1013, 1018, 1018, 1018, 1018, 1018, 0, 824, 824, 824, 824, 824, 824, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 825, 826, 826, 826, 826, 826, 826, 826, 826, 826, 826, 0, 0, 0, 0, 826, 826, 826, 826, 826, 826, 912, 912, 912, 912, 912, 924, 924, 924, 924, 924, 1038, 1088, 0, 0, 1038, 1088, 1038, 826, 826, 826, 826, 826, 826, 840, 840, 840, 840, 840, 840, 840, 840, 840, 840, 1088, 912, 0, 924, 840, 840, 840, 840, 840, 840, 916, 916, 916, 916, 916, 0, 0, 966, 0, 0, 912, 966, 0, 966, 912, 924, 924, 840, 840, 840, 840, 840, 840, 841, 841, 841, 841, 841, 841, 841, 841, 841, 841, 0, 966, 916, 0, 841, 841, 841, 841, 841, 841, 0, 915, 915, 915, 915, 915, 0, 1019, 0, 0, 916, 0, 1019, 966, 1019, 916, 0, 841, 841, 841, 841, 841, 841, 842, 842, 842, 842, 842, 842, 842, 842, 842, 842, 0, 0, 0, 0, 842, 842, 842, 842, 842, 842, 0, 915, 917, 917, 917, 917, 917, 0, 0, 974, 941, 915, 918, 918, 918, 918, 918, 842, 842, 842, 842, 842, 842, 915, 919, 919, 919, 919, 919, 920, 920, 920, 920, 920, 0, 0, 921, 921, 921, 921, 921, 1046, 0, 0, 0, 917, 0, 0, 922, 922, 922, 922, 922, 917, 1042, 917, 941, 974, 1042, 918, 1042, 974, 941, 1044, 0, 918, 0, 917, 1044, 0, 1044, 1046, 0, 920, 921, 917, 919, 919, 941, 974, 0, 918, 920, 925, 925, 925, 925, 925, 0, 921, 0, 922, 0, 1046, 1046, 920, 922, 921, 919, 0, 0, 922, 926, 926, 926, 926, 926, 0, 927, 927, 927, 927, 927, 922, 1076, 0, 0, 0, 922, 923, 923, 923, 923, 923, 0, 0, 0, 928, 928, 928, 928, 928, 1056, 0, 0, 0, 925, 1056, 0, 1056, 926, 929, 929, 929, 929, 929, 931, 931, 931, 931, 931, 927, 923, 1067, 0, 926, 927, 1076, 1067, 0, 1067, 927, 0, 926, 928, 1071, 1076, 0, 923, 1071, 923, 1071, 923, 927, 0, 923, 0, 0, 927, 1076, 928, 930, 930, 930, 930, 930, 0, 928, 0, 1074, 923, 1085, 923, 1074, 929, 1074, 1085, 0, 1085, 931, 938, 938, 938, 938, 938, 961, 961, 961, 961, 961, 963, 963, 963, 963, 963, 930, 0, 961, 0, 0, 0, 0, 963, 1051, 1051, 1051, 1051, 1051, 0, 971, 971, 971, 971, 971, 930, 0, 0, 0, 930, 976, 976, 976, 976, 976, 977, 977, 977, 977, 977, 938, 1087, 0, 1037, 938, 1087, 1021, 1087, 1037, 961, 1037, 971, 0, 0, 963, 1086, 1086, 1086, 1086, 1086, 0, 0, 938, 956, 956, 956, 956, 956, 956, 956, 956, 956, 956, 971, 971, 1037, 0, 956, 956, 956, 956, 956, 956, 976, 0, 0, 0, 0, 977, 978, 978, 978, 978, 978, 1021, 0, 0, 1037, 1021, 0, 956, 956, 956, 956, 956, 956, 959, 959, 959, 959, 959, 959, 959, 959, 959, 959, 1021, 0, 0, 0, 959, 959, 959, 959, 959, 959, 983, 983, 983, 983, 983, 984, 984, 984, 984, 984, 1094, 1094, 1094, 1094, 1094, 0, 978, 959, 959, 959, 959, 959, 959, 960, 960, 960, 960, 960, 960, 960, 960, 960, 960, 0, 0, 0, 0, 960, 960, 960, 960, 960, 960, 1095, 0, 0, 0, 0, 1095, 0, 1095, 0, 0, 983, 0, 0, 0, 0, 984, 0, 960, 960, 960, 960, 960, 960, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 962, 964, 964, 964, 964, 964, 964, 964, 964, 964, 964, 0, 0, 0, 0, 964, 964, 964, 964, 964, 964, 982, 982, 982, 982, 982, 985, 985, 985, 985, 985, 992, 992, 992, 992, 992, 0, 0, 964, 964, 964, 964, 964, 964, 993, 993, 993, 993, 993, 994, 994, 994, 994, 994, 0, 0, 0, 0, 982, 995, 995, 995, 995, 995, 0, 992, 996, 996, 996, 996, 996, 0, 0, 0, 982, 993, 0, 0, 0, 985, 0, 982, 0, 994, 992, 0, 0, 0, 992, 997, 997, 997, 997, 997, 0, 0, 0, 993, 993, 0, 0, 0, 994, 0, 0, 0, 994, 998, 998, 998, 998, 998, 995, 999, 999, 999, 999, 999, 0, 996, 1004, 1004, 1004, 1004, 1004, 1005, 1005, 1005, 1005, 1005, 1006, 1006, 1006, 1006, 1006, 1007, 1007, 1007, 1007, 1007, 0, 1089, 997, 1008, 1008, 1008, 1008, 1008, 1009, 1009, 1009, 1009, 1009, 1010, 1010, 1010, 1010, 1010, 0, 0, 998, 1014, 1014, 1014, 1014, 1014, 999, 1015, 1015, 1015, 1015, 1015, 1004, 1004, 0, 0, 0, 0, 1005, 0, 0, 0, 0, 1006, 0, 0, 0, 0, 1007, 1089, 0, 0, 0, 1089, 1004, 1014, 1008, 0, 0, 0, 0, 1009, 0, 0, 0, 0, 1010, 0, 0, 0, 0, 1089, 0, 0, 1014, 0, 0, 0, 1014, 0, 1015, 1028, 1028, 1028, 1028, 1028, 1028, 1028, 1028, 1028, 1028, 0, 0, 0, 0, 1028, 1028, 1028, 1028, 1028, 1028, 1031, 1031, 1031, 1031, 1031, 1048, 1048, 1048, 1048, 1048, 0, 0, 1031, 0, 0, 0, 0, 1028, 1028, 1028, 1028, 1028, 1028, 1029, 1029, 1029, 1029, 1029, 1029, 1029, 1029, 1029, 1029, 0, 0, 0, 0, 1029, 1029, 1029, 1029, 1029, 1029, 1032, 1032, 1032, 1032, 1032, 1043, 1043, 1043, 1043, 1043, 1031, 0, 1032, 0, 0, 1048, 0, 1029, 1029, 1029, 1029, 1029, 1029, 1030, 1030, 1030, 1030, 1030, 1030, 1030, 1030, 1030, 1030, 0, 0, 0, 0, 1030, 1030, 1030, 1030, 1030, 1030, 1053, 1053, 1053, 1053, 1053, 0, 1043, 0, 0, 0, 1032, 0, 0, 0, 0, 1043, 0, 1030, 1030, 1030, 1030, 1030, 1030, 1033, 1033, 1033, 1033, 1033, 1043, 0, 0, 0, 0, 0, 0, 1033, 0, 0, 0, 0, 1033, 1033, 1033, 1033, 1033, 1033, 1033, 1033, 1033, 1033, 0, 0, 0, 1053, 1033, 1033, 1033, 1033, 1033, 1033, 1052, 1052, 1052, 1052, 1052, 1054, 1054, 1054, 1054, 1054, 0, 0, 0, 0, 0, 0, 1033, 1033, 1033, 1033, 1033, 1033, 1033, 1035, 1035, 1035, 1035, 1035, 1035, 1035, 1035, 1035, 1035, 0, 1052, 0, 0, 1035, 1035, 1035, 1035, 1035, 1035, 1055, 1055, 1055, 1055, 1055, 0, 0, 0, 0, 0, 1052, 0, 0, 0, 1052, 1054, 0, 1035, 1035, 1035, 1035, 1035, 1035, 1064, 1064, 1064, 1064, 1064, 0, 0, 0, 0, 0, 0, 0, 1064, 1072, 1072, 1072, 1072, 1072, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1055, 1061, 1061, 1061, 1061, 1061, 1061, 1061, 1061, 1061, 1061, 0, 0, 0, 0, 1061, 1061, 1061, 1061, 1061, 1061, 0, 0, 1064, 0, 0, 1073, 1073, 1073, 1073, 1073, 0, 1072, 0, 0, 0, 1072, 0, 1061, 1061, 1061, 1061, 1061, 1061, 1062, 1062, 1062, 1062, 1062, 1062, 1062, 1062, 1062, 1062, 1072, 0, 0, 0, 1062, 1062, 1062, 1062, 1062, 1062, 1078, 1078, 1078, 1078, 1078, 0, 0, 0, 0, 0, 0, 1073, 0, 0, 0, 1073, 0, 1062, 1062, 1062, 1062, 1062, 1062, 1063, 1063, 1063, 1063, 1063, 1063, 1063, 1063, 1063, 1063, 1073, 0, 0, 0, 1063, 1063, 1063, 1063, 1063, 1063, 1101, 1101, 1101, 1101, 1101, 0, 0, 0, 0, 0, 1078, 0, 1101, 0, 0, 0, 0, 1063, 1063, 1063, 1063, 1063, 1063, 1065, 1065, 1065, 1065, 1065, 1065, 1065, 1065, 1065, 1065, 0, 0, 0, 0, 1065, 1065, 1065, 1065, 1065, 1065, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1101, 0, 0, 0, 0, 0, 0, 1065, 1065, 1065, 1065, 1065, 1065, 1066, 1066, 1066, 1066, 1066, 1066, 1066, 1066, 1066, 1066, 0, 0, 0, 0, 1066, 1066, 1066, 1066, 1066, 1066, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1066, 1066, 1066, 1066, 1066, 1066, 1080, 1080, 1080, 1080, 1080, 1080, 1080, 1080, 1080, 1080, 0, 0, 0, 0, 1080, 1080, 1080, 1080, 1080, 1080, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1080, 1080, 1080, 1080, 1080, 1080, 1081, 1081, 1081, 1081, 1081, 1081, 1081, 1081, 1081, 1081, 0, 0, 0, 0, 1081, 1081, 1081, 1081, 1081, 1081, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1081, 1081, 1081, 1081, 1081, 1081, 1082, 1082, 1082, 1082, 1082, 1082, 1082, 1082, 1082, 1082, 0, 0, 0, 0, 1082, 1082, 1082, 1082, 1082, 1082, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1082, 1082, 1082, 1082, 1082, 1082, 1083, 1083, 1083, 1083, 1083, 1083, 1083, 1083, 1083, 1083, 0, 0, 0, 0, 1083, 1083, 1083, 1083, 1083, 1083, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1083, 1083, 1083, 1083, 1083, 1083, 1084, 1084, 1084, 1084, 1084, 1084, 1084, 1084, 1084, 1084, 0, 0, 0, 0, 1084, 1084, 1084, 1084, 1084, 1084, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1084, 1084, 1084, 1084, 1084, 1084, 1091, 1091, 1091, 1091, 1091, 1091, 1091, 1091, 1091, 1091, 0, 0, 0, 0, 1091, 1091, 1091, 1091, 1091, 1091, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1091, 1091, 1091, 1091, 1091, 1091, 1092, 1092, 1092, 1092, 1092, 1092, 1092, 1092, 1092, 1092, 0, 0, 0, 0, 1092, 1092, 1092, 1092, 1092, 1092, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1092, 1092, 1092, 1092, 1092, 1092, 1093, 1093, 1093, 1093, 1093, 1093, 1093, 1093, 1093, 1093, 0, 0, 0, 0, 1093, 1093, 1093, 1093, 1093, 1093, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1093, 1093, 1093, 1093, 1093, 1093, 1099, 1099, 1099, 1099, 1099, 1099, 1099, 1099, 1099, 1099, 0, 0, 0, 0, 1099, 1099, 1099, 1099, 1099, 1099, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1099, 1099, 1099, 1099, 1099, 1099, 1103, 1103, 0, 1103, 1103, 1103, 1103, 1103, 1103, 1104, 1104, 1104, 1104, 1105, 1105, 0, 1105, 1105, 1105, 1105, 1105, 1105, 1106, 0, 0, 1106, 1107, 1107, 0, 1107, 1107, 1108, 1108, 0, 1108, 1108, 1108, 1108, 1108, 1108, 1109, 1109, 1109, 1109, 1109, 1109, 1109, 1109, 1109, 1110, 1110, 0, 1110, 1110, 1110, 1110, 1110, 1110, 1111, 1111, 1111, 1111, 1111, 1111, 1111, 1111, 1111, 1112, 1112, 1112, 1112, 1112, 1112, 1112, 1112, 1112, 1114, 0, 1114, 1114, 1115, 1115, 0, 1115, 1115, 1115, 1115, 1115, 1115, 1116, 1116, 1116, 1116, 1116, 1116, 1116, 1116, 1116, 1117, 1117, 1117, 1117, 1117, 1117, 1117, 1117, 1117, 1118, 1118, 1118, 1118, 1118, 1118, 1118, 1118, 1118, 1119, 1119, 0, 1119, 1119, 1119, 1119, 1119, 1119, 1120, 1120, 1120, 1120, 1120, 1120, 1120, 1120, 1120, 1121, 1121, 1121, 1121, 1121, 1121, 1121, 1121, 1121, 1122, 1122, 1122, 1122, 1122, 1122, 1122, 1122, 1122, 1123, 1123, 1123, 0, 1123, 1123, 1123, 1123, 1124, 1124, 1124, 1124, 1124, 1124, 1124, 1124, 1124, 1125, 1125, 1125, 1125, 1125, 1125, 1125, 1125, 1125, 1126, 1126, 1126, 1126, 1126, 1126, 1126, 1126, 1126, 1127, 1127, 0, 1127, 1127, 1127, 1127, 1127, 1127, 1128, 1128, 0, 1128, 1128, 1128, 1128, 1128, 1128, 1129, 1129, 1129, 1129, 1129, 1129, 1129, 1129, 1129, 1130, 1130, 1130, 1130, 1130, 1130, 1130, 1130, 1130, 1131, 1131, 1131, 1131, 1131, 1131, 1131, 1131, 1131, 1132, 1132, 1132, 1132, 1132, 1132, 1132, 1132, 1132, 1133, 1133, 1133, 1133, 1133, 1133, 1133, 1133, 1133, 1134, 1134, 1134, 1134, 1134, 1134, 1134, 1134, 1134, 1135, 1135, 1135, 1135, 1135, 1135, 1135, 1135, 1135, 1136, 1136, 0, 0, 1136, 1136, 1136, 1136, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102, 1102 } ; static yy_state_type yy_last_accepting_state; static char *yy_last_accepting_cpos; extern int yy_flex_debug; int yy_flex_debug = 0; /* The intent behind this definition is that it'll catch * any uses of REJECT which flex missed. */ #define REJECT reject_used_but_not_detected #define yymore() yymore_used_but_not_detected #define YY_MORE_ADJ 0 #define YY_RESTORE_YY_MORE_OFFSET char *yytext; #line 1 "css.l" #line 13 "css.l" /* Lex source for CSS tokenizing. Taken from http://www.w3.org/TR/CSS21/grammar.html#q2 Copyright (C) 2006, 2009-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #define YY_NO_INPUT #include "css-tokens.h" #if defined __clang__ || __GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 5) #pragma GCC diagnostic ignored "-Wunknown-pragmas" // clang mourns about the next one #pragma GCC diagnostic ignored "-Wunused-function" #pragma GCC diagnostic ignored "-Wunused-macros" #pragma GCC diagnostic ignored "-Wunused-parameter" #pragma GCC diagnostic ignored "-Wsign-compare" #pragma GCC diagnostic ignored "-Wswitch-default" #pragma GCC diagnostic ignored "-Wunreachable-code" // clang #pragma clang diagnostic ignored "-Wshorten-64-to-32" #ifndef __clang__ #pragma GCC diagnostic ignored "-Wsuggest-attribute=pure" #endif #endif #line 2452 "css.c" #line 2453 "css.c" #define INITIAL 0 #ifndef YY_NO_UNISTD_H /* Special case for "unistd.h", since it is non-ANSI. We include it way * down here because we want the user's section 1 to have been scanned first. * The user has a chance to override it with an option. */ #include #endif #ifndef YY_EXTRA_TYPE #define YY_EXTRA_TYPE void * #endif static int yy_init_globals ( void ); /* Accessor methods to globals. These are made visible to non-reentrant scanners for convenience. */ int yylex_destroy ( void ); int yyget_debug ( void ); void yyset_debug ( int debug_flag ); YY_EXTRA_TYPE yyget_extra ( void ); void yyset_extra ( YY_EXTRA_TYPE user_defined ); FILE *yyget_in ( void ); void yyset_in ( FILE * _in_str ); FILE *yyget_out ( void ); void yyset_out ( FILE * _out_str ); int yyget_leng ( void ); char *yyget_text ( void ); int yyget_lineno ( void ); void yyset_lineno ( int _line_number ); /* Macros after this point can all be overridden by user definitions in * section 1. */ #ifndef YY_SKIP_YYWRAP #ifdef __cplusplus extern "C" int yywrap ( void ); #else extern int yywrap ( void ); #endif #endif #ifndef YY_NO_UNPUT #endif #ifndef yytext_ptr static void yy_flex_strncpy ( char *, const char *, int ); #endif #ifdef YY_NEED_STRLEN static int yy_flex_strlen ( const char * ); #endif #ifndef YY_NO_INPUT #ifdef __cplusplus static int yyinput ( void ); #else static int input ( void ); #endif #endif /* Amount of stuff to slurp up with each read. */ #ifndef YY_READ_BUF_SIZE #ifdef __ia64__ /* On IA-64, the buffer size is 16k, not 8k */ #define YY_READ_BUF_SIZE 16384 #else #define YY_READ_BUF_SIZE 8192 #endif /* __ia64__ */ #endif /* Copy whatever the last rule matched to the standard output. */ #ifndef ECHO /* This used to be an fputs(), but since the string might contain NUL's, * we now use fwrite(). */ #define ECHO do { if (fwrite( yytext, (size_t) yyleng, 1, yyout )) {} } while (0) #endif /* Gets input and stuffs it into "buf". number of characters read, or YY_NULL, * is returned in "result". */ #ifndef YY_INPUT #define YY_INPUT(buf,result,max_size) \ if ( YY_CURRENT_BUFFER_LVALUE->yy_is_interactive ) \ { \ int c = '*'; \ int n; \ for ( n = 0; n < max_size && \ (c = getc( yyin )) != EOF && c != '\n'; ++n ) \ buf[n] = (char) c; \ if ( c == '\n' ) \ buf[n++] = (char) c; \ if ( c == EOF && ferror( yyin ) ) \ YY_FATAL_ERROR( "input in flex scanner failed" ); \ result = n; \ } \ else \ { \ errno=0; \ while ( (result = (int) fread(buf, 1, (yy_size_t) max_size, yyin)) == 0 && ferror(yyin)) \ { \ if( errno != EINTR) \ { \ YY_FATAL_ERROR( "input in flex scanner failed" ); \ break; \ } \ errno=0; \ clearerr(yyin); \ } \ }\ \ #endif /* No semi-colon after return; correct usage is to write "yyterminate();" - * we don't want an extra ';' after the "return" because that will cause * some compilers to complain about unreachable statements. */ #ifndef yyterminate #define yyterminate() return YY_NULL #endif /* Number of entries by which start-condition stack grows. */ #ifndef YY_START_STACK_INCR #define YY_START_STACK_INCR 25 #endif /* Report a fatal error. */ #ifndef YY_FATAL_ERROR #define YY_FATAL_ERROR(msg) yy_fatal_error( msg ) #endif /* end tables serialization structures and prototypes */ /* Default declaration of generated scanner - a define so the user can * easily add parameters. */ #ifndef YY_DECL #define YY_DECL_IS_OURS 1 extern int yylex (void); #define YY_DECL int yylex (void) #endif /* !YY_DECL */ /* Code executed at the beginning of each rule, after yytext and yyleng * have been set up. */ #ifndef YY_USER_ACTION #define YY_USER_ACTION #endif /* Code executed at the end of each rule. */ #ifndef YY_BREAK #define YY_BREAK /*LINTED*/break; #endif #define YY_RULE_SETUP \ YY_USER_ACTION /** The main scanner function which does all the work. */ YY_DECL { yy_state_type yy_current_state; char *yy_cp, *yy_bp; int yy_act; if ( !(yy_init) ) { (yy_init) = 1; #ifdef YY_USER_INIT YY_USER_INIT; #endif if ( ! (yy_start) ) (yy_start) = 1; /* first start state */ if ( ! yyin ) yyin = stdin; if ( ! yyout ) yyout = stdout; if ( ! YY_CURRENT_BUFFER ) { yyensure_buffer_stack (); YY_CURRENT_BUFFER_LVALUE = yy_create_buffer( yyin, YY_BUF_SIZE ); } yy_load_buffer_state( ); } { #line 112 "css.l" #line 2671 "css.c" while ( /*CONSTCOND*/1 ) /* loops until end-of-file is reached */ { yy_cp = (yy_c_buf_p); /* Support of yytext. */ *yy_cp = (yy_hold_char); /* yy_bp points to the position in yy_ch_buf of the start of * the current run. */ yy_bp = yy_cp; yy_current_state = (yy_start); yy_match: do { YY_CHAR yy_c = yy_ec[YY_SC_TO_UI(*yy_cp)] ; if ( yy_accept[yy_current_state] ) { (yy_last_accepting_state) = yy_current_state; (yy_last_accepting_cpos) = yy_cp; } while ( yy_chk[yy_base[yy_current_state] + yy_c] != yy_current_state ) { yy_current_state = (int) yy_def[yy_current_state]; if ( yy_current_state >= 1103 ) yy_c = yy_meta[yy_c]; } yy_current_state = yy_nxt[yy_base[yy_current_state] + yy_c]; ++yy_cp; } while ( yy_current_state != 1102 ); yy_cp = (yy_last_accepting_cpos); yy_current_state = (yy_last_accepting_state); yy_find_action: yy_act = yy_accept[yy_current_state]; YY_DO_BEFORE_ACTION; do_action: /* This label is used only to access EOF actions. */ switch ( yy_act ) { /* beginning of action switch */ case 0: /* must back up */ /* undo the effects of YY_DO_BEFORE_ACTION */ *yy_cp = (yy_hold_char); yy_cp = (yy_last_accepting_cpos); yy_current_state = (yy_last_accepting_state); goto yy_find_action; case 1: /* rule 1 can match eol */ YY_RULE_SETUP #line 114 "css.l" {return S;} YY_BREAK case 2: /* rule 2 can match eol */ YY_RULE_SETUP #line 116 "css.l" {return COMMENT;} YY_BREAK case 3: /* rule 3 can match eol */ YY_RULE_SETUP #line 117 "css.l" /* ignore comments */ YY_BREAK case 4: /* rule 4 can match eol */ YY_RULE_SETUP #line 118 "css.l" /* unclosed comment at EOF */ YY_BREAK case 5: YY_RULE_SETUP #line 120 "css.l" {return CDO;} YY_BREAK case 6: YY_RULE_SETUP #line 121 "css.l" {return CDC;} YY_BREAK case 7: YY_RULE_SETUP #line 122 "css.l" {return INCLUDES;} YY_BREAK case 8: YY_RULE_SETUP #line 123 "css.l" {return DASHMATCH;} YY_BREAK case 9: /* rule 9 can match eol */ YY_RULE_SETUP #line 125 "css.l" {return STRING;} YY_BREAK case 10: /* rule 10 can match eol */ YY_RULE_SETUP #line 126 "css.l" {return BAD_STRING;} YY_BREAK case 11: /* rule 11 can match eol */ YY_RULE_SETUP #line 128 "css.l" {return IDENT;} YY_BREAK case 12: /* rule 12 can match eol */ YY_RULE_SETUP #line 130 "css.l" {return HASH;} YY_BREAK case 13: /* rule 13 can match eol */ YY_RULE_SETUP #line 132 "css.l" {return IMPORT_SYM;} YY_BREAK case 14: /* rule 14 can match eol */ YY_RULE_SETUP #line 133 "css.l" {return PAGE_SYM;} YY_BREAK case 15: /* rule 15 can match eol */ YY_RULE_SETUP #line 134 "css.l" {return MEDIA_SYM;} YY_BREAK case 16: YY_RULE_SETUP #line 135 "css.l" {return CHARSET_SYM;} YY_BREAK case 17: /* rule 17 can match eol */ YY_RULE_SETUP #line 137 "css.l" {return IMPORTANT_SYM;} YY_BREAK case 18: /* rule 18 can match eol */ YY_RULE_SETUP #line 139 "css.l" {return EMS;} YY_BREAK case 19: /* rule 19 can match eol */ YY_RULE_SETUP #line 140 "css.l" {return EXS;} YY_BREAK case 20: /* rule 20 can match eol */ YY_RULE_SETUP #line 141 "css.l" {return LENGTH;} YY_BREAK case 21: /* rule 21 can match eol */ YY_RULE_SETUP #line 142 "css.l" {return LENGTH;} YY_BREAK case 22: /* rule 22 can match eol */ YY_RULE_SETUP #line 143 "css.l" {return LENGTH;} YY_BREAK case 23: /* rule 23 can match eol */ YY_RULE_SETUP #line 144 "css.l" {return LENGTH;} YY_BREAK case 24: /* rule 24 can match eol */ YY_RULE_SETUP #line 145 "css.l" {return LENGTH;} YY_BREAK case 25: /* rule 25 can match eol */ YY_RULE_SETUP #line 146 "css.l" {return LENGTH;} YY_BREAK case 26: /* rule 26 can match eol */ YY_RULE_SETUP #line 147 "css.l" {return ANGLE;} YY_BREAK case 27: /* rule 27 can match eol */ YY_RULE_SETUP #line 148 "css.l" {return ANGLE;} YY_BREAK case 28: /* rule 28 can match eol */ YY_RULE_SETUP #line 149 "css.l" {return ANGLE;} YY_BREAK case 29: /* rule 29 can match eol */ YY_RULE_SETUP #line 150 "css.l" {return TIME;} YY_BREAK case 30: /* rule 30 can match eol */ YY_RULE_SETUP #line 151 "css.l" {return TIME;} YY_BREAK case 31: /* rule 31 can match eol */ YY_RULE_SETUP #line 152 "css.l" {return FREQ;} YY_BREAK case 32: /* rule 32 can match eol */ YY_RULE_SETUP #line 153 "css.l" {return FREQ;} YY_BREAK case 33: /* rule 33 can match eol */ YY_RULE_SETUP #line 154 "css.l" {return DIMENSION;} YY_BREAK case 34: YY_RULE_SETUP #line 156 "css.l" {return PERCENTAGE;} YY_BREAK case 35: YY_RULE_SETUP #line 157 "css.l" {return NUMBER;} YY_BREAK case 36: /* rule 36 can match eol */ YY_RULE_SETUP #line 159 "css.l" {return URI;} YY_BREAK case 37: /* rule 37 can match eol */ YY_RULE_SETUP #line 160 "css.l" {return URI;} YY_BREAK case 38: /* rule 38 can match eol */ YY_RULE_SETUP #line 161 "css.l" {return BAD_URI;} YY_BREAK case 39: /* rule 39 can match eol */ YY_RULE_SETUP #line 163 "css.l" {return FUNCTION;} YY_BREAK case 40: YY_RULE_SETUP #line 165 "css.l" {return *yytext;} YY_BREAK case 41: YY_RULE_SETUP #line 167 "css.l" ECHO; YY_BREAK #line 2961 "css.c" case YY_STATE_EOF(INITIAL): yyterminate(); case YY_END_OF_BUFFER: { /* Amount of text matched not including the EOB char. */ int yy_amount_of_matched_text = (int) (yy_cp - (yytext_ptr)) - 1; /* Undo the effects of YY_DO_BEFORE_ACTION. */ *yy_cp = (yy_hold_char); YY_RESTORE_YY_MORE_OFFSET if ( YY_CURRENT_BUFFER_LVALUE->yy_buffer_status == YY_BUFFER_NEW ) { /* We're scanning a new file or input source. It's * possible that this happened because the user * just pointed yyin at a new source and called * yylex(). If so, then we have to assure * consistency between YY_CURRENT_BUFFER and our * globals. Here is the right place to do so, because * this is the first action (other than possibly a * back-up) that will match for the new input source. */ (yy_n_chars) = YY_CURRENT_BUFFER_LVALUE->yy_n_chars; YY_CURRENT_BUFFER_LVALUE->yy_input_file = yyin; YY_CURRENT_BUFFER_LVALUE->yy_buffer_status = YY_BUFFER_NORMAL; } /* Note that here we test for yy_c_buf_p "<=" to the position * of the first EOB in the buffer, since yy_c_buf_p will * already have been incremented past the NUL character * (since all states make transitions on EOB to the * end-of-buffer state). Contrast this with the test * in input(). */ if ( (yy_c_buf_p) <= &YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[(yy_n_chars)] ) { /* This was really a NUL. */ yy_state_type yy_next_state; (yy_c_buf_p) = (yytext_ptr) + yy_amount_of_matched_text; yy_current_state = yy_get_previous_state( ); /* Okay, we're now positioned to make the NUL * transition. We couldn't have * yy_get_previous_state() go ahead and do it * for us because it doesn't know how to deal * with the possibility of jamming (and we don't * want to build jamming into it because then it * will run more slowly). */ yy_next_state = yy_try_NUL_trans( yy_current_state ); yy_bp = (yytext_ptr) + YY_MORE_ADJ; if ( yy_next_state ) { /* Consume the NUL. */ yy_cp = ++(yy_c_buf_p); yy_current_state = yy_next_state; goto yy_match; } else { yy_cp = (yy_last_accepting_cpos); yy_current_state = (yy_last_accepting_state); goto yy_find_action; } } else switch ( yy_get_next_buffer( ) ) { case EOB_ACT_END_OF_FILE: { (yy_did_buffer_switch_on_eof) = 0; if ( yywrap( ) ) { /* Note: because we've taken care in * yy_get_next_buffer() to have set up * yytext, we can now set up * yy_c_buf_p so that if some total * hoser (like flex itself) wants to * call the scanner after we return the * YY_NULL, it'll still work - another * YY_NULL will get returned. */ (yy_c_buf_p) = (yytext_ptr) + YY_MORE_ADJ; yy_act = YY_STATE_EOF(YY_START); goto do_action; } else { if ( ! (yy_did_buffer_switch_on_eof) ) YY_NEW_FILE; } break; } case EOB_ACT_CONTINUE_SCAN: (yy_c_buf_p) = (yytext_ptr) + yy_amount_of_matched_text; yy_current_state = yy_get_previous_state( ); yy_cp = (yy_c_buf_p); yy_bp = (yytext_ptr) + YY_MORE_ADJ; goto yy_match; case EOB_ACT_LAST_MATCH: (yy_c_buf_p) = &YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[(yy_n_chars)]; yy_current_state = yy_get_previous_state( ); yy_cp = (yy_c_buf_p); yy_bp = (yytext_ptr) + YY_MORE_ADJ; goto yy_find_action; } break; } default: YY_FATAL_ERROR( "fatal flex scanner internal error--no action found" ); } /* end of action switch */ } /* end of scanning one token */ } /* end of user's declarations */ } /* end of yylex */ /* yy_get_next_buffer - try to read in a new buffer * * Returns a code representing an action: * EOB_ACT_LAST_MATCH - * EOB_ACT_CONTINUE_SCAN - continue scanning from current position * EOB_ACT_END_OF_FILE - end of file */ static int yy_get_next_buffer (void) { char *dest = YY_CURRENT_BUFFER_LVALUE->yy_ch_buf; char *source = (yytext_ptr); int number_to_move, i; int ret_val; if ( (yy_c_buf_p) > &YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[(yy_n_chars) + 1] ) YY_FATAL_ERROR( "fatal flex scanner internal error--end of buffer missed" ); if ( YY_CURRENT_BUFFER_LVALUE->yy_fill_buffer == 0 ) { /* Don't try to fill the buffer, so this is an EOF. */ if ( (yy_c_buf_p) - (yytext_ptr) - YY_MORE_ADJ == 1 ) { /* We matched a single character, the EOB, so * treat this as a final EOF. */ return EOB_ACT_END_OF_FILE; } else { /* We matched some text prior to the EOB, first * process it. */ return EOB_ACT_LAST_MATCH; } } /* Try to read more data. */ /* First move last chars to start of buffer. */ number_to_move = (int) ((yy_c_buf_p) - (yytext_ptr) - 1); for ( i = 0; i < number_to_move; ++i ) *(dest++) = *(source++); if ( YY_CURRENT_BUFFER_LVALUE->yy_buffer_status == YY_BUFFER_EOF_PENDING ) /* don't do the read, it's not guaranteed to return an EOF, * just force an EOF */ YY_CURRENT_BUFFER_LVALUE->yy_n_chars = (yy_n_chars) = 0; else { int num_to_read = YY_CURRENT_BUFFER_LVALUE->yy_buf_size - number_to_move - 1; while ( num_to_read <= 0 ) { /* Not enough room in the buffer - grow it. */ /* just a shorter name for the current buffer */ YY_BUFFER_STATE b = YY_CURRENT_BUFFER_LVALUE; int yy_c_buf_p_offset = (int) ((yy_c_buf_p) - b->yy_ch_buf); if ( b->yy_is_our_buffer ) { int new_size = b->yy_buf_size * 2; if ( new_size <= 0 ) b->yy_buf_size += b->yy_buf_size / 8; else b->yy_buf_size *= 2; b->yy_ch_buf = (char *) /* Include room in for 2 EOB chars. */ yyrealloc( (void *) b->yy_ch_buf, (yy_size_t) (b->yy_buf_size + 2) ); } else /* Can't grow it, we don't own it. */ b->yy_ch_buf = NULL; if ( ! b->yy_ch_buf ) YY_FATAL_ERROR( "fatal error - scanner input buffer overflow" ); (yy_c_buf_p) = &b->yy_ch_buf[yy_c_buf_p_offset]; num_to_read = YY_CURRENT_BUFFER_LVALUE->yy_buf_size - number_to_move - 1; } if ( num_to_read > YY_READ_BUF_SIZE ) num_to_read = YY_READ_BUF_SIZE; /* Read in more data. */ YY_INPUT( (&YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[number_to_move]), (yy_n_chars), num_to_read ); YY_CURRENT_BUFFER_LVALUE->yy_n_chars = (yy_n_chars); } if ( (yy_n_chars) == 0 ) { if ( number_to_move == YY_MORE_ADJ ) { ret_val = EOB_ACT_END_OF_FILE; yyrestart( yyin ); } else { ret_val = EOB_ACT_LAST_MATCH; YY_CURRENT_BUFFER_LVALUE->yy_buffer_status = YY_BUFFER_EOF_PENDING; } } else ret_val = EOB_ACT_CONTINUE_SCAN; if (((yy_n_chars) + number_to_move) > YY_CURRENT_BUFFER_LVALUE->yy_buf_size) { /* Extend the array by 50%, plus the number we really need. */ int new_size = (yy_n_chars) + number_to_move + ((yy_n_chars) >> 1); YY_CURRENT_BUFFER_LVALUE->yy_ch_buf = (char *) yyrealloc( (void *) YY_CURRENT_BUFFER_LVALUE->yy_ch_buf, (yy_size_t) new_size ); if ( ! YY_CURRENT_BUFFER_LVALUE->yy_ch_buf ) YY_FATAL_ERROR( "out of dynamic memory in yy_get_next_buffer()" ); /* "- 2" to take care of EOB's */ YY_CURRENT_BUFFER_LVALUE->yy_buf_size = (int) (new_size - 2); } (yy_n_chars) += number_to_move; YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[(yy_n_chars)] = YY_END_OF_BUFFER_CHAR; YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[(yy_n_chars) + 1] = YY_END_OF_BUFFER_CHAR; (yytext_ptr) = &YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[0]; return ret_val; } /* yy_get_previous_state - get the state just before the EOB char was reached */ static yy_state_type yy_get_previous_state (void) { yy_state_type yy_current_state; char *yy_cp; yy_current_state = (yy_start); for ( yy_cp = (yytext_ptr) + YY_MORE_ADJ; yy_cp < (yy_c_buf_p); ++yy_cp ) { YY_CHAR yy_c = (*yy_cp ? yy_ec[YY_SC_TO_UI(*yy_cp)] : 1); if ( yy_accept[yy_current_state] ) { (yy_last_accepting_state) = yy_current_state; (yy_last_accepting_cpos) = yy_cp; } while ( yy_chk[yy_base[yy_current_state] + yy_c] != yy_current_state ) { yy_current_state = (int) yy_def[yy_current_state]; if ( yy_current_state >= 1103 ) yy_c = yy_meta[yy_c]; } yy_current_state = yy_nxt[yy_base[yy_current_state] + yy_c]; } return yy_current_state; } /* yy_try_NUL_trans - try to make a transition on the NUL character * * synopsis * next_state = yy_try_NUL_trans( current_state ); */ static yy_state_type yy_try_NUL_trans (yy_state_type yy_current_state ) { int yy_is_jam; char *yy_cp = (yy_c_buf_p); YY_CHAR yy_c = 1; if ( yy_accept[yy_current_state] ) { (yy_last_accepting_state) = yy_current_state; (yy_last_accepting_cpos) = yy_cp; } while ( yy_chk[yy_base[yy_current_state] + yy_c] != yy_current_state ) { yy_current_state = (int) yy_def[yy_current_state]; if ( yy_current_state >= 1103 ) yy_c = yy_meta[yy_c]; } yy_current_state = yy_nxt[yy_base[yy_current_state] + yy_c]; yy_is_jam = (yy_current_state == 1102); return yy_is_jam ? 0 : yy_current_state; } #ifndef YY_NO_UNPUT #endif #ifndef YY_NO_INPUT #ifdef __cplusplus static int yyinput (void) #else static int input (void) #endif { int c; *(yy_c_buf_p) = (yy_hold_char); if ( *(yy_c_buf_p) == YY_END_OF_BUFFER_CHAR ) { /* yy_c_buf_p now points to the character we want to return. * If this occurs *before* the EOB characters, then it's a * valid NUL; if not, then we've hit the end of the buffer. */ if ( (yy_c_buf_p) < &YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[(yy_n_chars)] ) /* This was really a NUL. */ *(yy_c_buf_p) = '\0'; else { /* need more input */ int offset = (int) ((yy_c_buf_p) - (yytext_ptr)); ++(yy_c_buf_p); switch ( yy_get_next_buffer( ) ) { case EOB_ACT_LAST_MATCH: /* This happens because yy_g_n_b() * sees that we've accumulated a * token and flags that we need to * try matching the token before * proceeding. But for input(), * there's no matching to consider. * So convert the EOB_ACT_LAST_MATCH * to EOB_ACT_END_OF_FILE. */ /* Reset buffer status. */ yyrestart( yyin ); /*FALLTHROUGH*/ case EOB_ACT_END_OF_FILE: { if ( yywrap( ) ) return 0; if ( ! (yy_did_buffer_switch_on_eof) ) YY_NEW_FILE; #ifdef __cplusplus return yyinput(); #else return input(); #endif } case EOB_ACT_CONTINUE_SCAN: (yy_c_buf_p) = (yytext_ptr) + offset; break; } } } c = *(unsigned char *) (yy_c_buf_p); /* cast for 8-bit char's */ *(yy_c_buf_p) = '\0'; /* preserve yytext */ (yy_hold_char) = *++(yy_c_buf_p); return c; } #endif /* ifndef YY_NO_INPUT */ /** Immediately switch to a different input stream. * @param input_file A readable stream. * * @note This function does not reset the start condition to @c INITIAL . */ void yyrestart (FILE * input_file ) { if ( ! YY_CURRENT_BUFFER ){ yyensure_buffer_stack (); YY_CURRENT_BUFFER_LVALUE = yy_create_buffer( yyin, YY_BUF_SIZE ); } yy_init_buffer( YY_CURRENT_BUFFER, input_file ); yy_load_buffer_state( ); } /** Switch to a different input buffer. * @param new_buffer The new input buffer. * */ void yy_switch_to_buffer (YY_BUFFER_STATE new_buffer ) { /* TODO. We should be able to replace this entire function body * with * yypop_buffer_state(); * yypush_buffer_state(new_buffer); */ yyensure_buffer_stack (); if ( YY_CURRENT_BUFFER == new_buffer ) return; if ( YY_CURRENT_BUFFER ) { /* Flush out information for old buffer. */ *(yy_c_buf_p) = (yy_hold_char); YY_CURRENT_BUFFER_LVALUE->yy_buf_pos = (yy_c_buf_p); YY_CURRENT_BUFFER_LVALUE->yy_n_chars = (yy_n_chars); } YY_CURRENT_BUFFER_LVALUE = new_buffer; yy_load_buffer_state( ); /* We don't actually know whether we did this switch during * EOF (yywrap()) processing, but the only time this flag * is looked at is after yywrap() is called, so it's safe * to go ahead and always set it. */ (yy_did_buffer_switch_on_eof) = 1; } static void yy_load_buffer_state (void) { (yy_n_chars) = YY_CURRENT_BUFFER_LVALUE->yy_n_chars; (yytext_ptr) = (yy_c_buf_p) = YY_CURRENT_BUFFER_LVALUE->yy_buf_pos; yyin = YY_CURRENT_BUFFER_LVALUE->yy_input_file; (yy_hold_char) = *(yy_c_buf_p); } /** Allocate and initialize an input buffer state. * @param file A readable stream. * @param size The character buffer size in bytes. When in doubt, use @c YY_BUF_SIZE. * * @return the allocated buffer state. */ YY_BUFFER_STATE yy_create_buffer (FILE * file, int size ) { YY_BUFFER_STATE b; b = (YY_BUFFER_STATE) yyalloc( sizeof( struct yy_buffer_state ) ); if ( ! b ) YY_FATAL_ERROR( "out of dynamic memory in yy_create_buffer()" ); b->yy_buf_size = size; /* yy_ch_buf has to be 2 characters longer than the size given because * we need to put in 2 end-of-buffer characters. */ b->yy_ch_buf = (char *) yyalloc( (yy_size_t) (b->yy_buf_size + 2) ); if ( ! b->yy_ch_buf ) YY_FATAL_ERROR( "out of dynamic memory in yy_create_buffer()" ); b->yy_is_our_buffer = 1; yy_init_buffer( b, file ); return b; } /** Destroy the buffer. * @param b a buffer created with yy_create_buffer() * */ void yy_delete_buffer (YY_BUFFER_STATE b ) { if ( ! b ) return; if ( b == YY_CURRENT_BUFFER ) /* Not sure if we should pop here. */ YY_CURRENT_BUFFER_LVALUE = (YY_BUFFER_STATE) 0; if ( b->yy_is_our_buffer ) yyfree( (void *) b->yy_ch_buf ); yyfree( (void *) b ); } /* Initializes or reinitializes a buffer. * This function is sometimes called more than once on the same buffer, * such as during a yyrestart() or at EOF. */ static void yy_init_buffer (YY_BUFFER_STATE b, FILE * file ) { int oerrno = errno; yy_flush_buffer( b ); b->yy_input_file = file; b->yy_fill_buffer = 1; /* If b is the current buffer, then yy_init_buffer was _probably_ * called from yyrestart() or through yy_get_next_buffer. * In that case, we don't want to reset the lineno or column. */ if (b != YY_CURRENT_BUFFER){ b->yy_bs_lineno = 1; b->yy_bs_column = 0; } b->yy_is_interactive = 0; errno = oerrno; } /** Discard all buffered characters. On the next scan, YY_INPUT will be called. * @param b the buffer state to be flushed, usually @c YY_CURRENT_BUFFER. * */ void yy_flush_buffer (YY_BUFFER_STATE b ) { if ( ! b ) return; b->yy_n_chars = 0; /* We always need two end-of-buffer characters. The first causes * a transition to the end-of-buffer state. The second causes * a jam in that state. */ b->yy_ch_buf[0] = YY_END_OF_BUFFER_CHAR; b->yy_ch_buf[1] = YY_END_OF_BUFFER_CHAR; b->yy_buf_pos = &b->yy_ch_buf[0]; b->yy_at_bol = 1; b->yy_buffer_status = YY_BUFFER_NEW; if ( b == YY_CURRENT_BUFFER ) yy_load_buffer_state( ); } /** Pushes the new state onto the stack. The new state becomes * the current state. This function will allocate the stack * if necessary. * @param new_buffer The new state. * */ void yypush_buffer_state (YY_BUFFER_STATE new_buffer ) { if (new_buffer == NULL) return; yyensure_buffer_stack(); /* This block is copied from yy_switch_to_buffer. */ if ( YY_CURRENT_BUFFER ) { /* Flush out information for old buffer. */ *(yy_c_buf_p) = (yy_hold_char); YY_CURRENT_BUFFER_LVALUE->yy_buf_pos = (yy_c_buf_p); YY_CURRENT_BUFFER_LVALUE->yy_n_chars = (yy_n_chars); } /* Only push if top exists. Otherwise, replace top. */ if (YY_CURRENT_BUFFER) (yy_buffer_stack_top)++; YY_CURRENT_BUFFER_LVALUE = new_buffer; /* copied from yy_switch_to_buffer. */ yy_load_buffer_state( ); (yy_did_buffer_switch_on_eof) = 1; } /** Removes and deletes the top of the stack, if present. * The next element becomes the new top. * */ void yypop_buffer_state (void) { if (!YY_CURRENT_BUFFER) return; yy_delete_buffer(YY_CURRENT_BUFFER ); YY_CURRENT_BUFFER_LVALUE = NULL; if ((yy_buffer_stack_top) > 0) --(yy_buffer_stack_top); if (YY_CURRENT_BUFFER) { yy_load_buffer_state( ); (yy_did_buffer_switch_on_eof) = 1; } } /* Allocates the stack if it does not exist. * Guarantees space for at least one push. */ static void yyensure_buffer_stack (void) { yy_size_t num_to_alloc; if (!(yy_buffer_stack)) { /* First allocation is just for 2 elements, since we don't know if this * scanner will even need a stack. We use 2 instead of 1 to avoid an * immediate realloc on the next call. */ num_to_alloc = 1; /* After all that talk, this was set to 1 anyways... */ (yy_buffer_stack) = (struct yy_buffer_state**)yyalloc (num_to_alloc * sizeof(struct yy_buffer_state*) ); if ( ! (yy_buffer_stack) ) YY_FATAL_ERROR( "out of dynamic memory in yyensure_buffer_stack()" ); memset((yy_buffer_stack), 0, num_to_alloc * sizeof(struct yy_buffer_state*)); (yy_buffer_stack_max) = num_to_alloc; (yy_buffer_stack_top) = 0; return; } if ((yy_buffer_stack_top) >= ((yy_buffer_stack_max)) - 1){ /* Increase the buffer to prepare for a possible push. */ yy_size_t grow_size = 8 /* arbitrary grow size */; num_to_alloc = (yy_buffer_stack_max) + grow_size; (yy_buffer_stack) = (struct yy_buffer_state**)yyrealloc ((yy_buffer_stack), num_to_alloc * sizeof(struct yy_buffer_state*) ); if ( ! (yy_buffer_stack) ) YY_FATAL_ERROR( "out of dynamic memory in yyensure_buffer_stack()" ); /* zero only the new slots.*/ memset((yy_buffer_stack) + (yy_buffer_stack_max), 0, grow_size * sizeof(struct yy_buffer_state*)); (yy_buffer_stack_max) = num_to_alloc; } } /** Setup the input buffer state to scan directly from a user-specified character buffer. * @param base the character buffer * @param size the size in bytes of the character buffer * * @return the newly allocated buffer state object. */ YY_BUFFER_STATE yy_scan_buffer (char * base, yy_size_t size ) { YY_BUFFER_STATE b; if ( size < 2 || base[size-2] != YY_END_OF_BUFFER_CHAR || base[size-1] != YY_END_OF_BUFFER_CHAR ) /* They forgot to leave room for the EOB's. */ return NULL; b = (YY_BUFFER_STATE) yyalloc( sizeof( struct yy_buffer_state ) ); if ( ! b ) YY_FATAL_ERROR( "out of dynamic memory in yy_scan_buffer()" ); b->yy_buf_size = (int) (size - 2); /* "- 2" to take care of EOB's */ b->yy_buf_pos = b->yy_ch_buf = base; b->yy_is_our_buffer = 0; b->yy_input_file = NULL; b->yy_n_chars = b->yy_buf_size; b->yy_is_interactive = 0; b->yy_at_bol = 1; b->yy_fill_buffer = 0; b->yy_buffer_status = YY_BUFFER_NEW; yy_switch_to_buffer( b ); return b; } /** Setup the input buffer state to scan a string. The next call to yylex() will * scan from a @e copy of @a str. * @param yystr a NUL-terminated string to scan * * @return the newly allocated buffer state object. * @note If you want to scan bytes that may contain NUL values, then use * yy_scan_bytes() instead. */ YY_BUFFER_STATE yy_scan_string (const char * yystr ) { return yy_scan_bytes( yystr, (int) strlen(yystr) ); } /** Setup the input buffer state to scan the given bytes. The next call to yylex() will * scan from a @e copy of @a bytes. * @param yybytes the byte buffer to scan * @param _yybytes_len the number of bytes in the buffer pointed to by @a bytes. * * @return the newly allocated buffer state object. */ YY_BUFFER_STATE yy_scan_bytes (const char * yybytes, int _yybytes_len ) { YY_BUFFER_STATE b; char *buf; yy_size_t n; int i; /* Get memory for full buffer, including space for trailing EOB's. */ n = (yy_size_t) (_yybytes_len + 2); buf = (char *) yyalloc( n ); if ( ! buf ) YY_FATAL_ERROR( "out of dynamic memory in yy_scan_bytes()" ); for ( i = 0; i < _yybytes_len; ++i ) buf[i] = yybytes[i]; buf[_yybytes_len] = buf[_yybytes_len+1] = YY_END_OF_BUFFER_CHAR; b = yy_scan_buffer( buf, n ); if ( ! b ) YY_FATAL_ERROR( "bad buffer in yy_scan_bytes()" ); /* It's okay to grow etc. this buffer, and we should throw it * away when we're done. */ b->yy_is_our_buffer = 1; return b; } #ifndef YY_EXIT_FAILURE #define YY_EXIT_FAILURE 2 #endif static void yynoreturn yy_fatal_error (const char* msg ) { fprintf( stderr, "%s\n", msg ); exit( YY_EXIT_FAILURE ); } /* Redefine yyless() so it works in section 3 code. */ #undef yyless #define yyless(n) \ do \ { \ /* Undo effects of setting up yytext. */ \ int yyless_macro_arg = (n); \ YY_LESS_LINENO(yyless_macro_arg);\ yytext[yyleng] = (yy_hold_char); \ (yy_c_buf_p) = yytext + yyless_macro_arg; \ (yy_hold_char) = *(yy_c_buf_p); \ *(yy_c_buf_p) = '\0'; \ yyleng = yyless_macro_arg; \ } \ while ( 0 ) /* Accessor methods (get/set functions) to struct members. */ /** Get the current line number. * */ int yyget_lineno (void) { return yylineno; } /** Get the input stream. * */ FILE *yyget_in (void) { return yyin; } /** Get the output stream. * */ FILE *yyget_out (void) { return yyout; } /** Get the length of the current token. * */ int yyget_leng (void) { return yyleng; } /** Get the current token. * */ char *yyget_text (void) { return yytext; } /** Set the current line number. * @param _line_number line number * */ void yyset_lineno (int _line_number ) { yylineno = _line_number; } /** Set the input stream. This does not discard the current * input buffer. * @param _in_str A readable stream. * * @see yy_switch_to_buffer */ void yyset_in (FILE * _in_str ) { yyin = _in_str ; } void yyset_out (FILE * _out_str ) { yyout = _out_str ; } int yyget_debug (void) { return yy_flex_debug; } void yyset_debug (int _bdebug ) { yy_flex_debug = _bdebug ; } static int yy_init_globals (void) { /* Initialization is the same as for the non-reentrant scanner. * This function is called from yylex_destroy(), so don't allocate here. */ (yy_buffer_stack) = NULL; (yy_buffer_stack_top) = 0; (yy_buffer_stack_max) = 0; (yy_c_buf_p) = NULL; (yy_init) = 0; (yy_start) = 0; /* Defined in main.c */ #ifdef YY_STDINIT yyin = stdin; yyout = stdout; #else yyin = NULL; yyout = NULL; #endif /* For future reference: Set errno on error, since we are called by * yylex_init() */ return 0; } /* yylex_destroy is for both reentrant and non-reentrant scanners. */ int yylex_destroy (void) { /* Pop the buffer stack, destroying each element. */ while(YY_CURRENT_BUFFER){ yy_delete_buffer( YY_CURRENT_BUFFER ); YY_CURRENT_BUFFER_LVALUE = NULL; yypop_buffer_state(); } /* Destroy the stack itself. */ yyfree((yy_buffer_stack) ); (yy_buffer_stack) = NULL; /* Reset the globals. This is important in a non-reentrant scanner so the next time * yylex() is called, initialization will occur. */ yy_init_globals( ); return 0; } /* * Internal utility routines. */ #ifndef yytext_ptr static void yy_flex_strncpy (char* s1, const char * s2, int n ) { int i; for ( i = 0; i < n; ++i ) s1[i] = s2[i]; } #endif #ifdef YY_NEED_STRLEN static int yy_flex_strlen (const char * s ) { int n; for ( n = 0; s[n]; ++n ) ; return n; } #endif void *yyalloc (yy_size_t size ) { return malloc(size); } void *yyrealloc (void * ptr, yy_size_t size ) { /* The cast to (char *) in the following accommodates both * implementations that use char* generic pointers, and those * that use void* generic pointers. It works with the latter * because both ANSI C and C++ allow castless assignment from * any pointer type to void*, and deal with argument conversions * as though doing an assignment. */ return realloc(ptr, size); } void yyfree (void * ptr ) { free( (char *) ptr ); /* see yyrealloc() for (char *) cast */ } #define YYTABLES_NAME "yytables" #line 167 "css.l" wget-1.21.2/src/build_info.c0000644000000000000000000000250114115733377012510 00000000000000/* Autogenerated by build_info.pl - DO NOT EDIT */ /* This stores global variables that are initialized with preprocessor declarations for output with the --version flag. Copyright (C) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009 Free Software Foundation, Inc. */ #include "wget.h" #include #include "version.h" const char *compiled_features[] = { #if defined HAVE_LIBCARES "+cares", #else "-cares", #endif #if defined ENABLE_DIGEST "+digest", #else "-digest", #endif #if defined HAVE_GPGME "+gpgme", #else "-gpgme", #endif #if defined HAVE_SSL "+https", #else "-https", #endif #if defined ENABLE_IPV6 "+ipv6", #else "-ipv6", #endif #if defined ENABLE_IRI "+iri", #else "-iri", #endif #if SIZEOF_OFF_T >= 8 || defined WINDOWS "+large-file", #else "-large-file", #endif #if defined HAVE_METALINK "+metalink", #else "-metalink", #endif #if defined ENABLE_NLS "+nls", #else "-nls", #endif #if defined ENABLE_NTLM "+ntlm", #else "-ntlm", #endif #if defined ENABLE_OPIE "+opie", #else "-opie", #endif #if defined HAVE_LIBPSL "+psl", #else "-psl", #endif #if defined HAVE_LIBSSL || defined HAVE_LIBSSL32 "+ssl/openssl", #elif defined HAVE_LIBGNUTLS "+ssl/gnutls", #else "-ssl", #endif /* sentinel value */ NULL }; wget-1.21.2/src/ftp.c0000644000000000000000000027315514115732710011174 00000000000000/* File Transfer Protocol support. Copyright (C) 1996-2011, 2014-2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #include "wget.h" #include #include #include #include #include #include #include #include #include "utils.h" #include "url.h" #include "retr.h" #include "ftp.h" #include "ssl.h" #include "connect.h" #include "host.h" #include "netrc.h" #include "convert.h" /* for downloaded_file */ #include "recur.h" /* for INFINITE_RECURSION */ #include "warc.h" #include "c-strcase.h" #ifdef ENABLE_XATTR #include "xattr.h" #endif #ifdef __VMS # include "vms.h" #endif /* def __VMS */ /* File where the "ls -al" listing will be saved. */ #ifdef MSDOS #define LIST_FILENAME "_listing" #else #define LIST_FILENAME ".listing" #endif typedef struct { int st; /* connection status */ int cmd; /* command code */ int csock; /* control connection socket */ double dltime; /* time of the download in msecs */ enum stype rs; /* remote system reported by ftp server */ enum ustype rsu; /* when rs is ST_UNIX, here there are more details */ char *id; /* initial directory */ char *target; /* target file name */ struct url *proxy; /* FTWK-style proxy */ } ccon; /* Look for regexp "( *[0-9]+ *byte" (literal parenthesis) anywhere in the string S, and return the number converted to wgint, if found, 0 otherwise. */ static wgint ftp_expected_bytes (const char *s) { wgint res; while (1) { while (*s && *s != '(') ++s; if (!*s) return 0; ++s; /* skip the '(' */ res = str_to_wgint (s, (char **) &s, 10); if (!*s) return 0; while (*s && c_isspace (*s)) ++s; if (!*s) return 0; if (c_tolower (*s) != 'b') continue; if (c_strncasecmp (s, "byte", 4)) continue; else break; } return res; } #ifdef ENABLE_IPV6 /* * This function sets up a passive data connection with the FTP server. * It is merely a wrapper around ftp_epsv, ftp_lpsv and ftp_pasv. */ static uerr_t ftp_do_pasv (int csock, ip_address *addr, int *port) { uerr_t err; /* We need to determine the address family and need to call getpeername, so while we're at it, store the address to ADDR. ftp_pasv and ftp_lpsv can simply override it. */ if (!socket_ip_address (csock, addr, ENDPOINT_PEER)) abort (); /* If our control connection is over IPv6, then we first try EPSV and then * LPSV if the former is not supported. If the control connection is over * IPv4, we simply issue the good old PASV request. */ switch (addr->family) { case AF_INET: if (!opt.server_response) logputs (LOG_VERBOSE, "==> PASV ... "); err = ftp_pasv (csock, addr, port); break; case AF_INET6: if (!opt.server_response) logputs (LOG_VERBOSE, "==> EPSV ... "); err = ftp_epsv (csock, addr, port); /* If EPSV is not supported try LPSV */ if (err == FTPNOPASV) { if (!opt.server_response) logputs (LOG_VERBOSE, "==> LPSV ... "); err = ftp_lpsv (csock, addr, port); } break; default: abort (); } return err; } /* * This function sets up an active data connection with the FTP server. * It is merely a wrapper around ftp_eprt, ftp_lprt and ftp_port. */ static uerr_t ftp_do_port (int csock, int *local_sock) { uerr_t err; ip_address cip; if (!socket_ip_address (csock, &cip, ENDPOINT_PEER)) abort (); /* If our control connection is over IPv6, then we first try EPRT and then * LPRT if the former is not supported. If the control connection is over * IPv4, we simply issue the good old PORT request. */ switch (cip.family) { case AF_INET: if (!opt.server_response) logputs (LOG_VERBOSE, "==> PORT ... "); err = ftp_port (csock, local_sock); break; case AF_INET6: if (!opt.server_response) logputs (LOG_VERBOSE, "==> EPRT ... "); err = ftp_eprt (csock, local_sock); /* If EPRT is not supported try LPRT */ if (err == FTPPORTERR) { if (!opt.server_response) logputs (LOG_VERBOSE, "==> LPRT ... "); err = ftp_lprt (csock, local_sock); } break; default: abort (); } return err; } #else static uerr_t ftp_do_pasv (int csock, ip_address *addr, int *port) { if (!opt.server_response) logputs (LOG_VERBOSE, "==> PASV ... "); return ftp_pasv (csock, addr, port); } static uerr_t ftp_do_port (int csock, int *local_sock) { if (!opt.server_response) logputs (LOG_VERBOSE, "==> PORT ... "); return ftp_port (csock, local_sock); } #endif static void print_length (wgint size, wgint start, bool authoritative) { logprintf (LOG_VERBOSE, _("Length: %s"), number_to_static_string (size)); if (size >= 1024) logprintf (LOG_VERBOSE, " (%s)", human_readable (size, 10, 1)); if (start > 0) { if (size - start >= 1024) logprintf (LOG_VERBOSE, _(", %s (%s) remaining"), number_to_static_string (size - start), human_readable (size - start, 10, 1)); else logprintf (LOG_VERBOSE, _(", %s remaining"), number_to_static_string (size - start)); } logputs (LOG_VERBOSE, !authoritative ? _(" (unauthoritative)\n") : "\n"); } static uerr_t ftp_get_listing (struct url *, struct url *, ccon *, struct fileinfo **); static uerr_t get_ftp_greeting (int csock, ccon *con) { uerr_t err = 0; /* Get the server's greeting */ err = ftp_greeting (csock); if (err != FTPOK) { logputs (LOG_NOTQUIET, "Error in server response. Closing.\n"); fd_close (csock); con->csock = -1; } return err; } #ifdef HAVE_SSL static uerr_t init_control_ssl_connection (int csock, struct url *u, bool *using_control_security) { bool using_security = false; /* If '--ftps-implicit' was passed, perform the SSL handshake directly, * and do not send an AUTH command. * Otherwise send an AUTH sequence before login, * and perform the SSL handshake if accepted by server. */ if (!opt.ftps_implicit && !opt.server_response) logputs (LOG_VERBOSE, "==> AUTH TLS ... "); if (opt.ftps_implicit || ftp_auth (csock, SCHEME_FTPS) == FTPOK) { if (!ssl_connect_wget (csock, u->host, NULL)) { fd_close (csock); return CONSSLERR; } else if (!ssl_check_certificate (csock, u->host)) { fd_close (csock); return VERIFCERTERR; } if (!opt.ftps_implicit && !opt.server_response) logputs (LOG_VERBOSE, " done.\n"); /* If implicit FTPS was requested, we act as "normal" FTP, but over SSL. * We're not using RFC 2228 commands. */ using_security = true; } else { /* The server does not support 'AUTH TLS'. * Check if --ftps-fallback-to-ftp was passed. */ if (opt.ftps_fallback_to_ftp) { logputs (LOG_NOTQUIET, "Server does not support AUTH TLS. Falling back to FTP.\n"); using_security = false; } else { fd_close (csock); return FTPNOAUTH; } } *using_control_security = using_security; return NOCONERROR; } #endif /* Retrieves a file with denoted parameters through opening an FTP connection to the server. It always closes the data connection, and closes the control connection in case of error. If warc_tmp is non-NULL, the downloaded data will be written there as well. */ static uerr_t getftp (struct url *u, struct url *original_url, wgint passed_expected_bytes, wgint *qtyread, wgint restval, ccon *con, int count, wgint *last_expected_bytes, FILE *warc_tmp) { int csock, dtsock, local_sock, res; uerr_t err = RETROK; /* appease the compiler */ FILE *fp = NULL; char *respline, *tms; const char *user, *passwd, *tmrate; int cmd = con->cmd; wgint expected_bytes = 0; bool got_expected_bytes = false; bool rest_failed = false; int flags; wgint rd_size, previous_rd_size = 0; char type_char; bool try_again; bool list_a_used = false; #ifdef HAVE_SSL enum prot_level prot = (opt.ftps_clear_data_connection ? PROT_CLEAR : PROT_PRIVATE); /* these variables tell whether the target server * accepts the security extensions (RFC 2228) or not, * and whether we're actually using any of them * (encryption at the control connection only, * or both at control and data connections) */ bool using_control_security = false, using_data_security = false; #endif assert (con != NULL); assert (con->target != NULL); /* Debug-check of the sanity of the request by making sure that LIST and RETR are never both requested (since we can handle only one at a time. */ assert (!((cmd & DO_LIST) && (cmd & DO_RETR))); /* Make sure that at least *something* is requested. */ assert ((cmd & (DO_LIST | DO_CWD | DO_RETR | DO_LOGIN)) != 0); *qtyread = restval; /* Find the username with priority */ if (u->user) user = u->user; else if (opt.user && (opt.use_askpass || opt.ask_passwd)) user = opt.user; else if (opt.ftp_user) user = opt.ftp_user; else if (opt.user) user = opt.user; else user = NULL; /* Find the password with priority */ if (u->passwd) passwd = u->passwd; else if (opt.passwd && (opt.use_askpass || opt.ask_passwd)) passwd = opt.passwd; else if (opt.ftp_passwd) passwd = opt.ftp_passwd; else if (opt.passwd) passwd = opt.passwd; else passwd = NULL; /* Check for ~/.netrc if none of the above match */ if (opt.netrc && (!user || !passwd)) search_netrc (u->host, (const char **) &user, (const char **) &passwd, 1, NULL); if (!user) user = "anonymous"; if (!passwd) passwd = "-wget@"; dtsock = -1; local_sock = -1; con->dltime = 0; #ifdef HAVE_SSL if (u->scheme == SCHEME_FTPS) { /* Initialize SSL layer first */ if (!ssl_init ()) { scheme_disable (SCHEME_FTPS); logprintf (LOG_NOTQUIET, _("Could not initialize SSL. It will be disabled.\n")); err = SSLINITFAILED; return err; } /* If we're using the default FTP port and implicit FTPS was requested, * rewrite the port to the default *implicit* FTPS port. */ if (opt.ftps_implicit && u->port == DEFAULT_FTP_PORT) { DEBUGP (("Implicit FTPS was specified. Rewriting default port to %d.\n", DEFAULT_FTPS_IMPLICIT_PORT)); u->port = DEFAULT_FTPS_IMPLICIT_PORT; } } #endif if (!(cmd & DO_LOGIN)) { csock = con->csock; #ifdef HAVE_SSL using_data_security = con->st & DATA_CHANNEL_SECURITY; #endif } else /* cmd & DO_LOGIN */ { char *host = con->proxy ? con->proxy->host : u->host; int port = con->proxy ? con->proxy->port : u->port; /* Login to the server: */ /* First: Establish the control connection. */ csock = connect_to_host (host, port); if (csock == E_HOST) return HOSTERR; else if (csock < 0) return (retryable_socket_connect_error (errno) ? CONERROR : CONIMPOSSIBLE); if (cmd & LEAVE_PENDING) con->csock = csock; else con->csock = -1; #ifdef HAVE_SSL if (u->scheme == SCHEME_FTPS) { /* If we're in implicit FTPS mode, we have to set up SSL/TLS before everything else. * Otherwise we first read the server's greeting, and then send an "AUTH TLS". */ if (opt.ftps_implicit) { err = init_control_ssl_connection (csock, u, &using_control_security); if (err != NOCONERROR) return err; err = get_ftp_greeting (csock, con); if (err != FTPOK) return err; } else { err = get_ftp_greeting (csock, con); if (err != FTPOK) return err; err = init_control_ssl_connection (csock, u, &using_control_security); if (err != NOCONERROR) return err; } } else { err = get_ftp_greeting (csock, con); if (err != FTPOK) return err; } #else err = get_ftp_greeting (csock, con); if (err != FTPOK) return err; #endif /* Second: Login with proper USER/PASS sequence. */ logprintf (LOG_VERBOSE, _("Logging in as %s ... "), quotearg_style (escape_quoting_style, user)); if (opt.server_response) logputs (LOG_ALWAYS, "\n"); if (con->proxy) { /* If proxy is in use, log in as username@target-site. */ char *logname = concat_strings (user, "@", u->host, (char *) 0); err = ftp_login (csock, logname, passwd); xfree (logname); } else err = ftp_login (csock, user, passwd); /* FTPRERR, FTPSRVERR, WRITEFAILED, FTPLOGREFUSED, FTPLOGINC */ switch (err) { case FTPRERR: logputs (LOG_VERBOSE, "\n"); logputs (LOG_NOTQUIET, _("\ Error in server response, closing control connection.\n")); fd_close (csock); con->csock = -1; return err; case FTPSRVERR: logputs (LOG_VERBOSE, "\n"); logputs (LOG_NOTQUIET, _("Error in server greeting.\n")); fd_close (csock); con->csock = -1; return err; case WRITEFAILED: logputs (LOG_VERBOSE, "\n"); logputs (LOG_NOTQUIET, _("Write failed, closing control connection.\n")); fd_close (csock); con->csock = -1; return err; case FTPLOGREFUSED: logputs (LOG_VERBOSE, "\n"); logputs (LOG_NOTQUIET, _("The server refuses login.\n")); fd_close (csock); con->csock = -1; return FTPLOGREFUSED; case FTPLOGINC: logputs (LOG_VERBOSE, "\n"); logputs (LOG_NOTQUIET, _("Login incorrect.\n")); fd_close (csock); con->csock = -1; return FTPLOGINC; case FTPOK: if (!opt.server_response) logputs (LOG_VERBOSE, _("Logged in!\n")); break; default: abort (); } #ifdef HAVE_SSL if (using_control_security) { /* Send the PBSZ and PROT commands, in that order. * If we are here it means that the server has already accepted * some form of FTPS. Thus, these commands must work. * If they don't work, that's an error. There's no sense in honoring * --ftps-fallback-to-ftp or similar options. */ if (u->scheme == SCHEME_FTPS) { if (!opt.server_response) logputs (LOG_VERBOSE, "==> PBSZ 0 ... "); if ((err = ftp_pbsz (csock, 0)) == FTPNOPBSZ) { logputs (LOG_NOTQUIET, _("Server did not accept the 'PBSZ 0' command.\n")); return err; } if (!opt.server_response) logputs (LOG_VERBOSE, "done."); if (!opt.server_response) logprintf (LOG_VERBOSE, " ==> PROT %c ... ", (int) prot); if ((err = ftp_prot (csock, prot)) == FTPNOPROT) { logprintf (LOG_NOTQUIET, _("Server did not accept the 'PROT %c' command.\n"), (int) prot); return err; } if (!opt.server_response) logputs (LOG_VERBOSE, "done.\n"); if (prot != PROT_CLEAR) { using_data_security = true; con->st |= DATA_CHANNEL_SECURITY; } } } #endif /* Third: Get the system type */ if (!opt.server_response) logprintf (LOG_VERBOSE, "==> SYST ... "); err = ftp_syst (csock, &con->rs, &con->rsu); /* FTPRERR */ switch (err) { case FTPRERR: logputs (LOG_VERBOSE, "\n"); logputs (LOG_NOTQUIET, _("\ Error in server response, closing control connection.\n")); fd_close (csock); con->csock = -1; return err; case FTPSRVERR: logputs (LOG_VERBOSE, "\n"); logputs (LOG_NOTQUIET, _("Server error, can't determine system type.\n")); break; case FTPOK: /* Everything is OK. */ break; default: abort (); } if (!opt.server_response && err != FTPSRVERR) logputs (LOG_VERBOSE, _("done. ")); /* 2013-10-17 Andrea Urbani (matfanjol) According to the system type I choose which list command will be used. If I don't know that system, I will try, the first time of each session, "LIST -a" and "LIST". (see __LIST_A_EXPLANATION__ below) */ switch (con->rs) { case ST_VMS: /* About ST_VMS there is an old note: 2008-01-29 SMS. For a VMS FTP server, where "LIST -a" may not fail, but will never do what is desired here, skip directly to the simple "LIST" command (assumed to be the last one in the list). */ DEBUGP (("\nVMS: I know it and I will use \"LIST\" as standard list command\n")); con->st |= LIST_AFTER_LIST_A_CHECK_DONE; con->st |= AVOID_LIST_A; break; case ST_UNIX: if (con->rsu == UST_MULTINET) { DEBUGP (("\nUNIX MultiNet: I know it and I will use \"LIST\" " "as standard list command\n")); con->st |= LIST_AFTER_LIST_A_CHECK_DONE; con->st |= AVOID_LIST_A; } else if (con->rsu == UST_TYPE_L8) { DEBUGP (("\nUNIX TYPE L8: I know it and I will use \"LIST -a\" " "as standard list command\n")); con->st |= LIST_AFTER_LIST_A_CHECK_DONE; con->st |= AVOID_LIST; } break; default: break; } /* Fourth: Find the initial ftp directory */ if (!opt.server_response) logprintf (LOG_VERBOSE, "==> PWD ... "); err = ftp_pwd (csock, &con->id); /* FTPRERR */ switch (err) { case FTPRERR: logputs (LOG_VERBOSE, "\n"); logputs (LOG_NOTQUIET, _("\ Error in server response, closing control connection.\n")); fd_close (csock); con->csock = -1; return err; case FTPSRVERR : /* PWD unsupported -- assume "/". */ xfree (con->id); con->id = xstrdup ("/"); break; case FTPOK: /* Everything is OK. */ break; default: abort (); } #if 0 /* 2004-09-17 SMS. Don't help me out. Please. A reasonably recent VMS FTP server will cope just fine with UNIX file specifications. This code just spoils things. Discarding the device name, for example, is not a wise move. This code was disabled but left in as an example of what not to do. */ /* VMS will report something like "PUB$DEVICE:[INITIAL.FOLDER]". Convert it to "/INITIAL/FOLDER" */ if (con->rs == ST_VMS) { char *path = strchr (con->id, '['); char *pathend = path ? strchr (path + 1, ']') : NULL; if (!path || !pathend) DEBUGP (("Initial VMS directory not in the form [...]!\n")); else { char *idir = con->id; DEBUGP (("Preprocessing the initial VMS directory\n")); DEBUGP ((" old = '%s'\n", con->id)); /* We do the conversion in-place by copying the stuff between [ and ] to the beginning, and changing dots to slashes at the same time. */ *idir++ = '/'; for (++path; path < pathend; path++, idir++) *idir = *path == '.' ? '/' : *path; *idir = '\0'; DEBUGP ((" new = '%s'\n\n", con->id)); } } #endif /* 0 */ if (!opt.server_response) logputs (LOG_VERBOSE, _("done.\n")); /* Fifth: Set the FTP type. */ type_char = ftp_process_type (u->params); if (!opt.server_response) logprintf (LOG_VERBOSE, "==> TYPE %c ... ", type_char); err = ftp_type (csock, type_char); /* FTPRERR, WRITEFAILED, FTPUNKNOWNTYPE */ switch (err) { case FTPRERR: logputs (LOG_VERBOSE, "\n"); logputs (LOG_NOTQUIET, _("\ Error in server response, closing control connection.\n")); fd_close (csock); con->csock = -1; return err; case WRITEFAILED: logputs (LOG_VERBOSE, "\n"); logputs (LOG_NOTQUIET, _("Write failed, closing control connection.\n")); fd_close (csock); con->csock = -1; return err; case FTPUNKNOWNTYPE: logputs (LOG_VERBOSE, "\n"); logprintf (LOG_NOTQUIET, _("Unknown type `%c', closing control connection.\n"), type_char); fd_close (csock); con->csock = -1; return err; case FTPOK: /* Everything is OK. */ break; default: abort (); } if (!opt.server_response) logputs (LOG_VERBOSE, _("done. ")); } /* do login */ if (cmd & DO_CWD) { if (!*u->dir) logputs (LOG_VERBOSE, _("==> CWD not needed.\n")); else { const char *targ = NULL; char *target = u->dir; char targetbuf[1024]; int cwd_count; int cwd_end; int cwd_start; DEBUGP (("changing working directory\n")); /* Change working directory. To change to a non-absolute Unix directory, we need to prepend initial directory (con->id) to it. Absolute directories "just work". A relative directory is one that does not begin with '/' and, on non-Unix OS'es, one that doesn't begin with "[a-z]:". This is not done for OS400, which doesn't use "/"-delimited directories, nor does it support directory hierarchies. "CWD foo" followed by "CWD bar" leaves us in "bar", not in "foo/bar", as would be customary elsewhere. */ /* 2004-09-20 SMS. Why is this wise even on UNIX? It certainly fouls VMS. See below for a more reliable, more universal method. */ /* 2008-04-22 MJC. I'm not crazy about it either. I'm informed it's useful for misconfigured servers that have some dirs in the path with +x but -r, but this method is not RFC-conformant. I understand the need to deal with crappy server configurations, but it's far better to use the canonical method first, and fall back to kludges second. */ if (target[0] != '/' && !(con->rs != ST_UNIX && c_isalpha (target[0]) && target[1] == ':') && (con->rs != ST_OS400) && (con->rs != ST_VMS)) { char *ntarget, *p; size_t idlen = strlen (con->id); size_t len; /* Strip trailing slash(es) from con->id. */ while (idlen > 0 && con->id[idlen - 1] == '/') --idlen; len = idlen + 1 + strlen (target); if (len < sizeof (targetbuf)) p = ntarget = targetbuf; else p = ntarget = xmalloc (len + 1); memcpy (p, con->id, idlen); p += idlen; *p++ = '/'; strcpy (p, target); DEBUGP (("Prepended initial PWD to relative path:\n")); DEBUGP ((" pwd: '%s'\n old: '%s'\n new: '%s'\n", con->id, target, ntarget)); target = ntarget; } #if 0 /* 2004-09-17 SMS. Don't help me out. Please. A reasonably recent VMS FTP server will cope just fine with UNIX file specifications. This code just spoils things. Discarding the device name, for example, is not a wise move. This code was disabled but left in as an example of what not to do. */ /* If the FTP host runs VMS, we will have to convert the absolute directory path in UNIX notation to absolute directory path in VMS notation as VMS FTP servers do not like UNIX notation of absolute paths. "VMS notation" is [dir.subdir.subsubdir]. */ if (con->rs == ST_VMS) { char *tmpp; char *ntarget = (char *)alloca (strlen (target) + 2); /* We use a converted initial dir, so directories in TARGET will be separated with slashes, something like "/INITIAL/FOLDER/DIR/SUBDIR". Convert that to "[INITIAL.FOLDER.DIR.SUBDIR]". */ strcpy (ntarget, target); assert (*ntarget == '/'); *ntarget = '['; for (tmpp = ntarget + 1; *tmpp; tmpp++) if (*tmpp == '/') *tmpp = '.'; *tmpp++ = ']'; *tmpp = '\0'; DEBUGP (("Changed file name to VMS syntax:\n")); DEBUGP ((" Unix: '%s'\n VMS: '%s'\n", target, ntarget)); target = ntarget; } #endif /* 0 */ /* 2004-09-20 SMS. A relative directory is relative to the initial directory. Thus, what _is_ useful on VMS (and probably elsewhere) is to CWD to the initial directory (ideally, whatever the server reports, _exactly_, NOT badly UNIX-ixed), and then CWD to the (new) relative directory. This should probably be restructured as a function, called once or twice, but I'm lazy enough to take the badly indented loop short-cut for now. */ /* Decide on one pass (absolute) or two (relative). The VMS restriction may be relaxed when the squirrely code above is reformed. */ if ((con->rs == ST_VMS) && (target[0] != '/')) { cwd_start = 0; DEBUGP (("Using two-step CWD for relative path.\n")); } else { /* Go straight to the target. */ cwd_start = 1; } /* At least one VMS FTP server (TCPware V5.6-2) can switch to a UNIX emulation mode when given a UNIX-like directory specification (like "a/b/c"). If allowed to continue this way, LIST interpretation will be confused, because the system type (SYST response) will not be re-checked, and future UNIX-format directory listings (for multiple URLs or "-r") will be horribly misinterpreted. The cheap and nasty work-around is to do a "CWD []" after a UNIX-like directory specification is used. (A single-level directory is harmless.) This puts the TCPware server back into VMS mode, and does no harm on other servers. Unlike the rest of this block, this particular behavior _is_ VMS-specific, so it gets its own VMS test. */ if ((con->rs == ST_VMS) && (strchr (target, '/') != NULL)) { cwd_end = 3; DEBUGP (("Using extra \"CWD []\" step for VMS server.\n")); } else { cwd_end = 2; } /* 2004-09-20 SMS. */ /* Sorry about the deviant indenting. Laziness. */ for (cwd_count = cwd_start; cwd_count < cwd_end; cwd_count++) { switch (cwd_count) { case 0: /* Step one (optional): Go to the initial directory, exactly as reported by the server. */ targ = con->id; break; case 1: /* Step two: Go to the target directory. (Absolute or relative will work now.) */ targ = target; break; case 2: /* Step three (optional): "CWD []" to restore server VMS-ness. */ targ = "[]"; break; default: logprintf (LOG_ALWAYS, _("Logically impossible section reached in getftp()")); logprintf (LOG_ALWAYS, _("cwd_count: %d\ncwd_start: %d\ncwd_end: %d\n"), cwd_count, cwd_start, cwd_end); abort (); } if (!opt.server_response) logprintf (LOG_VERBOSE, "==> CWD (%d) %s ... ", cwd_count, quotearg_style (escape_quoting_style, target)); err = ftp_cwd (csock, targ); /* FTPRERR, WRITEFAILED, FTPNSFOD */ switch (err) { case FTPRERR: logputs (LOG_VERBOSE, "\n"); logputs (LOG_NOTQUIET, _("\ Error in server response, closing control connection.\n")); fd_close (csock); con->csock = -1; return err; case WRITEFAILED: logputs (LOG_VERBOSE, "\n"); logputs (LOG_NOTQUIET, _("Write failed, closing control connection.\n")); fd_close (csock); con->csock = -1; return err; case FTPNSFOD: logputs (LOG_VERBOSE, "\n"); logprintf (LOG_NOTQUIET, _("No such directory %s.\n\n"), quote (u->dir)); fd_close (csock); con->csock = -1; return err; case FTPOK: break; default: abort (); } if (!opt.server_response) logputs (LOG_VERBOSE, _("done.\n")); } /* for */ /* 2004-09-20 SMS. */ } /* else */ } else /* do not CWD */ logputs (LOG_VERBOSE, _("==> CWD not required.\n")); if ((cmd & DO_RETR) && passed_expected_bytes == 0) { if (opt.verbose) { if (!opt.server_response) logprintf (LOG_VERBOSE, "==> SIZE %s ... ", quotearg_style (escape_quoting_style, u->file)); } err = ftp_size (csock, u->file, &expected_bytes); /* FTPRERR */ switch (err) { case FTPRERR: case FTPSRVERR: logputs (LOG_VERBOSE, "\n"); logputs (LOG_NOTQUIET, _("\ Error in server response, closing control connection.\n")); fd_close (csock); con->csock = -1; return err; case FTPOK: got_expected_bytes = true; /* Everything is OK. */ break; default: abort (); } if (!opt.server_response) { logprintf (LOG_VERBOSE, "%s\n", expected_bytes ? number_to_static_string (expected_bytes) : _("done.\n")); } } if (cmd & DO_RETR && restval > 0 && restval == expected_bytes) { /* Server confirms that file has length restval. We should stop now. Some servers (f.e. NcFTPd) return error when receive REST 0 */ logputs (LOG_VERBOSE, _("File has already been retrieved.\n")); fd_close (csock); con->csock = -1; return RETRFINISHED; } do { try_again = false; /* If anything is to be retrieved, PORT (or PASV) must be sent. */ if (cmd & (DO_LIST | DO_RETR)) { if (opt.ftp_pasv) { ip_address passive_addr; int passive_port; err = ftp_do_pasv (csock, &passive_addr, &passive_port); /* FTPRERR, WRITEFAILED, FTPNOPASV, FTPINVPASV */ switch (err) { case FTPRERR: logputs (LOG_VERBOSE, "\n"); logputs (LOG_NOTQUIET, _("\ Error in server response, closing control connection.\n")); fd_close (csock); con->csock = -1; return err; case WRITEFAILED: logputs (LOG_VERBOSE, "\n"); logputs (LOG_NOTQUIET, _("Write failed, closing control connection.\n")); fd_close (csock); con->csock = -1; return err; case FTPNOPASV: logputs (LOG_VERBOSE, "\n"); logputs (LOG_NOTQUIET, _("Cannot initiate PASV transfer.\n")); break; case FTPINVPASV: logputs (LOG_VERBOSE, "\n"); logputs (LOG_NOTQUIET, _("Cannot parse PASV response.\n")); break; case FTPOK: break; default: abort (); } /* switch (err) */ if (err==FTPOK) { DEBUGP (("trying to connect to %s port %d\n", print_address (&passive_addr), passive_port)); dtsock = connect_to_ip (&passive_addr, passive_port, NULL); if (dtsock < 0) { int save_errno = errno; fd_close (csock); con->csock = -1; logprintf (LOG_VERBOSE, _("couldn't connect to %s port %d: %s\n"), print_address (&passive_addr), passive_port, strerror (save_errno)); return (retryable_socket_connect_error (save_errno) ? CONERROR : CONIMPOSSIBLE); } if (!opt.server_response) logputs (LOG_VERBOSE, _("done. ")); } else return err; /* * We do not want to fall back from PASSIVE mode to ACTIVE mode ! * The reason is the PORT command exposes the client's real IP address * to the server. Bad for someone who relies on privacy via a ftp proxy. */ } else { err = ftp_do_port (csock, &local_sock); /* FTPRERR, WRITEFAILED, bindport (FTPSYSERR), HOSTERR, FTPPORTERR */ switch (err) { case FTPRERR: logputs (LOG_VERBOSE, "\n"); logputs (LOG_NOTQUIET, _("\ Error in server response, closing control connection.\n")); fd_close (csock); con->csock = -1; fd_close (dtsock); fd_close (local_sock); return err; case WRITEFAILED: logputs (LOG_VERBOSE, "\n"); logputs (LOG_NOTQUIET, _("Write failed, closing control connection.\n")); fd_close (csock); con->csock = -1; fd_close (dtsock); fd_close (local_sock); return err; case CONSOCKERR: logputs (LOG_VERBOSE, "\n"); logprintf (LOG_NOTQUIET, "socket: %s\n", strerror (errno)); fd_close (csock); con->csock = -1; fd_close (dtsock); fd_close (local_sock); return err; case FTPSYSERR: logputs (LOG_VERBOSE, "\n"); logprintf (LOG_NOTQUIET, _("Bind error (%s).\n"), strerror (errno)); fd_close (dtsock); return err; case FTPPORTERR: logputs (LOG_VERBOSE, "\n"); logputs (LOG_NOTQUIET, _("Invalid PORT.\n")); fd_close (csock); con->csock = -1; fd_close (dtsock); fd_close (local_sock); return err; case FTPOK: break; default: abort (); } /* port switch */ if (!opt.server_response) logputs (LOG_VERBOSE, _("done. ")); } /* dtsock == -1 */ } /* cmd & (DO_LIST | DO_RETR) */ /* Restart if needed. */ if (restval && (cmd & DO_RETR)) { if (!opt.server_response) logprintf (LOG_VERBOSE, "==> REST %s ... ", number_to_static_string (restval)); err = ftp_rest (csock, restval); /* FTPRERR, WRITEFAILED, FTPRESTFAIL */ switch (err) { case FTPRERR: logputs (LOG_VERBOSE, "\n"); logputs (LOG_NOTQUIET, _("\ Error in server response, closing control connection.\n")); fd_close (csock); con->csock = -1; fd_close (dtsock); fd_close (local_sock); return err; case WRITEFAILED: logputs (LOG_VERBOSE, "\n"); logputs (LOG_NOTQUIET, _("Write failed, closing control connection.\n")); fd_close (csock); con->csock = -1; fd_close (dtsock); fd_close (local_sock); return err; case FTPRESTFAIL: logputs (LOG_VERBOSE, _("\nREST failed, starting from scratch.\n")); rest_failed = true; break; case FTPOK: break; default: abort (); } if (err != FTPRESTFAIL && !opt.server_response) logputs (LOG_VERBOSE, _("done. ")); } /* restval && cmd & DO_RETR */ if (cmd & DO_RETR) { /* If we're in spider mode, don't really retrieve anything except the directory listing and verify whether the given "file" exists. */ if (opt.spider) { bool exists = false; bool all_exist = true; struct fileinfo *f; uerr_t _res = ftp_get_listing (u, original_url, con, &f); /* Set the DO_RETR command flag again, because it gets unset when calling ftp_get_listing() and would otherwise cause an assertion failure earlier on when this function gets repeatedly called (e.g., when recursing). */ con->cmd |= DO_RETR; if (_res == RETROK) { while (f) { if (!strcmp (f->name, u->file)) { exists = true; break; } else { all_exist = false; } f = f->next; } if (exists) { logputs (LOG_VERBOSE, "\n"); logprintf (LOG_NOTQUIET, _("File %s exists.\n"), quote (u->file)); } else { logputs (LOG_VERBOSE, "\n"); logprintf (LOG_NOTQUIET, _("No such file %s.\n"), quote (u->file)); } } fd_close (csock); con->csock = -1; fd_close (dtsock); fd_close (local_sock); if (all_exist) { return RETRFINISHED; } else { return FTPNSFOD; } } if (opt.verbose) { if (!opt.server_response) { if (restval) logputs (LOG_VERBOSE, "\n"); logprintf (LOG_VERBOSE, "==> RETR %s ... ", quotearg_style (escape_quoting_style, u->file)); } } err = ftp_retr (csock, u->file); /* FTPRERR, WRITEFAILED, FTPNSFOD */ switch (err) { case FTPRERR: logputs (LOG_VERBOSE, "\n"); logputs (LOG_NOTQUIET, _("\ Error in server response, closing control connection.\n")); fd_close (csock); con->csock = -1; fd_close (dtsock); fd_close (local_sock); return err; case WRITEFAILED: logputs (LOG_VERBOSE, "\n"); logputs (LOG_NOTQUIET, _("Write failed, closing control connection.\n")); fd_close (csock); con->csock = -1; fd_close (dtsock); fd_close (local_sock); return err; case FTPNSFOD: logputs (LOG_VERBOSE, "\n"); logprintf (LOG_NOTQUIET, _("No such file %s.\n\n"), quote (u->file)); fd_close (dtsock); fd_close (local_sock); return err; case FTPOK: break; default: abort (); } if (!opt.server_response) logputs (LOG_VERBOSE, _("done.\n")); if (! got_expected_bytes) expected_bytes = *last_expected_bytes; } /* do retrieve */ if (cmd & DO_LIST) { if (!opt.server_response) logputs (LOG_VERBOSE, "==> LIST ... "); /* As Maciej W. Rozycki (macro@ds2.pg.gda.pl) says, `LIST' without arguments is better than `LIST .'; confirmed by RFC959. */ err = ftp_list (csock, NULL, con->st&AVOID_LIST_A, con->st&AVOID_LIST, &list_a_used); /* FTPRERR, WRITEFAILED */ switch (err) { case FTPRERR: logputs (LOG_VERBOSE, "\n"); logputs (LOG_NOTQUIET, _("\ Error in server response, closing control connection.\n")); fd_close (csock); con->csock = -1; fd_close (dtsock); fd_close (local_sock); return err; case WRITEFAILED: logputs (LOG_VERBOSE, "\n"); logputs (LOG_NOTQUIET, _("Write failed, closing control connection.\n")); fd_close (csock); con->csock = -1; fd_close (dtsock); fd_close (local_sock); return err; case FTPNSFOD: logputs (LOG_VERBOSE, "\n"); logprintf (LOG_NOTQUIET, _("No such file or directory %s.\n\n"), quote (".")); fd_close (dtsock); fd_close (local_sock); return err; case FTPOK: break; default: abort (); } if (!opt.server_response) logputs (LOG_VERBOSE, _("done.\n")); if (! got_expected_bytes) expected_bytes = *last_expected_bytes; } /* cmd & DO_LIST */ if (!(cmd & (DO_LIST | DO_RETR)) || (opt.spider && !(cmd & DO_LIST))) return RETRFINISHED; /* Some FTP servers return the total length of file after REST command, others just return the remaining size. */ if (passed_expected_bytes && restval && expected_bytes && (expected_bytes == passed_expected_bytes - restval)) { DEBUGP (("Lying FTP server found, adjusting.\n")); expected_bytes = passed_expected_bytes; } /* If no transmission was required, then everything is OK. */ if (!opt.ftp_pasv) /* we are not using passive mode so we need to accept */ { /* Wait for the server to connect to the address we're waiting at. */ dtsock = accept_connection (local_sock); if (dtsock < 0) { logprintf (LOG_NOTQUIET, "accept: %s\n", strerror (errno)); return CONERROR; } } /* Open the file -- if output_stream is set, use it instead. */ /* 2005-04-17 SMS. Note that having the output_stream ("-O") file opened in main (main.c) rather limits the ability in VMS to open the file differently for ASCII versus binary FTP here. (Of course, doing it there allows a open failure to be detected immediately, without first connecting to the server.) */ if (!output_stream || con->cmd & DO_LIST) { /* On VMS, alter the name as required. */ #ifdef __VMS char *targ; targ = ods_conform (con->target); if (targ != con->target) { xfree (con->target); con->target = targ; } #endif /* def __VMS */ mkalldirs (con->target); if (opt.backups) rotate_backups (con->target); /* 2005-04-15 SMS. For VMS, define common fopen() optional arguments, and a handy macro for use as a variable "binary" flag. Elsewhere, define a constant "binary" flag. Isn't it nice to have distinct text and binary file types? */ /* 2011-09-30 SMS. Added listing files to the set of non-"binary" (text, Stream_LF) files. (Wget works either way, but other programs, like, say, text editors, work better on listing files which have text attributes.) Now we use "binary" attributes for a binary ("IMAGE") transfer, unless "--ftp-stmlf" was specified, and we always use non-"binary" (text, Stream_LF) attributes for a listing file, or for an ASCII transfer. Tidied the VMS-specific BIN_TYPE_xxx macros, and changed the call to fopen_excl() (restored?) to use BIN_TYPE_FILE instead of "true". */ #ifdef __VMS # define BIN_TYPE_TRANSFER (type_char != 'A') # define BIN_TYPE_FILE \ ((!(cmd & DO_LIST)) && BIN_TYPE_TRANSFER && (opt.ftp_stmlf == 0)) # define FOPEN_OPT_ARGS "fop=sqo", "acc", acc_cb, &open_id # define FOPEN_OPT_ARGS_BIN "ctx=bin,stm", "rfm=fix", "mrs=512" FOPEN_OPT_ARGS #else /* def __VMS */ # define BIN_TYPE_FILE true #endif /* def __VMS [else] */ if (restval && !(con->cmd & DO_LIST)) { #ifdef __VMS int open_id; if (BIN_TYPE_FILE) { open_id = 3; fp = fopen (con->target, "ab", FOPEN_OPT_ARGS_BIN); } else { open_id = 4; fp = fopen (con->target, "a", FOPEN_OPT_ARGS); } #else /* def __VMS */ fp = fopen (con->target, "ab"); #endif /* def __VMS [else] */ } else if (opt.noclobber || opt.always_rest || opt.timestamping || opt.dirstruct || opt.output_document || count > 0) { if (opt.unlink_requested && file_exists_p (con->target, NULL)) { if (unlink (con->target) < 0) { logprintf (LOG_NOTQUIET, "%s: %s\n", con->target, strerror (errno)); fd_close (csock); con->csock = -1; fd_close (dtsock); fd_close (local_sock); return UNLINKERR; } } #ifdef __VMS int open_id; if (BIN_TYPE_FILE) { open_id = 5; fp = fopen (con->target, "wb", FOPEN_OPT_ARGS_BIN); } else { open_id = 6; fp = fopen (con->target, "w", FOPEN_OPT_ARGS); } #else /* def __VMS */ fp = fopen (con->target, "wb"); #endif /* def __VMS [else] */ } else { fp = fopen_excl (con->target, BIN_TYPE_FILE); if (!fp && errno == EEXIST) { /* We cannot just invent a new name and use it (which is what functions like unique_create typically do) because we told the user we'd use this name. Instead, return and retry the download. */ logprintf (LOG_NOTQUIET, _("%s has sprung into existence.\n"), con->target); fd_close (csock); con->csock = -1; fd_close (dtsock); fd_close (local_sock); return FOPEN_EXCL_ERR; } } if (!fp) { logprintf (LOG_NOTQUIET, "%s: %s\n", con->target, strerror (errno)); fd_close (csock); con->csock = -1; fd_close (dtsock); fd_close (local_sock); return FOPENERR; } } else fp = output_stream; if (passed_expected_bytes) { print_length (passed_expected_bytes, restval, true); expected_bytes = passed_expected_bytes; /* for fd_read_body's progress bar */ } else if (expected_bytes) print_length (expected_bytes, restval, false); #ifdef HAVE_SSL if (u->scheme == SCHEME_FTPS && using_data_security) { /* We should try to restore the existing SSL session in the data connection * and fall back to establishing a new session if the server doesn't want to restore it. */ if (!opt.ftps_resume_ssl || !ssl_connect_wget (dtsock, u->host, &csock)) { if (opt.ftps_resume_ssl) logputs (LOG_NOTQUIET, "Server does not want to resume the SSL session. Trying with a new one.\n"); if (!ssl_connect_wget (dtsock, u->host, NULL)) { fd_close (csock); fd_close (dtsock); err = CONERROR; logputs (LOG_NOTQUIET, "Could not perform SSL handshake.\n"); goto exit_error; } } else logputs (LOG_NOTQUIET, "Resuming SSL session in data connection.\n"); if (!ssl_check_certificate (dtsock, u->host)) { fd_close (csock); fd_close (dtsock); err = CONERROR; goto exit_error; } } #endif /* Get the contents of the document. */ flags = 0; if (restval && rest_failed) flags |= rb_skip_startpos; rd_size = 0; res = fd_read_body (con->target, dtsock, fp, expected_bytes ? expected_bytes - restval : 0, restval, &rd_size, qtyread, &con->dltime, flags, warc_tmp); tms = datetime_str (time (NULL)); tmrate = retr_rate (rd_size, con->dltime); total_download_time += con->dltime; #ifdef ENABLE_XATTR if (opt.enable_xattr) set_file_metadata (u, NULL, fp); #endif fd_close (local_sock); /* Close the local file. */ if (!output_stream || con->cmd & DO_LIST) fclose (fp); /* If fd_read_body couldn't write to fp or warc_tmp, bail out. */ if (res == -2 || (warc_tmp != NULL && res == -3)) { logprintf (LOG_NOTQUIET, _("%s: %s, closing control connection.\n"), con->target, strerror (errno)); fd_close (csock); con->csock = -1; fd_close (dtsock); if (res == -2) return FWRITEERR; else if (res == -3) return WARC_TMP_FWRITEERR; } else if (res == -1) { logprintf (LOG_NOTQUIET, _("%s (%s) - Data connection: %s; "), tms, tmrate, fd_errstr (dtsock)); if (opt.server_response) logputs (LOG_ALWAYS, "\n"); } fd_close (dtsock); /* Get the server to tell us if everything is retrieved. */ err = ftp_response (csock, &respline); if (err != FTPOK) { /* The control connection is decidedly closed. Print the time only if it hasn't already been printed. */ if (res != -1) logprintf (LOG_NOTQUIET, "%s (%s) - ", tms, tmrate); logputs (LOG_NOTQUIET, _("Control connection closed.\n")); /* If there is an error on the control connection, close it, but return FTPRETRINT, since there is a possibility that the whole file was retrieved nevertheless (but that is for ftp_loop_internal to decide). */ fd_close (csock); con->csock = -1; return FTPRETRINT; } /* err != FTPOK */ *last_expected_bytes = ftp_expected_bytes (respline); /* If retrieval failed for any reason, return FTPRETRINT, but do not close socket, since the control connection is still alive. If there is something wrong with the control connection, it will become apparent later. */ if (*respline != '2') { if (res != -1) logprintf (LOG_NOTQUIET, "%s (%s) - ", tms, tmrate); logputs (LOG_NOTQUIET, _("Data transfer aborted.\n")); #ifdef HAVE_SSL if (!c_strncasecmp (respline, "425", 3) && u->scheme == SCHEME_FTPS) { logputs (LOG_NOTQUIET, "FTPS server rejects new SSL sessions in the data connection.\n"); xfree (respline); return FTPRESTFAIL; } #endif xfree (respline); return FTPRETRINT; } xfree (respline); if (res == -1) { /* What now? The data connection was erroneous, whereas the response says everything is OK. We shall play it safe. */ return FTPRETRINT; } if (!(cmd & LEAVE_PENDING)) { /* Closing the socket is faster than sending 'QUIT' and the effect is the same. */ fd_close (csock); con->csock = -1; } /* If it was a listing, and opt.server_response is true, print it out. */ if (con->cmd & DO_LIST) { if (opt.server_response) { /* 2005-02-25 SMS. Much of this work may already have been done, but repeating it should do no damage beyond wasting time. */ /* On VMS, alter the name as required. */ #ifdef __VMS char *targ; targ = ods_conform (con->target); if (targ != con->target) { xfree (con->target); con->target = targ; } #endif /* def __VMS */ mkalldirs (con->target); fp = fopen (con->target, "r"); if (!fp) logprintf (LOG_ALWAYS, "%s: %s\n", con->target, strerror (errno)); else { char *line = NULL; size_t bufsize = 0; ssize_t len; /* The lines are being read with getline because of no-buffering on opt.lfile. */ while ((len = getline (&line, &bufsize, fp)) > 0) { while (len > 0 && (line[len - 1] == '\n' || line[len - 1] == '\r')) line[--len] = '\0'; logprintf (LOG_ALWAYS, "%s\n", quotearg_style (escape_quoting_style, line)); } xfree (line); fclose (fp); } } /* server_response */ /* 2013-10-17 Andrea Urbani (matfanjol) < __LIST_A_EXPLANATION__ > After the SYST command, looks if it knows that system. If yes, wget will force the use of "LIST" or "LIST -a". If no, wget will try, only the first time of each session, before the "LIST -a" command and after the "LIST". If "LIST -a" works and returns more or equal data of the "LIST", "LIST -a" will be the standard list command for all the session. If "LIST -a" fails or returns less data than "LIST" (think on the case of an existing file called "-a"), "LIST" will be the standard list command for all the session. ("LIST -a" is used to get also the hidden files) */ if (!(con->st & LIST_AFTER_LIST_A_CHECK_DONE)) { /* We still have to check "LIST" after the first "LIST -a" to see if with "LIST" we get more data than "LIST -a", that means "LIST -a" returned files/folders with "-a" name. */ if (con->st & AVOID_LIST_A) { /* LIST was used in this cycle. Let's see the result. */ if (rd_size > previous_rd_size) { /* LIST returns more data than "LIST -a". "LIST" is the official command to use. */ con->st |= LIST_AFTER_LIST_A_CHECK_DONE; DEBUGP (("LIST returned more data than \"LIST -a\": " "I will use \"LIST\" as standard list command\n")); } else if (previous_rd_size > rd_size) { /* "LIST -a" returned more data then LIST. "LIST -a" is the official command to use. */ con->st |= LIST_AFTER_LIST_A_CHECK_DONE; con->st |= AVOID_LIST; con->st &= ~AVOID_LIST_A; /* Sorry, please, download again the "LIST -a"... */ try_again = true; DEBUGP (("LIST returned less data than \"LIST -a\": I will " "use \"LIST -a\" as standard list command\n")); } else { /* LIST and "LIST -a" return the same data. */ if (rd_size == 0) { /* Same empty data. We will check both again because we cannot check if "LIST -a" has returned an empty folder instead of a folder content. */ con->st &= ~AVOID_LIST_A; } else { /* Same data, so, better to take "LIST -a" that shows also hidden files/folders (when present) */ con->st |= LIST_AFTER_LIST_A_CHECK_DONE; con->st |= AVOID_LIST; con->st &= ~AVOID_LIST_A; DEBUGP (("LIST returned the same amount of data of " "\"LIST -a\": I will use \"LIST -a\" as standard " "list command\n")); } } } else { /* In this cycle "LIST -a" should being used. Is it true? */ if (list_a_used) { /* Yes, it is. OK, let's save the amount of data and try again with LIST */ previous_rd_size = rd_size; try_again = true; con->st |= AVOID_LIST_A; } else { /* No: something happens and LIST was used. This means "LIST -a" raises an error. */ con->st |= LIST_AFTER_LIST_A_CHECK_DONE; con->st |= AVOID_LIST_A; DEBUGP (("\"LIST -a\" failed: I will use \"LIST\" " "as standard list command\n")); } } } } } while (try_again); return RETRFINISHED; exit_error: /* If fp is a regular file, close and try to remove it */ if (fp && (!output_stream || con->cmd & DO_LIST)) fclose (fp); return err; } /* A one-file FTP loop. This is the part where FTP retrieval is retried, and retried, and retried, and... This loop either gets commands from con, or (if ON_YOUR_OWN is set), makes them up to retrieve the file given by the URL. */ static uerr_t ftp_loop_internal (struct url *u, struct url *original_url, struct fileinfo *f, ccon *con, char **local_file, bool force_full_retrieve) { int count, orig_lp; wgint restval, len = 0, qtyread = 0; char *tms, *locf; const char *tmrate = NULL; uerr_t err; struct stat st; /* Declare WARC variables. */ bool warc_enabled = (opt.warc_filename != NULL); FILE *warc_tmp = NULL; ip_address warc_ip_buf, *warc_ip = NULL; wgint last_expected_bytes = 0; /* Get the target, and set the name for the message accordingly. */ if ((f == NULL) && (con->target)) { /* Explicit file (like ".listing"). */ locf = con->target; } else { /* URL-derived file. Consider "-O file" name. */ xfree (con->target); con->target = url_file_name (opt.trustservernames || !original_url ? u : original_url, NULL); if (!opt.output_document) locf = con->target; else locf = opt.output_document; } /* If the output_document was given, then this check was already done and the file didn't exist. Hence the !opt.output_document */ /* If we receive .listing file it is necessary to determine system type of the ftp server even if opn.noclobber is given. Thus we must ignore opt.noclobber in order to establish connection with the server and get system type. */ if (opt.noclobber && !opt.output_document && file_exists_p (con->target, NULL) && !((con->cmd & DO_LIST) && !(con->cmd & DO_RETR))) { logprintf (LOG_VERBOSE, _("File %s already there; not retrieving.\n"), quote (con->target)); /* If the file is there, we suppose it's retrieved OK. */ return RETROK; } /* Remove it if it's a link. */ remove_link (con->target); count = 0; if (con->st & ON_YOUR_OWN) con->st = ON_YOUR_OWN; orig_lp = con->cmd & LEAVE_PENDING ? 1 : 0; /* THE loop. */ do { /* Increment the pass counter. */ ++count; sleep_between_retrievals (count); if (con->st & ON_YOUR_OWN) { con->cmd = 0; con->cmd |= (DO_RETR | LEAVE_PENDING); if (con->csock != -1) con->cmd &= ~ (DO_LOGIN | DO_CWD); else con->cmd |= (DO_LOGIN | DO_CWD); } else /* not on your own */ { if (con->csock != -1) con->cmd &= ~DO_LOGIN; else con->cmd |= DO_LOGIN; if (con->st & DONE_CWD) con->cmd &= ~DO_CWD; else con->cmd |= DO_CWD; } /* For file RETR requests, we can write a WARC record. We record the file contents to a temporary file. */ if (warc_enabled && (con->cmd & DO_RETR) && warc_tmp == NULL) { warc_tmp = warc_tempfile (); if (warc_tmp == NULL) return WARC_TMP_FOPENERR; if (!con->proxy && con->csock != -1) { warc_ip = &warc_ip_buf; socket_ip_address (con->csock, warc_ip, ENDPOINT_PEER); } } /* Decide whether or not to restart. */ if (con->cmd & DO_LIST) restval = 0; else if (force_full_retrieve) restval = 0; else if (opt.start_pos >= 0) restval = opt.start_pos; else if (opt.always_rest && stat (locf, &st) == 0 && S_ISREG (st.st_mode)) /* When -c is used, continue from on-disk size. (Can't use hstat.len even if count>1 because we don't want a failed first attempt to clobber existing data.) */ restval = st.st_size; else if (count > 1) restval = qtyread; /* start where the previous run left off */ else restval = 0; /* Get the current time string. */ tms = datetime_str (time (NULL)); /* Print fetch message, if opt.verbose. */ if (opt.verbose) { char *hurl = url_string (u, URL_AUTH_HIDE_PASSWD); char tmp[256]; strcpy (tmp, " "); if (count > 1) sprintf (tmp, _("(try:%2d)"), count); logprintf (LOG_VERBOSE, "--%s-- %s\n %s => %s\n", tms, hurl, tmp, quote (locf)); #ifdef WINDOWS ws_changetitle (hurl); #endif xfree (hurl); } /* Send getftp the proper length, if fileinfo was provided. */ if (f && f->type != FT_SYMLINK) len = f->size; else len = 0; /* If we are working on a WARC record, getftp should also write to the warc_tmp file. */ err = getftp (u, original_url, len, &qtyread, restval, con, count, &last_expected_bytes, warc_tmp); if (con->csock == -1) con->st &= ~DONE_CWD; else con->st |= DONE_CWD; switch (err) { case HOSTERR: case CONIMPOSSIBLE: case FWRITEERR: case FOPENERR: case FTPNSFOD: case FTPLOGINC: case FTPNOPASV: case FTPNOAUTH: case FTPNOPBSZ: case FTPNOPROT: case UNLINKERR: case WARC_TMP_FWRITEERR: case CONSSLERR: case CONTNOTSUPPORTED: case VERIFCERTERR: #ifdef HAVE_SSL if (err == FTPNOAUTH) logputs (LOG_NOTQUIET, "Server does not support AUTH TLS.\n"); if (opt.ftps_implicit) logputs (LOG_NOTQUIET, "Server does not like implicit FTPS connections.\n"); #endif /* Fatal errors, give up. */ if (warc_tmp != NULL) { fclose (warc_tmp); warc_tmp = NULL; } return err; case CONSOCKERR: case CONERROR: case FTPSRVERR: case FTPRERR: case WRITEFAILED: case FTPUNKNOWNTYPE: case FTPSYSERR: case FTPPORTERR: case FTPLOGREFUSED: case FTPINVPASV: case FOPEN_EXCL_ERR: printwhat (count, opt.ntry); /* non-fatal errors */ if (err == FOPEN_EXCL_ERR) { /* Re-determine the file name. */ xfree (con->target); con->target = url_file_name (u, NULL); locf = con->target; } continue; case FTPRETRINT: /* If the control connection was closed, the retrieval will be considered OK if f->size == len. */ if (!f || qtyread != f->size) { printwhat (count, opt.ntry); continue; } break; case RETRFINISHED: /* Great! */ break; default: /* Not as great. */ abort (); } tms = datetime_str (time (NULL)); if (!opt.spider) tmrate = retr_rate (qtyread - restval, con->dltime); /* If we get out of the switch above without continue'ing, we've successfully downloaded a file. Remember this fact. */ downloaded_file (FILE_DOWNLOADED_NORMALLY, locf); if (con->st & ON_YOUR_OWN) { fd_close (con->csock); con->csock = -1; } if (!opt.spider) { bool write_to_stdout = (opt.output_document && HYPHENP (opt.output_document)); logprintf (LOG_VERBOSE, write_to_stdout ? _("%s (%s) - written to stdout %s[%s]\n\n") : _("%s (%s) - %s saved [%s]\n\n"), tms, tmrate, write_to_stdout ? "" : quote (locf), number_to_static_string (qtyread)); } if (!opt.verbose && !opt.quiet) { /* Need to hide the password from the URL. The `if' is here so that we don't do the needless allocation every time. */ char *hurl = url_string (u, URL_AUTH_HIDE_PASSWD); logprintf (LOG_NONVERBOSE, "%s URL: %s [%s] -> \"%s\" [%d]\n", tms, hurl, number_to_static_string (qtyread), locf, count); xfree (hurl); } if (warc_enabled && (con->cmd & DO_RETR)) { /* Create and store a WARC resource record for the retrieved file. */ bool warc_res; warc_res = warc_write_resource_record (NULL, u->url, NULL, NULL, warc_ip, NULL, warc_tmp, -1); if (! warc_res) return WARC_ERR; /* warc_write_resource_record has also closed warc_tmp. */ warc_tmp = NULL; } if (con->cmd & DO_LIST) /* This is a directory listing file. */ { if (!opt.remove_listing) /* --dont-remove-listing was specified, so do count this towards the number of bytes and files downloaded. */ { total_downloaded_bytes += qtyread; numurls++; } /* Deletion of listing files is not controlled by --delete-after, but by the more specific option --dont-remove-listing, and the code to do this deletion is in another function. */ } else if (!opt.spider) /* This is not a directory listing file. */ { /* Unlike directory listing files, don't pretend normal files weren't downloaded if they're going to be deleted. People seeding proxies, for instance, may want to know how many bytes and files they've downloaded through it. */ total_downloaded_bytes += qtyread; numurls++; if (opt.delete_after && !input_file_url (opt.input_filename)) { DEBUGP (("\ Removing file due to --delete-after in ftp_loop_internal():\n")); logprintf (LOG_VERBOSE, _("Removing %s.\n"), locf); if (unlink (locf)) logprintf (LOG_NOTQUIET, "unlink: %s\n", strerror (errno)); } } /* Restore the original leave-pendingness. */ if (orig_lp) con->cmd |= LEAVE_PENDING; else con->cmd &= ~LEAVE_PENDING; if (local_file) *local_file = xstrdup (locf); if (warc_tmp != NULL) { fclose (warc_tmp); warc_tmp = NULL; } return RETROK; } while (!opt.ntry || (count < opt.ntry)); if (con->csock != -1 && (con->st & ON_YOUR_OWN)) { fd_close (con->csock); con->csock = -1; } if (warc_tmp != NULL) fclose (warc_tmp); return TRYLIMEXC; } /* Return the directory listing in a reusable format. The directory is specified in u->dir. */ static uerr_t ftp_get_listing (struct url *u, struct url *original_url, ccon *con, struct fileinfo **f) { uerr_t err; char *uf; /* url file name */ char *lf; /* list file name */ char *old_target = con->target; con->st &= ~ON_YOUR_OWN; con->cmd |= (DO_LIST | LEAVE_PENDING); con->cmd &= ~DO_RETR; /* Find the listing file name. We do it by taking the file name of the URL and replacing the last component with the listing file name. */ uf = url_file_name (u, NULL); lf = file_merge (uf, LIST_FILENAME); xfree (uf); DEBUGP ((_("Using %s as listing tmp file.\n"), quote (lf))); con->target = xstrdup (lf); xfree (lf); err = ftp_loop_internal (u, original_url, NULL, con, NULL, false); lf = xstrdup (con->target); xfree (con->target); con->target = old_target; if (err == RETROK) { *f = ftp_parse_ls (lf, con->rs); if (opt.remove_listing) { if (unlink (lf)) logprintf (LOG_NOTQUIET, "unlink: %s\n", strerror (errno)); else logprintf (LOG_VERBOSE, _("Removed %s.\n"), quote (lf)); } } else *f = NULL; xfree (lf); con->cmd &= ~DO_LIST; return err; } static uerr_t ftp_retrieve_dirs (struct url *, struct url *, struct fileinfo *, ccon *); static uerr_t ftp_retrieve_glob (struct url *, struct url *, ccon *, int); static struct fileinfo *delelement (struct fileinfo **, struct fileinfo **); /* Retrieve a list of files given in struct fileinfo linked list. If a file is a symbolic link, do not retrieve it, but rather try to set up a similar link on the local disk, if the symlinks are supported. If opt.recursive is set, after all files have been retrieved, ftp_retrieve_dirs will be called to retrieve the directories. */ static uerr_t ftp_retrieve_list (struct url *u, struct url *original_url, struct fileinfo *f, ccon *con) { static int depth = 0; uerr_t err; struct fileinfo *orig; wgint local_size; time_t tml; bool dlthis; /* Download this (file). */ const char *actual_target = NULL; bool force_full_retrieve = false; /* Increase the depth. */ ++depth; if (opt.reclevel != INFINITE_RECURSION && depth > opt.reclevel) { DEBUGP ((_("Recursion depth %d exceeded max. depth %d.\n"), depth, opt.reclevel)); --depth; return RECLEVELEXC; } assert (f != NULL); orig = f; con->st &= ~ON_YOUR_OWN; if (!(con->st & DONE_CWD)) con->cmd |= DO_CWD; else con->cmd &= ~DO_CWD; con->cmd |= (DO_RETR | LEAVE_PENDING); if (con->csock < 0) con->cmd |= DO_LOGIN; else con->cmd &= ~DO_LOGIN; err = RETROK; /* in case it's not used */ while (f) { char *old_target, *ofile; if (opt.quota && total_downloaded_bytes > opt.quota) { --depth; return QUOTEXC; } old_target = con->target; ofile = xstrdup (u->file); url_set_file (u, f->name); con->target = url_file_name (u, NULL); err = RETROK; dlthis = true; if (opt.timestamping && f->type == FT_PLAINFILE) { struct stat st; /* If conversion of HTML files retrieved via FTP is ever implemented, we'll need to stat() .orig here when -K has been specified. I'm not implementing it now since files on an FTP server are much more likely than files on an HTTP server to legitimately have a .orig suffix. */ if (!stat (con->target, &st)) { bool eq_size; bool cor_val; /* Else, get it from the file. */ local_size = st.st_size; tml = st.st_mtime; #ifdef WINDOWS /* Modification time granularity is 2 seconds for Windows, so increase local time by 1 second for later comparison. */ tml++; #endif /* Compare file sizes only for servers that tell us correct values. Assume sizes being equal for servers that lie about file size. */ cor_val = (con->rs == ST_UNIX || con->rs == ST_WINNT); eq_size = cor_val ? (local_size == f->size) : true; if (f->tstamp <= tml && eq_size) { /* Remote file is older, file sizes can be compared and are both equal. */ logprintf (LOG_VERBOSE, _("\ Remote file no newer than local file %s -- not retrieving.\n"), quote (con->target)); dlthis = false; } else if (f->tstamp > tml) { /* Remote file is newer */ force_full_retrieve = true; logprintf (LOG_VERBOSE, _("\ Remote file is newer than local file %s -- retrieving.\n\n"), quote (con->target)); } else { /* Sizes do not match */ logprintf (LOG_VERBOSE, _("\ The sizes do not match (local %s) -- retrieving.\n\n"), number_to_static_string (local_size)); } } } /* opt.timestamping && f->type == FT_PLAINFILE */ switch (f->type) { case FT_SYMLINK: /* If opt.retr_symlinks is defined, we treat symlinks as if they were normal files. There is currently no way to distinguish whether they might be directories, and follow them. */ if (!opt.retr_symlinks) { #ifdef HAVE_SYMLINK if (!f->linkto) logputs (LOG_NOTQUIET, _("Invalid name of the symlink, skipping.\n")); else { struct stat st; /* Check whether we already have the correct symbolic link. */ int rc = lstat (con->target, &st); if (rc == 0) { size_t len = strlen (f->linkto) + 1; if (S_ISLNK (st.st_mode)) { char buf[1024], *link_target; size_t n; bool res; if (len < sizeof (buf)) link_target = buf; else link_target = xmalloc (len); n = readlink (con->target, link_target, len); res = (n == len - 1) && (memcmp (link_target, f->linkto, n) == 0); if (link_target != buf) xfree (link_target); if (res) { logprintf (LOG_VERBOSE, _("\ Already have correct symlink %s -> %s\n\n"), quote_n (0, con->target), quote_n (1, f->linkto)); dlthis = false; break; } } } logprintf (LOG_VERBOSE, _("Creating symlink %s -> %s\n"), quote_n (0, con->target), quote_n (1, f->linkto)); /* Unlink before creating symlink! */ unlink (con->target); if (symlink (f->linkto, con->target) == -1) logprintf (LOG_NOTQUIET, "symlink: %s\n", strerror (errno)); logputs (LOG_VERBOSE, "\n"); } /* have f->linkto */ #else /* not HAVE_SYMLINK */ logprintf (LOG_NOTQUIET, _("Symlinks not supported, skipping symlink %s.\n"), quote (con->target)); #endif /* not HAVE_SYMLINK */ } else /* opt.retr_symlinks */ { if (dlthis) { err = ftp_loop_internal (u, original_url, f, con, NULL, force_full_retrieve); } } /* opt.retr_symlinks */ break; case FT_DIRECTORY: if (!opt.recursive) logprintf (LOG_NOTQUIET, _("Skipping directory %s.\n"), quote (f->name)); break; case FT_PLAINFILE: /* Call the retrieve loop. */ if (dlthis) { err = ftp_loop_internal (u, original_url, f, con, NULL, force_full_retrieve); } break; case FT_UNKNOWN: default: logprintf (LOG_NOTQUIET, _("%s: unknown/unsupported file type.\n"), quote (f->name)); break; } /* switch */ /* 2004-12-15 SMS. * Set permissions _before_ setting the times, as setting the * permissions changes the modified-time, at least on VMS. * Also, use the opt.output_document name here, too, as * appropriate. (Do the test once, and save the result.) */ set_local_file (&actual_target, con->target); /* If downloading a plain file, and the user requested it, then set valid (non-zero) permissions. */ if (dlthis && (actual_target != NULL) && (f->type == FT_PLAINFILE) && opt.preserve_perm) { if (f->perms) { if (chmod (actual_target, f->perms)) logprintf (LOG_NOTQUIET, _("Failed to set permissions for %s.\n"), actual_target); } else DEBUGP (("Unrecognized permissions for %s.\n", actual_target)); } /* Set the time-stamp information to the local file. Symlinks are not to be stamped because it sets the stamp on the original. :( */ if (actual_target != NULL) { if (opt.useservertimestamps && !(f->type == FT_SYMLINK && !opt.retr_symlinks) && f->tstamp != -1 && dlthis && file_exists_p (con->target, NULL)) { touch (actual_target, f->tstamp); } else if (f->tstamp == -1) logprintf (LOG_NOTQUIET, _("%s: corrupt time-stamp.\n"), actual_target); } xfree (con->target); con->target = old_target; url_set_file (u, ofile); xfree (ofile); /* Break on fatals. */ if (err == QUOTEXC || err == HOSTERR || err == FWRITEERR || err == WARC_ERR || err == WARC_TMP_FOPENERR || err == WARC_TMP_FWRITEERR) break; con->cmd &= ~ (DO_CWD | DO_LOGIN); f = f->next; } /* We do not want to call ftp_retrieve_dirs here */ if (opt.recursive && !(opt.reclevel != INFINITE_RECURSION && depth >= opt.reclevel)) err = ftp_retrieve_dirs (u, original_url, orig, con); else if (opt.recursive) DEBUGP ((_("Will not retrieve dirs since depth is %d (max %d).\n"), depth, opt.reclevel)); --depth; return err; } /* Retrieve the directories given in a file list. This function works by simply going through the linked list and calling ftp_retrieve_glob on each directory entry. The function knows about excluded directories. */ static uerr_t ftp_retrieve_dirs (struct url *u, struct url *original_url, struct fileinfo *f, ccon *con) { char buf[1024]; char *container = buf; int container_size = sizeof (buf); for (; f; f = f->next) { int size; char *odir, *newdir; if (opt.quota && total_downloaded_bytes > opt.quota) break; if (f->type != FT_DIRECTORY) continue; /* Allocate u->dir off stack, but reallocate only if a larger string is needed. It's a pity there's no "realloca" for an item on the bottom of the stack. */ size = strlen (u->dir) + 1 + strlen (f->name) + 1; if (size > container_size) { if (container == buf) container = xmalloc (size); else container = xrealloc (container, size); container_size = size; } newdir = container; odir = u->dir; if (*odir == '\0' || (*odir == '/' && *(odir + 1) == '\0')) /* If ODIR is empty or just "/", simply append f->name to ODIR. (In the former case, to preserve u->dir being relative; in the latter case, to avoid double slash.) */ sprintf (newdir, "%s%s", odir, f->name); else /* Else, use a separator. */ sprintf (newdir, "%s/%s", odir, f->name); DEBUGP (("Composing new CWD relative to the initial directory.\n")); DEBUGP ((" odir = '%s'\n f->name = '%s'\n newdir = '%s'\n\n", odir, f->name, newdir)); if (!accdir (newdir)) { logprintf (LOG_VERBOSE, _("\ Not descending to %s as it is excluded/not-included.\n"), quote (newdir)); continue; } con->st &= ~DONE_CWD; odir = xstrdup (u->dir); /* because url_set_dir will free u->dir. */ url_set_dir (u, newdir); ftp_retrieve_glob (u, original_url, con, GLOB_GETALL); url_set_dir (u, odir); xfree (odir); /* Set the time-stamp? */ } if (container != buf) xfree (container); if (opt.quota && total_downloaded_bytes > opt.quota) return QUOTEXC; else return RETROK; } /* Return true if S has a leading '/' or contains '../' */ static bool has_insecure_name_p (const char *s) { if (*s == '/') return true; if (strstr (s, "../") != 0) return true; return false; } /* Test if the file node is invalid. This can occur due to malformed or * maliciously crafted listing files being returned by the server. * * Currently, this function only tests if there are multiple entries in the * listing file by the same name. However this function can be expanded as more * such illegal listing formats are discovered. */ static bool is_invalid_entry (struct fileinfo *f) { struct fileinfo *cur = f; char *f_name = f->name; /* If the node we're currently checking has a duplicate later, we eliminate * the current node and leave the next one intact. */ while (cur->next) { cur = cur->next; if (strcmp (f_name, cur->name) == 0) return true; } return false; } /* A near-top-level function to retrieve the files in a directory. The function calls ftp_get_listing, to get a linked list of files. Then it weeds out the file names that do not match the pattern. ftp_retrieve_list is called with this updated list as an argument. If the argument ACTION is GLOB_GETONE, just download the file (but first get the listing, so that the time-stamp is heeded); if it's GLOB_GLOBALL, use globbing; if it's GLOB_GETALL, download the whole directory. */ static uerr_t ftp_retrieve_glob (struct url *u, struct url *original_url, ccon *con, int action) { struct fileinfo *f, *start; uerr_t res; con->cmd |= LEAVE_PENDING; res = ftp_get_listing (u, original_url, con, &start); if (res != RETROK) return res; // Set the function used for glob matching. int (*matcher) (const char *, const char *, int) = opt.ignore_case ? fnmatch_nocase : fnmatch; // Set the function used to compare strings #ifdef __VMS /* 2009-09-09 SMS. * Odd-ball compiler ("HP C V7.3-009 on OpenVMS Alpha V7.3-2") * bug causes spurious %CC-E-BADCONDIT complaint with this * "?:" statement. (Different linkage attributes for strcmp() * and strcasecmp().) Converting to "if" changes the * complaint to %CC-W-PTRMISMATCH on "cmp = strcmp;". Adding * the senseless type cast clears the complaint, and looks * harmless. */ int (*cmp) (const char *, const char *) = opt.ignore_case ? strcasecmp : (int (*)())strcmp; #else /* def __VMS */ int (*cmp) (const char *, const char *) = opt.ignore_case ? strcasecmp : strcmp; #endif /* def __VMS [else] */ f = start; while (f) { // Weed out files that do not confirm to the global rules given in // opt.accepts and opt.rejects if ((opt.accepts || opt.rejects) && f->type != FT_DIRECTORY && !acceptable (f->name)) { logprintf (LOG_VERBOSE, _("Rejecting %s.\n"), quote (f->name)); f = delelement (&f, &start); continue; } // Identify and eliminate possibly harmful names or invalid entries. if (has_insecure_name_p (f->name) || is_invalid_entry (f)) { logprintf (LOG_VERBOSE, _("Rejecting %s (Invalid Entry).\n"), quote (f->name)); f = delelement (&f, &start); continue; } if (opt.acceptregex || opt.rejectregex) { // accept_url() takes the full URL. char buf[1024]; char *url = buf; if ((unsigned) snprintf(buf, sizeof(buf), "%s%s%s", u->url, f->name, f->type == FT_DIRECTORY ? "/" : "") >= sizeof(buf)) { url = aprintf("%s%s%s", u->url, f->name, f->type == FT_DIRECTORY ? "/" : ""); } if (!accept_url (url)) { logprintf (LOG_VERBOSE, _ ("%s is excluded/not-included through regex.\n"), url); f = delelement (&f, &start); if (url != buf) xfree(url); continue; } if (url != buf) xfree(url); } /* Now weed out the files that do not match our globbing pattern. If we are dealing with a globbing pattern, that is. */ if (*u->file) { if (action == GLOB_GLOBALL) { int matchres = matcher (u->file, f->name, 0); if (matchres == -1) { logprintf (LOG_NOTQUIET, _("Error matching %s against %s: %s\n"), u->file, quotearg_style (escape_quoting_style, f->name), strerror (errno)); freefileinfo (start); return RETRBADPATTERN; } if (matchres == FNM_NOMATCH) { f = delelement (&f, &start); /* delete the element from the list */ continue; } } else if (action == GLOB_GETONE) { if (0 != cmp(u->file, f->name)) { f = delelement (&f, &start); continue; } } } f = f->next; } /* * Now that preprocessing of the file listing is over, let's try to download * all the remaining files in our listing. */ if (start) { /* Just get everything. */ res = ftp_retrieve_list (u, original_url, start, con); } else { if (action == GLOB_GLOBALL) { /* No luck. */ /* #### This message SUCKS. We should see what was the reason that nothing was retrieved. */ logprintf (LOG_VERBOSE, _("No matches on pattern %s.\n"), quote (u->file)); } else if (action == GLOB_GETONE) /* GLOB_GETONE or GLOB_GETALL */ { /* Let's try retrieving it anyway. */ con->st |= ON_YOUR_OWN; res = ftp_loop_internal (u, original_url, NULL, con, NULL, false); return res; } /* If action == GLOB_GETALL, and the file list is empty, there's no point in trying to download anything or in complaining about it. (An empty directory should not cause complaints.) */ } freefileinfo (start); if (opt.quota && total_downloaded_bytes > opt.quota) return QUOTEXC; else return res; } /* The wrapper that calls an appropriate routine according to contents of URL. Inherently, its capabilities are limited on what can be encoded into a URL. */ uerr_t ftp_loop (struct url *u, struct url *original_url, char **local_file, int *dt, struct url *proxy, bool recursive, bool glob) { ccon con; /* FTP connection */ uerr_t res; *dt = 0; xzero (con); con.csock = -1; con.st = ON_YOUR_OWN; con.rs = ST_UNIX; con.id = NULL; con.proxy = proxy; /* If the file name is empty, the user probably wants a directory index. We'll provide one, properly HTML-ized. Unless opt.htmlify is 0, of course. :-) */ if (!*u->file && !recursive) { struct fileinfo *f; res = ftp_get_listing (u, original_url, &con, &f); if (res == RETROK) { if (opt.htmlify && !opt.spider) { struct url *url_file = opt.trustservernames ? u : original_url; char *filename = (opt.output_document ? xstrdup (opt.output_document) : (con.target ? xstrdup (con.target) : url_file_name (url_file, NULL))); res = ftp_index (filename, u, f); if (res == FTPOK && opt.verbose) { if (!opt.output_document) { struct stat st; wgint sz; if (stat (filename, &st) == 0) sz = st.st_size; else sz = -1; logprintf (LOG_NOTQUIET, _("Wrote HTML-ized index to %s [%s].\n"), quote (filename), number_to_static_string (sz)); } else logprintf (LOG_NOTQUIET, _("Wrote HTML-ized index to %s.\n"), quote (filename)); } xfree (filename); } freefileinfo (f); } } else { bool ispattern = false; if (glob) { /* Treat the URL as a pattern if the file name part of the URL path contains wildcards. (Don't check for u->file because it is unescaped and therefore doesn't leave users the option to escape literal '*' as %2A.) */ char *file_part = strrchr (u->path, '/'); if (!file_part) file_part = u->path; ispattern = has_wildcards_p (file_part); } if (ispattern || recursive || opt.timestamping || opt.preserve_perm) { /* ftp_retrieve_glob is a catch-all function that gets called if we need globbing, time-stamping, recursion or preserve permissions. Its third argument is just what we really need. */ res = ftp_retrieve_glob (u, original_url, &con, ispattern ? GLOB_GLOBALL : GLOB_GETONE); } else { res = ftp_loop_internal (u, original_url, NULL, &con, local_file, false); } } if (res == FTPOK) res = RETROK; if (res == RETROK) *dt |= RETROKF; /* If a connection was left, quench it. */ if (con.csock != -1) fd_close (con.csock); xfree (con.id); xfree (con.target); return res; } /* Delete an element from the fileinfo linked list. Returns the address of the next element, or NULL if the list is exhausted. It can modify the start of the list. */ static struct fileinfo * delelement (struct fileinfo **f, struct fileinfo **start) { struct fileinfo *prev = (*f)->prev; struct fileinfo *next = (*f)->next; xfree ((*f)->name); xfree ((*f)->linkto); xfree (*f); *f = NULL; if (next) next->prev = prev; if (prev) prev->next = next; else *start = next; return next; } /* Free the fileinfo linked list of files. */ void freefileinfo (struct fileinfo *f) { while (f) { struct fileinfo *next = f->next; xfree (f->name); if (f->linkto) xfree (f->linkto); xfree (f); f = next; } } wget-1.21.2/src/css-url.c0000644000000000000000000001462214115732710011763 00000000000000/* Collect URLs from CSS source. Copyright (C) 1998, 2000-2003, 2009-2011, 2014-2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ /* Note that this is not an actual CSS parser, but just a lexical scanner with a tiny bit more smarts bolted on top. A full parser is somewhat overkill for this job. The only things we're interested in are @import rules and url() tokens, so it's easy enough to grab those without truly understanding the input. The only downside to this is that we might be coerced into downloading files that a browser would ignore. That might merit some more investigation. */ #include #include #include #include #include #include #include "wget.h" #include "utils.h" #include "convert.h" #include "html-url.h" #include "css-tokens.h" #include "css-url.h" #include "xstrndup.h" /* from lex.yy.c */ extern char *yytext; extern int yyleng; typedef struct yy_buffer_state *YY_BUFFER_STATE; extern YY_BUFFER_STATE yy_scan_bytes (const char *bytes,int len ); extern void yy_delete_buffer (YY_BUFFER_STATE b); extern int yylex (void); extern void yylex_destroy(void); /* Given a detected URI token, get only the URI specified within. Also adjust the starting position and length of the string. A URI can be specified with or without quotes, and the quotes can be single or double quotes. In addition there can be whitespace after the opening parenthesis and before the closing parenthesis. */ static char * get_uri_string (const char *at, int *pos, int *length) { if (*length < 4) return NULL; if (0 != strncasecmp (at + *pos, "url(", 4)) return NULL; *pos += 4; *length -= 5; /* url() */ /* skip leading space */ while (*length > 0 && isspace (at[*pos])) { (*pos)++; if (--(*length) == 0) return NULL; } /* skip trailing space */ while (*length > 0 && isspace (at[*pos + *length - 1])) { (*length)--; } /* trim off quotes */ if (*length >= 2 && (at[*pos] == '\'' || at[*pos] == '"')) { (*pos)++; *length -= 2; } if (*length <= 0) return NULL; return xstrndup (at + *pos, *length); } void get_urls_css (struct map_context *ctx, int offset, int buf_length) { int token; /*char tmp[2048];*/ int buffer_pos = 0; int pos, length; char *uri; YY_BUFFER_STATE b; /* tell flex to scan from this buffer */ b = yy_scan_bytes (ctx->text + offset, buf_length); while((token = yylex()) != CSSEOF) { /*DEBUGP (("%s ", token_names[token]));*/ /* @import "foo.css" or @import url(foo.css) */ if(token == IMPORT_SYM) { do { buffer_pos += yyleng; } while((token = yylex()) == S); /*DEBUGP (("%s ", token_names[token]));*/ if (token == STRING || token == URI) { /*DEBUGP (("Got URI "));*/ pos = buffer_pos + offset; length = yyleng; if (token == URI) { uri = get_uri_string (ctx->text, &pos, &length); } else if (length >= 2) { /* cut out quote characters */ pos++; length -= 2; uri = xmalloc (length + 1); memcpy (uri, yytext + 1, length); uri[length] = '\0'; } else uri = NULL; if (uri) { struct urlpos *up = append_url (uri, pos, length, ctx); DEBUGP (("Found @import: [%s] at %d [%s]\n", yytext, buffer_pos, uri)); if (up) { up->link_inline_p = 1; up->link_css_p = 1; up->link_expect_css = 1; } xfree(uri); } } } /* background-image: url(foo.png) note that we don't care what property this is actually on. */ else if(token == URI) { pos = buffer_pos + offset; length = yyleng; uri = get_uri_string (ctx->text, &pos, &length); if (uri) { struct urlpos *up = append_url (uri, pos, length, ctx); DEBUGP (("Found URI: [%s] at %d [%s]\n", yytext, buffer_pos, uri)); if (up) { up->link_inline_p = 1; up->link_css_p = 1; } xfree (uri); } } buffer_pos += yyleng; } yy_delete_buffer(b); yylex_destroy(); DEBUGP (("\n")); } struct urlpos * get_urls_css_file (const char *file, const char *url) { struct file_memory *fm; struct map_context ctx; /* Load the file. */ fm = wget_read_file (file); if (!fm) { logprintf (LOG_NOTQUIET, "%s: %s\n", file, strerror (errno)); return NULL; } DEBUGP (("Loaded %s (size %s).\n", file, number_to_static_string (fm->length))); ctx.text = fm->content; ctx.head = NULL; ctx.base = NULL; ctx.parent_base = url ? url : opt.base_href; ctx.document_file = file; ctx.nofollow = 0; get_urls_css (&ctx, 0, fm->length); wget_read_file_free (fm); return ctx.head; } wget-1.21.2/src/mswindows.h0000644000000000000000000000437614115732710012437 00000000000000/* Declarations for windows Copyright (C) 1996-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #ifndef MSWINDOWS_H #define MSWINDOWS_H #ifndef WGET_H # error This file should not be included directly. #endif /* Prevent inclusion of in . */ #ifndef WIN32_LEAN_AND_MEAN # define WIN32_LEAN_AND_MEAN #endif #include #include #include #ifndef EAI_SYSTEM # define EAI_SYSTEM -1 /* value doesn't matter */ #endif /* Declares file access functions, such as open, create, access, and chmod. Unix declares these in unistd.h and fcntl.h. */ #include /* Declares getpid(). */ #include /* Declares inet_ntop() and inet_pton(). */ #include #include #define PATH_SEPARATOR '\\' /* ioctl needed by set_windows_fd_as_blocking_socket() */ #include /* Public functions. */ void ws_startup (void); void ws_changetitle (const char *); void ws_percenttitle (double); char *ws_mypath (void); void windows_main (char **); void set_windows_fd_as_blocking_socket (int); #endif /* MSWINDOWS_H */ wget-1.21.2/src/iri.h0000644000000000000000000000536314115732710011165 00000000000000/* Internationalization related declarations. Copyright (C) 2008-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #ifndef IRI_H #define IRI_H struct iri { char *uri_encoding; /* Encoding of the uri to fetch */ char *content_encoding; /* Encoding of links inside the fetched file */ char *orig_url; /* */ bool utf8_encode; /* Will/Is the current url encoded in utf8 */ }; #ifdef ENABLE_IRI char *parse_charset (const char *str); const char *find_locale (void); bool check_encoding_name (const char *encoding); const char *locale_to_utf8 (const char *str); char *idn_encode (const struct iri *i, const char *host); char *idn_decode (const char *host); bool remote_to_utf8 (const struct iri *i, const char *str, char **newstr); struct iri *iri_new (void); struct iri *iri_dup (const struct iri *); void iri_free (struct iri *i); void set_uri_encoding (struct iri *i, const char *charset, bool force); void set_content_encoding (struct iri *i, const char *charset); #else /* ENABLE_IRI */ extern struct iri dummy_iri; #define parse_charset(str) NULL #define find_locale() NULL #define check_encoding_name(str) false #define locale_to_utf8(str) (str) #define idn_encode(a,b) NULL #define idn_decode(str) NULL #define idn2_free(str) ((void)0) #define remote_to_utf8(a,b,c) false #define iri_new() (&dummy_iri) #define iri_dup(a) (&dummy_iri) #define iri_free(a) #define set_uri_encoding(a,b,c) #define set_content_encoding(a,b) #endif /* ENABLE_IRI */ #endif /* IRI_H */ wget-1.21.2/src/warc.h0000644000000000000000000000220514115732710011326 00000000000000/* Declarations of WARC helper methods. */ #ifndef WARC_H #define WARC_H #include "host.h" void warc_init (void); void warc_close (void); void warc_uuid_str (char *id_str, size_t urn_size); char * warc_timestamp (char *timestamp, size_t timestamp_size); FILE * warc_tempfile (void); bool warc_write_request_record (const char *url, const char *timestamp_str, const char *concurrent_to_uuid, const ip_address *ip, FILE *body, off_t payload_offset); bool warc_write_response_record (const char *url, const char *timestamp_str, const char *concurrent_to_uuid, const ip_address *ip, FILE *body, off_t payload_offset, const char *mime_type, int response_code, const char *redirect_location); bool warc_write_resource_record (const char *resource_uuid, const char *url, const char *timestamp_str, const char *concurrent_to_uuid, const ip_address *ip, const char *content_type, FILE *body, off_t payload_offset); bool warc_write_metadata_record (const char *record_uuid, const char *url, const char *timestamp_str, const char *concurrent_to_uuid, ip_address *ip, const char *content_type, FILE *body, off_t payload_offset); #endif /* WARC_H */ wget-1.21.2/src/log.c0000644000000000000000000007141114115732710011153 00000000000000/* Messages logging. Copyright (C) 1998-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #include "wget.h" #include #include #include #include #include #include #include #include "utils.h" #include "exits.h" #include "log.h" /* 2005-10-25 SMS. VMS log files are often VFC record format, not stream, so fputs() can produce multiple records, even when there's no newline terminator in the buffer. The result is unsightly output with spurious newlines. Using fprintf() instead of fputs(), along with inhibiting some fflush() activity below, seems to solve the problem. */ #ifdef __VMS # define FPUTS( s, f) fprintf( (f), "%s", (s)) #else /* def __VMS */ # define FPUTS( s, f) fputs( (s), (f)) #endif /* def __VMS [else] */ /* This file implements support for "logging". Logging means printing output, plus several additional features: - Cataloguing output by importance. You can specify that a log message is "verbose" or "debug", and it will not be printed unless in verbose or debug mode, respectively. - Redirecting the log to the file. When Wget's output goes to the terminal, and Wget receives SIGHUP, all further output is redirected to a log file. When this is the case, Wget can also print the last several lines of "context" to the log file so that it does not begin in the middle of a line. For this to work, the logging code stores the last several lines of context. Callers may request for certain output not to be stored. - Inhibiting output. When Wget receives SIGHUP, but redirecting the output fails, logging is inhibited. */ /* The file descriptor used for logging. This is NULL before log_init is called; logging functions log to stderr then. log_init sets it either to stderr or to a file pointer obtained from fopen(). If logging is inhibited, logfp is set back to NULL. */ static FILE *logfp; /* Descriptor of the stdout|stderr */ static FILE *stdlogfp; /* Descriptor of the wget.log* file (if created) */ static FILE *filelogfp; /* Name of log file */ static char *logfile; /* Is interactive shell ? */ static int shell_is_interactive; /* A second file descriptor pointing to the temporary log file for the WARC writer. If WARC writing is disabled, this is NULL. */ static FILE *warclogfp; /* If true, it means logging is inhibited, i.e. nothing is printed or stored. */ static bool inhibit_logging; /* Whether the last output lines are stored for use as context. */ static bool save_context_p; /* Whether the log is flushed after each command. */ static bool flush_log_p = true; /* Whether any output has been received while flush_log_p was 0. */ static bool needs_flushing; /* In the event of a hang-up, and if its output was on a TTY, Wget redirects its output to `wget-log'. For the convenience of reading this newly-created log, we store the last several lines ("screenful", hence the choice of 24) of Wget output, and dump them as context when the time comes. */ #define SAVED_LOG_LINES 24 /* log_lines is a circular buffer that stores SAVED_LOG_LINES lines of output. log_line_current always points to the position in the buffer that will be written to next. When log_line_current reaches SAVED_LOG_LINES, it is reset to zero. The problem here is that we'd have to either (re)allocate and free strings all the time, or limit the lines to an arbitrary number of characters. Instead of settling for either of these, we do both: if the line is smaller than a certain "usual" line length (128 chars by default), a preallocated memory is used. The rare lines that are longer than 128 characters are malloc'ed and freed separately. This gives good performance with minimum memory consumption and fragmentation. */ #define STATIC_LENGTH 128 static struct log_ln { char static_line[STATIC_LENGTH + 1]; /* statically allocated line. */ char *malloced_line; /* malloc'ed line, for lines of output larger than 80 characters. */ char *content; /* this points either to malloced_line or to the appropriate static_line. If this is NULL, it means the line has not yet been used. */ } log_lines[SAVED_LOG_LINES]; /* The current position in the ring. */ static int log_line_current = -1; /* Whether the most recently written line was "trailing", i.e. did not finish with \n. This is an important piece of information because the code is always careful to append data to trailing lines, rather than create new ones. */ static bool trailing_line; static void check_redirect_output (void); #define ROT_ADVANCE(num) do { \ if (++num >= SAVED_LOG_LINES) \ num = 0; \ } while (0) /* Free the log line index with NUM. This calls free on ln->malloced_line if it's non-NULL, and it also resets ln->malloced_line and ln->content to NULL. */ static void free_log_line (int num) { struct log_ln *ln = log_lines + num; xfree (ln->malloced_line); ln->content = NULL; } /* Append bytes in the range [start, end) to one line in the log. The region is not supposed to contain newlines, except for the last character (at end[-1]). */ static void saved_append_1 (const char *start, const char *end) { int len = end - start; if (!len) return; /* First, check whether we need to append to an existing line or to create a new one. */ if (!trailing_line) { /* Create a new line. */ struct log_ln *ln; if (log_line_current == -1) log_line_current = 0; else free_log_line (log_line_current); ln = log_lines + log_line_current; if (len > STATIC_LENGTH) { ln->malloced_line = strdupdelim (start, end); ln->content = ln->malloced_line; } else { memcpy (ln->static_line, start, len); ln->static_line[len] = '\0'; ln->content = ln->static_line; } } else { /* Append to the last line. If the line is malloc'ed, we just call realloc and append the new string. If the line is static, we have to check whether appending the new string would make it exceed STATIC_LENGTH characters, and if so, convert it to malloc(). */ struct log_ln *ln = log_lines + log_line_current; if (ln->malloced_line) { /* Resize malloc'ed line and append. */ int old_len = strlen (ln->malloced_line); ln->malloced_line = xrealloc (ln->malloced_line, old_len + len + 1); memcpy (ln->malloced_line + old_len, start, len); ln->malloced_line[old_len + len] = '\0'; /* might have changed due to realloc */ ln->content = ln->malloced_line; } else { int old_len = strlen (ln->static_line); if (old_len + len > STATIC_LENGTH) { /* Allocate memory and concatenate the old and the new contents. */ ln->malloced_line = xmalloc (old_len + len + 1); memcpy (ln->malloced_line, ln->static_line, old_len); memcpy (ln->malloced_line + old_len, start, len); ln->malloced_line[old_len + len] = '\0'; ln->content = ln->malloced_line; } else { /* Just append to the old, statically allocated contents. */ memcpy (ln->static_line + old_len, start, len); ln->static_line[old_len + len] = '\0'; ln->content = ln->static_line; } } } trailing_line = !(end[-1] == '\n'); if (!trailing_line) ROT_ADVANCE (log_line_current); } /* Log the contents of S, as explained above. If S consists of multiple lines, they are logged separately. If S does not end with a newline, it will form a "trailing" line, to which things will get appended the next time this function is called. */ static void saved_append (const char *s) { while (*s) { const char *end = strchr (s, '\n'); if (!end) end = s + strlen (s); else ++end; saved_append_1 (s, end); s = end; } } /* Check X against opt.verbose and opt.quiet. The semantics is as follows: * LOG_ALWAYS - print the message unconditionally; * LOG_NOTQUIET - print the message if opt.quiet is non-zero; * LOG_NONVERBOSE - print the message if opt.verbose is zero; * LOG_VERBOSE - print the message if opt.verbose is non-zero. */ #define CHECK_VERBOSE(x) \ switch (x) \ { \ case LOG_PROGRESS: \ if (!opt.show_progress) \ return; \ break; \ case LOG_ALWAYS: \ break; \ case LOG_NOTQUIET: \ if (opt.quiet) \ return; \ break; \ case LOG_NONVERBOSE: \ if (opt.verbose || opt.quiet) \ return; \ break; \ case LOG_VERBOSE: \ if (!opt.verbose) \ return; \ } /* Returns the file descriptor for logging. This is LOGFP, except if called before log_init, in which case it returns stderr. This is useful in case someone calls a logging function before log_init. If logging is inhibited, return NULL. */ static FILE * get_log_fp (void) { if (inhibit_logging) return NULL; if (logfp) return logfp; return stderr; } static FILE * get_progress_fp (void) { if (opt.show_progress == true) return stderr; return get_log_fp(); } /* Returns the file descriptor for the secondary log file. This is WARCLOGFP, except if called before log_init, in which case it returns stderr. This is useful in case someone calls a logging function before log_init. If logging is inhibited, return NULL. */ static FILE * get_warc_log_fp (void) { if (inhibit_logging) return NULL; if (warclogfp) return warclogfp; if (logfp) return NULL; return stderr; } /* Sets the file descriptor for the secondary log file. */ void log_set_warc_log_fp (FILE * fp) { warclogfp = fp; } /* Log a literal string S. The string is logged as-is, without a newline appended. */ void logputs (enum log_options o, const char *s) { FILE *fp; FILE *warcfp; int errno_save = errno; check_redirect_output (); if (o == LOG_PROGRESS) fp = get_progress_fp (); else fp = get_log_fp (); errno = errno_save; if (fp == NULL) return; warcfp = get_warc_log_fp (); errno = errno_save; CHECK_VERBOSE (o); FPUTS (s, fp); if (warcfp != NULL) FPUTS (s, warcfp); if (save_context_p) saved_append (s); if (flush_log_p) logflush (); else needs_flushing = true; errno = errno_save; } struct logvprintf_state { char *bigmsg; int expected_size; int allocated; }; /* Print a message to the log. A copy of message will be saved to saved_log, for later reusal by log_dump_context(). Normally we'd want this function to loop around vsnprintf until sufficient room is allocated, as the Linux man page recommends. However each call to vsnprintf() must be preceded by va_start and followed by va_end. Since calling va_start/va_end is possible only in the function that contains the `...' declaration, we cannot call vsnprintf more than once. Therefore this function saves its state to logvprintf_state and signals the parent to call it again. (An alternative approach would be to use va_copy, but that's not portable.) */ static bool GCC_FORMAT_ATTR (2, 0) log_vprintf_internal (struct logvprintf_state *state, const char *fmt, va_list args) { char smallmsg[128]; char *write_ptr = smallmsg; int available_size = sizeof (smallmsg); int numwritten; FILE *fp = get_log_fp (); FILE *warcfp = get_warc_log_fp (); if (fp == NULL) return false; if (!save_context_p && warcfp == NULL) { /* In the simple case just call vfprintf(), to avoid needless allocation and games with vsnprintf(). */ vfprintf (fp, fmt, args); goto flush; } if (state->allocated != 0) { write_ptr = state->bigmsg; available_size = state->allocated; } /* The GNU coding standards advise not to rely on the return value of sprintf(). However, vsnprintf() is a relatively new function missing from legacy systems. Therefore I consider it safe to assume that its return value is meaningful. On the systems where vsnprintf() is not available, we use the implementation from snprintf.c which does return the correct value. */ numwritten = vsnprintf (write_ptr, available_size, fmt, args); /* vsnprintf() will not step over the limit given by available_size. If it fails, it returns either -1 (older implementations) or the number of characters (not counting the terminating \0) that *would have* been written if there had been enough room (C99). In the former case, we double available_size and malloc to get a larger buffer, and try again. In the latter case, we use the returned information to build a buffer of the correct size. */ if (numwritten == -1) { /* Writing failed, and we don't know the needed size. Try again with doubled size. */ int newsize = available_size << 1; state->bigmsg = xrealloc (state->bigmsg, newsize); state->allocated = newsize; return false; } else if (numwritten >= available_size) { /* Writing failed, but we know exactly how much space we need. */ int newsize = numwritten + 1; state->bigmsg = xrealloc (state->bigmsg, newsize); state->allocated = newsize; return false; } /* Writing succeeded. */ if (save_context_p) saved_append (write_ptr); FPUTS (write_ptr, fp); if (warcfp != NULL && warcfp != fp) FPUTS (write_ptr, warcfp); xfree (state->bigmsg); flush: if (flush_log_p) logflush (); else needs_flushing = true; return true; } /* Flush LOGFP. Useful while flushing is disabled. */ void logflush (void) { FILE *fp = get_log_fp (); FILE *warcfp = get_warc_log_fp (); if (fp) { /* 2005-10-25 SMS. On VMS, flush only for a terminal. See note at FPUTS macro, above. */ #ifdef __VMS if (isatty( fileno( fp))) { fflush (fp); } #else /* def __VMS */ fflush (fp); #endif /* def __VMS [else] */ } if (warcfp != NULL) fflush (warcfp); needs_flushing = false; } /* Enable or disable log flushing. */ void log_set_flush (bool flush) { if (flush == flush_log_p) return; if (flush == false) { /* Disable flushing by setting flush_log_p to 0. */ flush_log_p = false; } else { /* Re-enable flushing. If anything was printed in no-flush mode, flush the log now. */ if (needs_flushing) logflush (); flush_log_p = true; } } /* (Temporarily) disable storing log to memory. Returns the old status of storing, with which this function can be called again to reestablish storing. */ bool log_set_save_context (bool savep) { bool old = save_context_p; save_context_p = savep; return old; } /* Print a message to the screen or to the log. The first argument defines the verbosity of the message, and the rest are as in printf(3). */ void logprintf (enum log_options o, const char *fmt, ...) { va_list args; struct logvprintf_state lpstate; bool done; int errno_saved = errno; check_redirect_output (); errno = errno_saved; if (inhibit_logging) return; CHECK_VERBOSE (o); xzero (lpstate); errno = 0; do { va_start (args, fmt); done = log_vprintf_internal (&lpstate, fmt, args); va_end (args); if (done && errno == EPIPE) exit (WGET_EXIT_GENERIC_ERROR); } while (!done); errno = errno_saved; } #ifdef ENABLE_DEBUG /* The same as logprintf(), but does anything only if opt.debug is true. */ void debug_logprintf (const char *fmt, ...) { if (opt.debug) { va_list args; struct logvprintf_state lpstate; bool done; #ifndef TESTING check_redirect_output (); #endif if (inhibit_logging) return; xzero (lpstate); do { va_start (args, fmt); done = log_vprintf_internal (&lpstate, fmt, args); va_end (args); } while (!done); } } #endif /* ENABLE_DEBUG */ /* Open FILE and set up a logging stream. If FILE cannot be opened, exit with status of 1. */ void log_init (const char *file, bool appendp) { if (file) { if (HYPHENP (file)) { stdlogfp = stdout; logfp = stdlogfp; } else { filelogfp = fopen (file, appendp ? "a" : "w"); if (!filelogfp) { fprintf (stderr, "%s: %s: %s\n", exec_name, file, strerror (errno)); exit (WGET_EXIT_GENERIC_ERROR); } logfp = filelogfp; } } else { /* The log goes to stderr to avoid collisions with the output if the user specifies `-O -'. #### Francois Pinard suggests that it's a better idea to print to stdout by default, and to stderr only if the user actually specifies `-O -'. He says this inconsistency is harder to document, but is overall easier on the user. */ stdlogfp = stderr; logfp = stdlogfp; if (1 #ifdef HAVE_ISATTY && isatty (fileno (logfp)) #endif ) { /* If the output is a TTY, enable save context, i.e. store the most recent several messages ("context") and dump them to a log file in case SIGHUP or SIGUSR1 is received (or Ctrl+Break is pressed under Windows). */ save_context_p = true; } } #ifndef WINDOWS /* Initialize this values so we don't have to ask every time we print line */ shell_is_interactive = isatty (STDIN_FILENO); #endif } /* Close LOGFP (only if we opened it, not if it's stderr), inhibit further logging and free the memory associated with it. */ void log_close (void) { int i; if (logfp && logfp != stderr && logfp != stdout) { if (logfp == stdlogfp) stdlogfp = NULL; if (logfp == filelogfp) filelogfp = NULL; fclose (logfp); } logfp = NULL; inhibit_logging = true; save_context_p = false; for (i = 0; i < SAVED_LOG_LINES; i++) free_log_line (i); log_line_current = -1; trailing_line = false; } /* Dump saved lines to logfp. */ static void log_dump_context (void) { int num = log_line_current; FILE *fp = get_log_fp (); FILE *warcfp = get_warc_log_fp (); if (!fp) return; if (num == -1) return; if (trailing_line) ROT_ADVANCE (num); do { struct log_ln *ln = log_lines + num; if (ln->content) { FPUTS (ln->content, fp); if (warcfp != NULL) FPUTS (ln->content, warcfp); } ROT_ADVANCE (num); } while (num != log_line_current); if (trailing_line) if (log_lines[log_line_current].content) { FPUTS (log_lines[log_line_current].content, fp); if (warcfp != NULL) FPUTS (log_lines[log_line_current].content, warcfp); } fflush (fp); fflush (warcfp); } /* String escape functions. */ /* Return the number of non-printable characters in SOURCE. Non-printable characters are determined as per c-ctype.c. */ static int count_nonprint (const char *source) { const char *p; int cnt; for (p = source, cnt = 0; *p; p++) if (!c_isprint (*p)) ++cnt; return cnt; } /* Copy SOURCE to DEST, escaping non-printable characters. Non-printable refers to anything outside the non-control ASCII range (32-126) which means that, for example, CR, LF, and TAB are considered non-printable along with ESC, BS, and other control chars. This is by design: it makes sure that messages from remote servers cannot be easily used to deceive the users by mimicking Wget's output. Disallowing non-ASCII characters is another necessary security measure, which makes sure that remote servers cannot garble the screen or guess the local charset and perform homographic attacks. Of course, the above mandates that escnonprint only be used in contexts expected to be ASCII, such as when printing host names, URL components, HTTP headers, FTP server messages, and the like. ESCAPE is the leading character of the escape sequence. BASE should be the base of the escape sequence, and must be either 8 for octal or 16 for hex. DEST must point to a location with sufficient room to store an encoded version of SOURCE. */ static void copy_and_escape (const char *source, char *dest, char escape, int base) { const char *from = source; char *to = dest; unsigned char c; /* Copy chars from SOURCE to DEST, escaping non-printable ones. */ switch (base) { case 8: while ((c = *from++) != '\0') if (c_isprint (c)) *to++ = c; else { *to++ = escape; *to++ = '0' + (c >> 6); *to++ = '0' + ((c >> 3) & 7); *to++ = '0' + (c & 7); } break; case 16: while ((c = *from++) != '\0') if (c_isprint (c)) *to++ = c; else { *to++ = escape; *to++ = XNUM_TO_DIGIT (c >> 4); *to++ = XNUM_TO_DIGIT (c & 0xf); } break; default: abort (); } *to = '\0'; } #define RING_SIZE 3 struct ringel { char *buffer; int size; }; static struct ringel ring[RING_SIZE]; /* ring data */ static const char * escnonprint_internal (const char *str, char escape, int base) { static int ringpos; /* current ring position */ int nprcnt; assert (base == 8 || base == 16); nprcnt = count_nonprint (str); if (nprcnt == 0) /* If there are no non-printable chars in STR, don't bother copying anything, just return STR. */ return str; { /* Set up a pointer to the current ring position, so we can write simply r->X instead of ring[ringpos].X. */ struct ringel *r = ring + ringpos; /* Every non-printable character is replaced with the escape char and three (or two, depending on BASE) *additional* chars. Size must also include the length of the original string and one additional char for the terminating \0. */ int needed_size = strlen (str) + 1 + (base == 8 ? 3 * nprcnt : 2 * nprcnt); /* If the current buffer is uninitialized or too small, (re)allocate it. */ if (r->buffer == NULL || r->size < needed_size) { r->buffer = xrealloc (r->buffer, needed_size); r->size = needed_size; } copy_and_escape (str, r->buffer, escape, base); ringpos = (ringpos + 1) % RING_SIZE; return r->buffer; } } /* Return a pointer to a static copy of STR with the non-printable characters escaped as \ooo. If there are no non-printable characters in STR, STR is returned. See copy_and_escape for more information on which characters are considered non-printable. DON'T call this function on translated strings because escaping will break them. Don't call it on literal strings from the source, which are by definition trusted. If newlines are allowed in the string, escape and print it line by line because escaping the whole string will convert newlines to \012. (This is so that expectedly single-line messages cannot use embedded newlines to mimic Wget's output and deceive the user.) escnonprint doesn't quote its escape character because it is notf meant as a general and reversible quoting mechanism, but as a quick way to defang binary junk sent by malicious or buggy servers. NOTE: since this function can return a pointer to static data, be careful to copy its result before calling it again. However, to be more useful with printf, it maintains an internal ring of static buffers to return. Currently the ring size is 3, which means you can print up to three values in the same printf; if more is needed, bump RING_SIZE. */ const char * escnonprint (const char *str) { return escnonprint_internal (str, '\\', 8); } /* Return a pointer to a static copy of STR with the non-printable characters escaped as %XX. If there are no non-printable characters in STR, STR is returned. See escnonprint for usage details. */ const char * escnonprint_uri (const char *str) { return escnonprint_internal (str, '%', 16); } #if defined DEBUG_MALLOC || defined TESTING void log_cleanup (void) { size_t i; for (i = 0; i < countof (ring); i++) xfree (ring[i].buffer); } #endif /* When SIGHUP or SIGUSR1 are received, the output is redirected elsewhere. Such redirection is only allowed once. */ static const char *redirect_request_signal_name; /* Redirect output to `wget-log' or back to stdout/stderr. */ void redirect_output (bool to_file, const char *signal_name) { if (to_file && logfp != filelogfp) { if (signal_name) { fprintf (stderr, "\n%s received.", signal_name); } if (!filelogfp) { filelogfp = unique_create (DEFAULT_LOGFILE, false, &logfile); if (filelogfp) { fprintf (stderr, _("\nRedirecting output to %s.\n"), quote (logfile)); /* Store signal name to tell wget it's permanent redirect to log file */ redirect_request_signal_name = signal_name; logfp = filelogfp; /* Dump the context output to the newly opened log. */ log_dump_context (); } else { /* Eek! Opening the alternate log file has failed. Nothing we can do but disable printing completely. */ fprintf (stderr, _("%s: %s; disabling logging.\n"), (logfile) ? logfile : DEFAULT_LOGFILE, strerror (errno)); inhibit_logging = true; } } else { fprintf (stderr, _("\nRedirecting output to %s.\n"), quote (logfile)); logfp = filelogfp; log_dump_context (); } } else if (!to_file && logfp != stdlogfp) { logfp = stdlogfp; log_dump_context (); } } /* Check whether there's a need to redirect output. */ static void check_redirect_output (void) { #if !defined(WINDOWS) && !defined(__VMS) /* If it was redirected already to log file by SIGHUP, SIGUSR1 or -o parameter, * it was permanent. * If there was no SIGHUP or SIGUSR1 and shell is interactive * we check if process is fg or bg before every line is printed.*/ if (!redirect_request_signal_name && shell_is_interactive && !opt.lfilename) { pid_t foreground_pgrp = tcgetpgrp (STDIN_FILENO); if (foreground_pgrp != -1 && foreground_pgrp != getpgrp () && !opt.quiet) { /* Process backgrounded */ redirect_output (true,NULL); } else { /* Process foregrounded */ redirect_output (false,NULL); } } #endif /* !defined(WINDOWS) && !defined(__VMS) */ } wget-1.21.2/src/hsts.c0000644000000000000000000005535114115732710011360 00000000000000/* HTTP Strict Transport Security (HSTS) support. Copyright (C) 1996-2012, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #include "wget.h" #ifdef HAVE_HSTS #include "hsts.h" #include "utils.h" #include "host.h" /* for is_valid_ip_address() */ #include "hash.h" #include "c-ctype.h" #ifdef TESTING #include "init.h" /* for ajoin_dir_file() */ #include "../tests/unit-tests.h" #endif #include #include #include #include #include #include #include #include struct hsts_store { struct hash_table *table; time_t last_mtime; bool changed; }; struct hsts_kh { char *host; int explicit_port; }; struct hsts_kh_info { time_t created; time_t max_age; bool include_subdomains; }; enum hsts_kh_match { NO_MATCH, SUPERDOMAIN_MATCH, CONGRUENT_MATCH }; #define hsts_is_host_name_valid(host) (!is_valid_ip_address (host)) #define hsts_is_scheme_valid(scheme) (scheme == SCHEME_HTTPS) #define hsts_is_host_eligible(scheme, host) \ (hsts_is_scheme_valid (scheme) && hsts_is_host_name_valid (host)) #define DEFAULT_HTTP_PORT 80 #define DEFAULT_SSL_PORT 443 #define MAKE_EXPLICIT_PORT(s, p) (s == SCHEME_HTTPS ? (p == DEFAULT_SSL_PORT ? 0 : p) \ : (p == DEFAULT_HTTP_PORT ? 0 : p)) /* Hashing and comparison functions for the hash table */ #ifdef __clang__ __attribute__((no_sanitize("integer"))) #endif static unsigned long hsts_hash_func (const void *key) { struct hsts_kh *k = (struct hsts_kh *) key; const char *h = NULL; unsigned int hash = k->explicit_port; for (h = k->host; *h; h++) hash = hash * 31 + *h; return hash; } static int hsts_cmp_func (const void *h1, const void *h2) { struct hsts_kh *kh1 = (struct hsts_kh *) h1, *kh2 = (struct hsts_kh *) h2; return (!strcmp (kh1->host, kh2->host)) && (kh1->explicit_port == kh2->explicit_port); } /* Private functions. Feel free to make some of these public when needed. */ static struct hsts_kh_info * hsts_find_entry (hsts_store_t store, const char *host, int explicit_port, enum hsts_kh_match *match_type, struct hsts_kh *kh) { struct hsts_kh *k = NULL; struct hsts_kh_info *khi = NULL; enum hsts_kh_match match = NO_MATCH; char *pos = NULL; char *org_ptr = NULL; k = (struct hsts_kh *) xnew (struct hsts_kh); k->host = xstrdup_lower (host); k->explicit_port = explicit_port; /* save pointer so that we don't get into trouble later when freeing */ org_ptr = k->host; khi = (struct hsts_kh_info *) hash_table_get (store->table, k); if (khi) { match = CONGRUENT_MATCH; goto end; } while (match == NO_MATCH && (pos = strchr (k->host, '.')) && pos - k->host > 0 && strchr (pos + 1, '.')) { k->host += (pos - k->host + 1); khi = (struct hsts_kh_info *) hash_table_get (store->table, k); if (khi) match = SUPERDOMAIN_MATCH; } end: /* restore pointer or we'll get a SEGV */ k->host = org_ptr; /* copy parameters to previous frame */ if (match_type) *match_type = match; if (kh) memcpy (kh, k, sizeof (struct hsts_kh)); else xfree (k->host); xfree (k); return khi; } static bool hsts_new_entry_internal (hsts_store_t store, const char *host, int port, time_t created, time_t max_age, bool include_subdomains, bool check_validity, bool check_expired, bool check_duplicates) { struct hsts_kh *kh = xnew (struct hsts_kh); struct hsts_kh_info *khi = xnew0 (struct hsts_kh_info); bool success = false; kh->host = xstrdup_lower (host); kh->explicit_port = MAKE_EXPLICIT_PORT (SCHEME_HTTPS, port); khi->created = created; khi->max_age = max_age; khi->include_subdomains = include_subdomains; /* Check validity */ if (check_validity && !hsts_is_host_name_valid (host)) goto bail; if (check_expired && ((khi->created + khi->max_age) < khi->created)) goto bail; if (check_duplicates && hash_table_contains (store->table, kh)) goto bail; /* Now store the new entry */ hash_table_put (store->table, kh, khi); success = true; bail: if (!success) { /* abort! */ xfree (kh->host); xfree (kh); xfree (khi); } return success; } /* Creates a new entry, but does not check whether that entry already exists. This function assumes that check has already been done by the caller. */ static bool hsts_add_entry (hsts_store_t store, const char *host, int port, time_t max_age, bool include_subdomains) { time_t t = time (NULL); /* It might happen time() returned -1 */ return (t == (time_t)(-1) ? false : hsts_new_entry_internal (store, host, port, t, max_age, include_subdomains, false, true, false)); } /* Creates a new entry, unless an identical one already exists. */ static bool hsts_new_entry (hsts_store_t store, const char *host, int port, time_t created, time_t max_age, bool include_subdomains) { return hsts_new_entry_internal (store, host, port, created, max_age, include_subdomains, true, true, true); } static void hsts_remove_entry (hsts_store_t store, struct hsts_kh *kh) { hash_table_remove (store->table, kh); } static bool hsts_store_merge (hsts_store_t store, const char *host, int port, time_t created, time_t max_age, bool include_subdomains) { enum hsts_kh_match match_type = NO_MATCH; struct hsts_kh_info *khi = NULL; bool success = false; port = MAKE_EXPLICIT_PORT (SCHEME_HTTPS, port); khi = hsts_find_entry (store, host, port, &match_type, NULL); if (khi && match_type == CONGRUENT_MATCH && created > khi->created) { /* update the entry with the new info */ khi->created = created; khi->max_age = max_age; khi->include_subdomains = include_subdomains; success = true; } else if (!khi) success = hsts_new_entry (store, host, port, created, max_age, include_subdomains); return success; } static bool hsts_read_database (hsts_store_t store, FILE *fp, bool merge_with_existing_entries) { char *line = NULL, *p; size_t len = 0; int items_read; bool result = false; bool (*func)(hsts_store_t, const char *, int, time_t, time_t, bool); char host[256]; int port; time_t created, max_age; int include_subdomains; func = (merge_with_existing_entries ? hsts_store_merge : hsts_new_entry); while (getline (&line, &len, fp) > 0) { for (p = line; c_isspace (*p); p++) ; if (*p == '#') continue; items_read = sscanf (p, "%255s %d %d %lu %lu", host, &port, &include_subdomains, (unsigned long *) &created, (unsigned long *) &max_age); if (items_read == 5) func (store, host, port, created, max_age, !!include_subdomains); } xfree (line); result = true; return result; } static void hsts_store_dump (hsts_store_t store, FILE *fp) { hash_table_iterator it; /* Print preliminary comments. We don't care if any of these fail. */ fputs ("# HSTS 1.0 Known Hosts database for GNU Wget.\n", fp); fputs ("# Edit at your own risk.\n", fp); fputs ("# \t\t\t\t\n", fp); /* Now cycle through the HSTS store in memory and dump the entries */ for (hash_table_iterate (store->table, &it); hash_table_iter_next (&it);) { struct hsts_kh *kh = (struct hsts_kh *) it.key; struct hsts_kh_info *khi = (struct hsts_kh_info *) it.value; if (fprintf (fp, "%s\t%d\t%d\t%lu\t%lu\n", kh->host, kh->explicit_port, khi->include_subdomains, (unsigned long) khi->created, (unsigned long) khi->max_age) < 0) { logprintf (LOG_ALWAYS, "Could not write the HSTS database correctly.\n"); break; } } } /* * Test: * - The file is a regular file (ie. not a symlink), and * - The file is not world-writable. */ static bool hsts_file_access_valid (const char *filename) { struct stat st; if (stat (filename, &st) == -1) return false; return #ifndef WINDOWS /* * The world-writable concept is a Unix-centric notion. * We bypass this test on Windows. */ !(st.st_mode & S_IWOTH) && #endif S_ISREG (st.st_mode); } /* HSTS API */ /* Changes the given URLs according to the HSTS policy. If there's no host in the store that either congruently or not, matches the given URL, no changes are made. Returns true if the URL was changed, or false if it was left intact. */ bool hsts_match (hsts_store_t store, struct url *u) { bool url_changed = false; struct hsts_kh_info *entry = NULL; struct hsts_kh *kh = xnew(struct hsts_kh); enum hsts_kh_match match = NO_MATCH; int port = MAKE_EXPLICIT_PORT (u->scheme, u->port); /* avoid doing any computation if we're already in HTTPS */ if (!hsts_is_scheme_valid (u->scheme)) { entry = hsts_find_entry (store, u->host, port, &match, kh); if (entry) { if ((entry->created + entry->max_age) >= time(NULL)) { if ((match == CONGRUENT_MATCH) || (match == SUPERDOMAIN_MATCH && entry->include_subdomains)) { /* we found a matching Known HSTS Host rewrite the URL */ u->scheme = SCHEME_HTTPS; if (u->port == 80) u->port = 443; url_changed = true; store->changed = true; } } else { hsts_remove_entry (store, kh); store->changed = true; } } xfree (kh->host); } xfree (kh); return url_changed; } /* Add a new HSTS Known Host to the HSTS store. If the host already exists, its information is updated, or it'll be removed from the store if max_age is zero. Bear in mind that the store is kept in memory, and will not be written to disk until hsts_store_save is called. This function regrows the in-memory HSTS store if necessary. Currently, for a host to be taken into consideration, two conditions have to be met: - Connection must be through a secure channel (HTTPS). - The host must not be an IPv4 or IPv6 address. The RFC 6797 states that hosts that match IPv4 or IPv6 format should be discarded at URI rewrite time. But we short-circuit that check here, since there's no point in storing a host that will never be matched. Returns true if a new entry was actually created, or false if an existing entry was updated/deleted. */ bool hsts_store_entry (hsts_store_t store, enum url_scheme scheme, const char *host, int port, time_t max_age, bool include_subdomains) { bool result = false; enum hsts_kh_match match = NO_MATCH; struct hsts_kh *kh = xnew(struct hsts_kh); struct hsts_kh_info *entry = NULL; if (hsts_is_host_eligible (scheme, host)) { port = MAKE_EXPLICIT_PORT (scheme, port); entry = hsts_find_entry (store, host, port, &match, kh); if (entry && match == CONGRUENT_MATCH) { if (max_age == 0) { hsts_remove_entry (store, kh); store->changed = true; } else if (max_age > 0) { /* RFC 6797 states that 'max_age' is a TTL relative to the * reception of the STS header so we have to update the * 'created' field too. The RFC also states that we have to * update the entry each time we see HSTS header. * See also Section 11.2. */ time_t t = time (NULL); if (t != (time_t)(-1) && t != entry->created) { entry->created = t; entry->max_age = max_age; entry->include_subdomains = include_subdomains; store->changed = true; } } /* we ignore negative max_ages */ } else if (entry == NULL || match == SUPERDOMAIN_MATCH) { /* Either we didn't find a matching host, or we got a superdomain match. In either case, we create a new entry. We have to perform an explicit check because it might happen we got a non-existent entry with max_age == 0. */ result = hsts_add_entry (store, host, port, max_age, include_subdomains); if (result) store->changed = true; } /* we ignore new entries with max_age == 0 */ xfree (kh->host); } xfree (kh); return result; } hsts_store_t hsts_store_open (const char *filename) { hsts_store_t store = NULL; file_stats_t fstats; store = xnew0 (struct hsts_store); store->table = hash_table_new (0, hsts_hash_func, hsts_cmp_func); store->last_mtime = 0; store->changed = false; if (file_exists_p (filename, &fstats)) { if (hsts_file_access_valid (filename)) { struct stat st; FILE *fp = fopen_stat (filename, "r", &fstats); if (!fp || !hsts_read_database (store, fp, false)) { /* abort! */ hsts_store_close (store); xfree (store); if (fp) fclose (fp); goto out; } if (fstat (fileno (fp), &st) == 0) store->last_mtime = st.st_mtime; fclose (fp); } else { /* * If we're not reading the HSTS database, * then by all means act as if HSTS was disabled. */ hsts_store_close (store); xfree (store); logprintf (LOG_NOTQUIET, "Will not apply HSTS. " "The HSTS database must be a regular and non-world-writable file.\n"); } } out: return store; } void hsts_store_save (hsts_store_t store, const char *filename) { struct stat st; FILE *fp = NULL; int fd = 0; if (filename && hash_table_count (store->table) > 0) { fp = fopen (filename, "a+"); if (fp) { /* Lock the file to avoid potential race conditions */ fd = fileno (fp); flock (fd, LOCK_EX); /* If the file has changed, merge the changes with our in-memory data before dumping them to the file. Otherwise we could potentially overwrite the data stored by other Wget processes. */ if (store->last_mtime && stat (filename, &st) == 0 && st.st_mtime > store->last_mtime) hsts_read_database (store, fp, true); /* We've merged the latest changes so we can now truncate the file and dump everything. */ fseek (fp, 0, SEEK_SET); ftruncate (fd, 0); /* now dump to the file */ hsts_store_dump (store, fp); /* fclose is expected to unlock the file for us */ fclose (fp); } } } bool hsts_store_has_changed (hsts_store_t store) { return (store ? store->changed : false); } void hsts_store_close (hsts_store_t store) { hash_table_iterator it; /* free all the host fields */ for (hash_table_iterate (store->table, &it); hash_table_iter_next (&it);) { xfree (((struct hsts_kh *) it.key)->host); xfree (it.key); xfree (it.value); } hash_table_destroy (store->table); } #ifdef TESTING /* I know I'm really evil because I'm writing macros that change control flow. But we're testing, who will tell? :D */ #define TEST_URL_RW(s, u, p) do { \ if (test_url_rewrite (s, u, p, true)) \ return test_url_rewrite (s, u, p, true); \ } while (0) #define TEST_URL_NORW(s, u, p) do { \ if (test_url_rewrite (s, u, p, false)) \ return test_url_rewrite (s, u, p, false); \ } while (0) static char * get_hsts_store_filename (void) { char *filename = NULL; FILE *fp = NULL; if (opt.homedir) { filename = ajoin_dir_file (opt.homedir, ".wget-hsts-test"); fp = fopen (filename, "w"); if (fp) fclose (fp); } return filename; } static hsts_store_t open_hsts_test_store (void) { char *filename = NULL; hsts_store_t table = NULL; filename = get_hsts_store_filename (); table = hsts_store_open (filename); xfree (filename); return table; } static void close_hsts_test_store (hsts_store_t store) { char *filename; if ((filename = get_hsts_store_filename ())) { unlink (filename); xfree (filename); } xfree (store); } static const char* test_url_rewrite (hsts_store_t s, const char *url, int port, bool rewrite) { bool result; struct url u; u.host = xstrdup (url); u.port = port; u.scheme = SCHEME_HTTP; result = hsts_match (s, &u); if (rewrite) { if (port == 80) mu_assert("URL: port should've been rewritten to 443", u.port == 443); else mu_assert("URL: port should've been left intact", u.port == port); mu_assert("URL: scheme should've been rewritten to HTTPS", u.scheme == SCHEME_HTTPS); mu_assert("result should've been true", result == true); } else { mu_assert("URL: port should've been left intact", u.port == port); mu_assert("URL: scheme should've been left intact", u.scheme == SCHEME_HTTP); mu_assert("result should've been false", result == false); } xfree (u.host); return NULL; } const char * test_hsts_new_entry (void) { enum hsts_kh_match match = NO_MATCH; struct hsts_kh_info *khi; hsts_store_t s; bool created; s = open_hsts_test_store (); mu_assert("Could not open the HSTS store. This could be due to lack of memory.", s != NULL); created = hsts_store_entry (s, SCHEME_HTTP, "www.foo.com", 80, 1234, true); mu_assert("No entry should have been created.", created == false); created = hsts_store_entry (s, SCHEME_HTTPS, "www.foo.com", 443, 1234, true); mu_assert("A new entry should have been created", created == true); khi = hsts_find_entry (s, "www.foo.com", MAKE_EXPLICIT_PORT (SCHEME_HTTPS, 443), &match, NULL); mu_assert("Should've been a congruent match", match == CONGRUENT_MATCH); mu_assert("No valid HSTS info was returned", khi != NULL); mu_assert("Variable 'max_age' should be 1234", khi->max_age == 1234); mu_assert("Variable 'include_subdomains' should be asserted", khi->include_subdomains == true); khi = hsts_find_entry (s, "b.www.foo.com", MAKE_EXPLICIT_PORT (SCHEME_HTTPS, 443), &match, NULL); mu_assert("Should've been a superdomain match", match == SUPERDOMAIN_MATCH); mu_assert("No valid HSTS info was returned", khi != NULL); mu_assert("Variable 'max_age' should be 1234", khi->max_age == 1234); mu_assert("Variable 'include_subdomains' should be asserted", khi->include_subdomains == true); khi = hsts_find_entry (s, "ww.foo.com", MAKE_EXPLICIT_PORT (SCHEME_HTTPS, 443), &match, NULL); mu_assert("Should've been no match", match == NO_MATCH); khi = hsts_find_entry (s, "foo.com", MAKE_EXPLICIT_PORT (SCHEME_HTTPS, 443), &match, NULL); mu_assert("Should've been no match", match == NO_MATCH); khi = hsts_find_entry (s, ".foo.com", MAKE_EXPLICIT_PORT (SCHEME_HTTPS, 443), &match, NULL); mu_assert("Should've been no match", match == NO_MATCH); khi = hsts_find_entry (s, ".www.foo.com", MAKE_EXPLICIT_PORT (SCHEME_HTTPS, 443), &match, NULL); mu_assert("Should've been no match", match == NO_MATCH); hsts_store_close (s); close_hsts_test_store (s); return NULL; } const char* test_hsts_url_rewrite_superdomain (void) { hsts_store_t s; bool created; s = open_hsts_test_store (); mu_assert("Could not open the HSTS store", s != NULL); created = hsts_store_entry (s, SCHEME_HTTPS, "www.foo.com", 443, 1234, true); mu_assert("A new entry should've been created", created == true); TEST_URL_RW (s, "www.foo.com", 80); TEST_URL_RW (s, "bar.www.foo.com", 80); hsts_store_close (s); close_hsts_test_store (s); return NULL; } const char* test_hsts_url_rewrite_congruent (void) { hsts_store_t s; bool created; s = open_hsts_test_store (); mu_assert("Could not open the HSTS store", s != NULL); created = hsts_store_entry (s, SCHEME_HTTPS, "foo.com", 443, 1234, false); mu_assert("A new entry should've been created", created == true); TEST_URL_RW (s, "foo.com", 80); TEST_URL_NORW (s, "www.foo.com", 80); hsts_store_close (s); close_hsts_test_store (s); return NULL; } const char* test_hsts_read_database (void) { hsts_store_t table; char *file = NULL; FILE *fp = NULL; time_t created = time(NULL) - 10; if (opt.homedir) { file = ajoin_dir_file (opt.homedir, ".wget-hsts-testing"); fp = fopen (file, "w"); if (fp) { fputs ("# dummy comment\n", fp); fprintf (fp, "foo.example.com\t0\t1\t%lu\t123\n",(unsigned long) created); fprintf (fp, "bar.example.com\t0\t0\t%lu\t456\n", (unsigned long) created); fprintf (fp, "test.example.com\t8080\t0\t%lu\t789\n", (unsigned long) created); fclose (fp); table = hsts_store_open (file); TEST_URL_RW (table, "foo.example.com", 80); TEST_URL_RW (table, "www.foo.example.com", 80); TEST_URL_RW (table, "bar.example.com", 80); TEST_URL_NORW(table, "www.bar.example.com", 80); TEST_URL_RW (table, "test.example.com", 8080); hsts_store_close (table); close_hsts_test_store (table); unlink (file); } xfree (file); } return NULL; } #endif /* TESTING */ #endif /* HAVE_HSTS */ wget-1.21.2/src/html-url.h0000644000000000000000000000451014115732710012137 00000000000000/* Declarations for html-url.c. Copyright (C) 1995-1997, 2009-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #ifndef HTML_URL_H #define HTML_URL_H #include #include "utils.h" #include "convert.h" #include "iri.h" struct map_context { char *text; /* HTML text. */ char *base; /* Base URI of the document, possibly changed through . */ const char *parent_base; /* Base of the current document. */ const char *document_file; /* File name of this document. */ bool nofollow; /* whether NOFOLLOW was specified in a tag. */ struct urlpos *head; /* List of URLs that is being built. */ }; struct urlpos *get_urls_file (const char *); struct urlpos *get_urls_html (const char *, const char *, bool *, struct iri *); struct urlpos *get_urls_html_fm (const char *, const struct file_memory *, const char *, bool *, struct iri *); struct urlpos *append_url (const char *, int, int, struct map_context *); void free_urlpos (struct urlpos *); void cleanup_html_url (void); #endif /* HTML_URL_H */ wget-1.21.2/src/xattr.c0000644000000000000000000000472714115732710011542 00000000000000/* xattr.h -- POSIX Extended Attribute support. Copyright (C) 2016, 2018-2021 Free Software Foundation, Inc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, see . */ #include "wget.h" #include #include #include "log.h" #include "utils.h" #include "xattr.h" #ifdef USE_XATTR static int write_xattr_metadata (const char *name, const char *value, FILE *fp) { int retval = -1; if (name && value && fp) { retval = fsetxattr (fileno (fp), name, value, strlen (value), 0); /* FreeBSD's extattr_set_fd returns the length of the extended attribute. */ retval = (retval < 0) ? retval : 0; if (retval) DEBUGP (("Failed to set xattr %s.\n", quote(name))); } return retval; } #else /* USE_XATTR */ static int write_xattr_metadata (const char *name, const char *value, FILE *fp) { (void)name; (void)value; (void)fp; return 0; } #endif /* USE_XATTR */ int set_file_metadata (const struct url *origin_url, const struct url *referrer_url, FILE *fp) { /* Save metadata about where the file came from (requested, final URLs) to * user POSIX Extended Attributes of retrieved file. * * For more details about the user namespace see * [http://freedesktop.org/wiki/CommonExtendedAttributes] and * [http://0pointer.de/lennart/projects/mod_mime_xattr/]. */ int retval = -1; char *value; if (!origin_url || !fp) return retval; value = url_string (origin_url, URL_AUTH_HIDE); retval = write_xattr_metadata ("user.xdg.origin.url", escnonprint_uri (value), fp); xfree (value); if (!retval && referrer_url) { struct url u; memset(&u, 0, sizeof(u)); u.scheme = referrer_url->scheme; u.host = referrer_url->host; u.port = referrer_url->port; value = url_string (&u, 0); retval = write_xattr_metadata ("user.xdg.referrer.url", escnonprint_uri (value), fp); xfree (value); } return retval; } wget-1.21.2/src/metalink.h0000644000000000000000000000554214115732710012205 00000000000000/* Declarations for metalink.c. Copyright (C) 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #if ! defined METALINK_H && defined HAVE_METALINK #define METALINK_H #include #include "dirname.h" #include "wget.h" #ifdef HAVE_SSL # define RES_TYPE_SUPPORTED(x)\ ((!x) || !strcmp (x, "http") || !strcmp (x, "https") || !strcmp (x, "ftp") || !strcmp (x, "ftps")) #else # define RES_TYPE_SUPPORTED(x)\ ((!x) || !strcmp (x, "ftp") || !strcmp (x, "http")) #endif #define DEFAULT_PRI 999999 #define VALID_PRI_RANGE(x) ((x) > 0 && (x) < 1000000) uerr_t retrieve_from_metalink (const metalink_t *metalink); int metalink_res_cmp (const void *res1, const void *res2); int metalink_meta_cmp (const void* meta1, const void* meta2); int metalink_check_safe_path (const char *path); void replace_metalink_basename (char **name, char *ref); char *get_metalink_basename (char *name); void append_suffix_number (char **str, const char *sep, wgint num); void clean_metalink_string (char **str); void dequote_metalink_string (char **str); void badhash_suffix (char *name); void badhash_or_remove (char *name); uerr_t fetch_metalink_file (const char *url_str, bool resume, bool metalink_http, const char *filename, char **destname); bool find_key_value (const char *start, const char *end, const char *key, char **value); bool has_key (const char *start, const char *end, const char *key); const char *find_key_values (const char *start, const char *end, char **key, char **value); #endif /* METALINK_H */ wget-1.21.2/src/openssl.c0000644000000000000000000011120114115732710012045 00000000000000/* SSL support via OpenSSL library. Copyright (C) 2000-2012, 2015, 2018-2021 Free Software Foundation, Inc. Originally contributed by Christian Fraenkel. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #include "wget.h" #include #include #include #include #include #include #include #include #include #include #if OPENSSL_VERSION_NUMBER >= 0x00907000 #include #ifndef OPENSSL_NO_ENGINE #include #endif #endif #include #include "utils.h" #include "connect.h" #include "ptimer.h" #include "url.h" #include "ssl.h" #include #ifdef WINDOWS # include #endif /* Application-wide SSL context. This is common to all SSL connections. */ static SSL_CTX *ssl_ctx; /* Initialize the SSL's PRNG using various methods. */ static void init_prng (void) { char namebuf[256]; const char *random_file; /* Seed from a file specified by the user. This will be the file specified with --random-file, $RANDFILE, if set, or ~/.rnd, if it exists. */ if (opt.random_file) random_file = opt.random_file; else { /* Get the random file name using RAND_file_name. */ namebuf[0] = '\0'; random_file = RAND_file_name (namebuf, sizeof (namebuf)); if (!file_exists_p (random_file, NULL)) random_file = NULL; } if (random_file && *random_file) /* Seed at most 16k (apparently arbitrary value borrowed from curl) from random file. */ { int _err = RAND_load_file (random_file, 16384); if(_err == -1) /* later the thread error queue will be cleared */ if ( (_err = ERR_peek_last_error ()) ) logprintf (LOG_VERBOSE, "WARNING: Could not load random file: %s, %s\n", opt.random_file, ERR_reason_error_string(_err)); } #ifdef HAVE_RAND_EGD /* Get random data from EGD if opt.egd_file was used. */ if (opt.egd_file && *opt.egd_file) RAND_egd (opt.egd_file); #endif #ifdef WINDOWS /* Under Windows, we can try to seed the PRNG using screen content. This may or may not work, depending on whether we'll calling Wget interactively. */ RAND_screen (); if (RAND_status ()) return; #endif #if 0 /* don't do this by default */ { int maxrand = 500; /* Still not random enough, presumably because neither /dev/random nor EGD were available. Try to seed OpenSSL's PRNG with libc PRNG. This is cryptographically weak and defeats the purpose of using OpenSSL, which is why it is highly discouraged. */ logprintf (LOG_NOTQUIET, _("WARNING: using a weak random seed.\n")); while (RAND_status () == 0 && maxrand-- > 0) { unsigned char rnd = random_number (256); RAND_seed (&rnd, sizeof (rnd)); } } #endif } /* Print errors in the OpenSSL error stack. */ static void print_errors (void) { unsigned long err; while ((err = ERR_get_error ()) != 0) logprintf (LOG_NOTQUIET, "OpenSSL: %s\n", ERR_error_string (err, NULL)); } /* Convert keyfile type as used by options.h to a type as accepted by SSL_CTX_use_certificate_file and SSL_CTX_use_PrivateKey_file. (options.h intentionally doesn't use values from openssl/ssl.h so it doesn't depend specifically on OpenSSL for SSL functionality.) */ static int key_type_to_ssl_type (enum keyfile_type type) { switch (type) { case keyfile_pem: return SSL_FILETYPE_PEM; case keyfile_asn1: return SSL_FILETYPE_ASN1; default: abort (); } } /* SSL has been initialized */ static int ssl_true_initialized = 0; /* Create an SSL Context and set default paths etc. Called the first time an HTTP download is attempted. Returns true on success, false otherwise. */ bool ssl_init (void) { SSL_METHOD const *meth; long ssl_options = 0; char *ciphers_string = NULL; #if !defined(LIBRESSL_VERSION_NUMBER) && (OPENSSL_VERSION_NUMBER >= 0x10100000L) int ssl_proto_version = 0; #endif #if OPENSSL_VERSION_NUMBER >= 0x00907000 if (ssl_true_initialized == 0) { #if !defined(LIBRESSL_VERSION_NUMBER) && (OPENSSL_VERSION_NUMBER >= 0x10100000L) OPENSSL_init_ssl (OPENSSL_INIT_LOAD_CONFIG | OPENSSL_INIT_ENGINE_ALL_BUILTIN, NULL); #else OPENSSL_config (NULL); #endif ssl_true_initialized = 1; } #endif if (ssl_ctx) /* The SSL has already been initialized. */ return true; /* Init the PRNG. If that fails, bail out. */ init_prng (); if (RAND_status () != 1) { logprintf (LOG_NOTQUIET, _("Could not seed PRNG; consider using --random-file.\n")); goto error; } #if defined(LIBRESSL_VERSION_NUMBER) || (OPENSSL_VERSION_NUMBER < 0x10100000L) SSL_library_init (); SSL_load_error_strings (); SSLeay_add_all_algorithms (); SSLeay_add_ssl_algorithms (); #endif switch (opt.secure_protocol) { #if !defined OPENSSL_NO_SSL2 && OPENSSL_VERSION_NUMBER < 0x10100000L case secure_protocol_sslv2: meth = SSLv2_client_method (); break; #endif #ifndef OPENSSL_NO_SSL3_METHOD case secure_protocol_sslv3: meth = SSLv3_client_method (); break; #endif case secure_protocol_auto: case secure_protocol_pfs: meth = SSLv23_client_method (); ssl_options |= SSL_OP_NO_SSLv2 | SSL_OP_NO_SSLv3; break; case secure_protocol_tlsv1: #if !defined(LIBRESSL_VERSION_NUMBER) && (OPENSSL_VERSION_NUMBER >= 0x10100000L) meth = TLS_client_method(); ssl_proto_version = TLS1_VERSION; #else meth = TLSv1_client_method (); #endif break; #if OPENSSL_VERSION_NUMBER >= 0x10001000 case secure_protocol_tlsv1_1: #if !defined(LIBRESSL_VERSION_NUMBER) && (OPENSSL_VERSION_NUMBER >= 0x10100000L) meth = TLS_client_method(); ssl_proto_version = TLS1_1_VERSION; #else meth = TLSv1_1_client_method (); #endif break; case secure_protocol_tlsv1_2: #if !defined(LIBRESSL_VERSION_NUMBER) && (OPENSSL_VERSION_NUMBER >= 0x10100000L) meth = TLS_client_method(); ssl_proto_version = TLS1_2_VERSION; #else meth = TLSv1_2_client_method (); #endif break; case secure_protocol_tlsv1_3: #if !defined(LIBRESSL_VERSION_NUMBER) && (OPENSSL_VERSION_NUMBER >= 0x10100000L) && defined TLS1_3_VERSION meth = TLS_client_method(); ssl_proto_version = TLS1_3_VERSION; #else logprintf (LOG_NOTQUIET, _("Your OpenSSL version is too old to support TLS 1.3\n")); goto error; #endif break; #else case secure_protocol_tlsv1_1: logprintf (LOG_NOTQUIET, _("Your OpenSSL version is too old to support TLSv1.1\n")); goto error; case secure_protocol_tlsv1_2: logprintf (LOG_NOTQUIET, _("Your OpenSSL version is too old to support TLSv1.2\n")); goto error; #endif default: logprintf (LOG_NOTQUIET, _("OpenSSL: unimplemented 'secure-protocol' option value %d\n"), opt.secure_protocol); logprintf (LOG_NOTQUIET, _("Please report this issue to bug-wget@gnu.org\n")); abort (); } /* The type cast below accommodates older OpenSSL versions (0.9.8) where SSL_CTX_new() is declared without a "const" argument. */ ssl_ctx = SSL_CTX_new ((SSL_METHOD *)meth); if (!ssl_ctx) goto error; if (ssl_options) SSL_CTX_set_options (ssl_ctx, ssl_options); #if ((OPENSSL_VERSION_NUMBER >= 0x10101000L) && \ !defined(LIBRESSL_VERSION_NUMBER) && \ !defined(OPENSSL_IS_BORINGSSL)) SSL_CTX_set_post_handshake_auth (ssl_ctx, 1); #endif #if !defined(LIBRESSL_VERSION_NUMBER) && (OPENSSL_VERSION_NUMBER >= 0x10100000L) if (ssl_proto_version) SSL_CTX_set_min_proto_version(ssl_ctx, ssl_proto_version); #endif /* OpenSSL ciphers: https://www.openssl.org/docs/apps/ciphers.html * * Rules: * 1. --ciphers overrides everything * 2. We allow RSA key exchange by default (secure_protocol_auto) * 3. We disallow RSA key exchange if PFS was requested (secure_protocol_pfs) */ if (!opt.tls_ciphers_string) { if (opt.secure_protocol == secure_protocol_auto) ciphers_string = "HIGH:!aNULL:!RC4:!MD5:!SRP:!PSK"; else if (opt.secure_protocol == secure_protocol_pfs) ciphers_string = "HIGH:!aNULL:!RC4:!MD5:!SRP:!PSK:!kRSA"; } else { ciphers_string = opt.tls_ciphers_string; } if (ciphers_string && !SSL_CTX_set_cipher_list(ssl_ctx, ciphers_string)) { logprintf(LOG_NOTQUIET, _("OpenSSL: Invalid cipher list: %s\n"), ciphers_string); goto error; } SSL_CTX_set_default_verify_paths (ssl_ctx); SSL_CTX_load_verify_locations (ssl_ctx, opt.ca_cert, opt.ca_directory); #ifdef X509_V_FLAG_PARTIAL_CHAIN /* Set X509_V_FLAG_PARTIAL_CHAIN to allow the client to anchor trust in * a non-self-signed certificate. This defies RFC 4158 (Path Building) * which defines a trust anchor in terms of a self-signed certificate. * However, it substantially reduces attack surface by pruning the tree * of unneeded trust points. For example, the cross-certified * Let's Encrypt X3 CA, which protects gnu.org and appears as an * intermediate CA to clients, can be used as a trust anchor without * the entire IdentTrust PKI. */ X509_VERIFY_PARAM *param = X509_VERIFY_PARAM_new(); if (param) { /* We only want X509_V_FLAG_PARTIAL_CHAIN, but the OpenSSL docs * say to use X509_V_FLAG_TRUSTED_FIRST also. It looks like * X509_V_FLAG_TRUSTED_FIRST applies to a collection of trust * anchors and not a single trust anchor. */ (void) X509_VERIFY_PARAM_set_flags (param, X509_V_FLAG_TRUSTED_FIRST | X509_V_FLAG_PARTIAL_CHAIN); if (SSL_CTX_set1_param (ssl_ctx, param) == 0) logprintf(LOG_NOTQUIET, _("OpenSSL: Failed set trust to partial chain\n")); /* We continue on error */ X509_VERIFY_PARAM_free (param); } else { logprintf(LOG_NOTQUIET, _("OpenSSL: Failed to allocate verification param\n")); /* We continue on error */ } #endif if (opt.crl_file) { X509_STORE *store = SSL_CTX_get_cert_store (ssl_ctx); X509_LOOKUP *lookup; if (!(lookup = X509_STORE_add_lookup (store, X509_LOOKUP_file ())) || (!X509_load_crl_file (lookup, opt.crl_file, X509_FILETYPE_PEM))) goto error; X509_STORE_set_flags (store, X509_V_FLAG_CRL_CHECK | X509_V_FLAG_CRL_CHECK_ALL); } /* SSL_VERIFY_NONE instructs OpenSSL not to abort SSL_connect if the certificate is invalid. We verify the certificate separately in ssl_check_certificate, which provides much better diagnostics than examining the error stack after a failed SSL_connect. */ SSL_CTX_set_verify (ssl_ctx, SSL_VERIFY_NONE, NULL); /* Use the private key from the cert file unless otherwise specified. */ if (opt.cert_file && !opt.private_key) { opt.private_key = xstrdup (opt.cert_file); opt.private_key_type = opt.cert_type; } /* Use cert from private key file unless otherwise specified. */ if (opt.private_key && !opt.cert_file) { opt.cert_file = xstrdup (opt.private_key); opt.cert_type = opt.private_key_type; } if (opt.cert_file) if (SSL_CTX_use_certificate_file (ssl_ctx, opt.cert_file, key_type_to_ssl_type (opt.cert_type)) != 1) goto error; if (opt.private_key) if (SSL_CTX_use_PrivateKey_file (ssl_ctx, opt.private_key, key_type_to_ssl_type (opt.private_key_type)) != 1) goto error; /* Since fd_write unconditionally assumes partial writes (and handles them correctly), allow them in OpenSSL. */ SSL_CTX_set_mode (ssl_ctx, SSL_MODE_ENABLE_PARTIAL_WRITE); /* The OpenSSL library can handle renegotiations automatically, so tell it to do so. */ SSL_CTX_set_mode (ssl_ctx, SSL_MODE_AUTO_RETRY); return true; error: if (ssl_ctx) SSL_CTX_free (ssl_ctx); print_errors (); return false; } void ssl_cleanup (void) { } struct openssl_transport_context { SSL *conn; /* SSL connection handle */ SSL_SESSION *sess; /* SSL session info */ char *last_error; /* last error printed with openssl_errstr */ }; typedef int (*ssl_fn_t)(SSL *, void *, int); #ifdef OPENSSL_RUN_WITHTIMEOUT struct scwt_context { SSL *ssl; int result; }; static void ssl_connect_with_timeout_callback(void *arg) { struct scwt_context *ctx = (struct scwt_context *)arg; ctx->result = SSL_connect(ctx->ssl); } static int ssl_connect_with_timeout(int fd _GL_UNUSED, SSL *conn, double timeout) { struct scwt_context scwt_ctx; scwt_ctx.ssl = conn; errno = 0; if (run_with_timeout(timeout, ssl_connect_with_timeout_callback, &scwt_ctx)) { errno = ETIMEDOUT; return -1; } return scwt_ctx.result; } struct openssl_read_args { int fd; struct openssl_transport_context *ctx; ssl_fn_t fn; char *buf; int bufsize; int retval; }; static void openssl_read_peek_callback(void *arg) { struct openssl_read_args *args = (struct openssl_read_args *) arg; struct openssl_transport_context *ctx = args->ctx; ssl_fn_t fn = args->fn; SSL *conn = ctx->conn; char *buf = args->buf; int bufsize = args->bufsize; int ret; do { ret = fn (conn, buf, bufsize); } while (ret == -1 && SSL_get_error (conn, ret) == SSL_ERROR_SYSCALL && errno == EINTR); args->retval = ret; } static int openssl_read_peek (int fd, char *buf, int bufsize, void *arg, double timeout, ssl_fn_t fn) { struct openssl_transport_context *ctx = arg; int ret = SSL_pending (ctx->conn); if (ret) ret = fn (ctx->conn, buf, MIN (bufsize, ret)); else { struct openssl_read_args args; args.fd = fd; args.buf = buf; args.bufsize = bufsize; args.fn = fn; args.ctx = ctx; if (timeout == -1) timeout = opt.read_timeout; if (run_with_timeout(timeout, openssl_read_peek_callback, &args)) { errno = ETIMEDOUT; ret = -1; } else ret = args.retval; } return ret; } #else /* OPENSSL_RUN_WITHTIMEOUT */ #ifdef F_GETFL #define NONBLOCK_DECL int flags = 0; #define FD_SET_NONBLOCKED(_fd) \ flags = fcntl (_fd, F_GETFL, 0); \ if (flags < 0) \ return flags; \ if (fcntl (_fd, F_SETFL, flags | O_NONBLOCK)) \ return -1; #define FD_SET_BLOCKED(_fd) \ if (fcntl (_fd, F_SETFL, flags) < 0) \ return -1; #else #define NONBLOCK_DECL #define FD_SET_NONBLOCKED(_fd) \ {\ const int one = 1;\ if (ioctl (_fd, FIONBIO, &one) < 0)\ return -1;\ } #define FD_SET_BLOCKED(_fd) \ {\ const int zero = 0;\ if (ioctl (_fd, FIONBIO, &zero) < 0)\ return -1;\ } #endif /* F_GETFL */ #define TIMER_INIT(_fd, _ret, _timeout) \ { \ NONBLOCK_DECL \ int timed_out = 0; \ FD_SET_NONBLOCKED(_fd) \ struct ptimer *timer = ptimer_new (); \ if (timer == NULL) \ _ret = -1; \ else \ { \ double next_timeout = _timeout; #define TIMER_FREE(_fd) \ ptimer_destroy (timer); \ } \ FD_SET_BLOCKED(_fd) \ if (timed_out) \ { \ errno = ETIMEDOUT; \ } \ } #define TIMER_WAIT(_fd, _conn, _ret, _timeout) \ { \ int wait_for; \ int err = SSL_get_error(_conn, _ret); \ if (err == SSL_ERROR_WANT_READ) \ wait_for = WAIT_FOR_READ; \ else if (err == SSL_ERROR_WANT_WRITE) \ wait_for = WAIT_FOR_WRITE; \ else \ break; \ err = select_fd_nb (_fd, next_timeout, wait_for); \ if (err <= 0) \ { \ if (err == 0) \ timedout: \ timed_out = 1; \ _ret = -1; \ break; \ } \ next_timeout = _timeout - ptimer_measure (timer); \ if (next_timeout <= 0) \ goto timedout; \ } static int ssl_connect_with_timeout(int fd, SSL *conn, double timeout) { int ret; errno = 0; if (timeout == 0) ret = SSL_connect(conn); else { TIMER_INIT(fd, ret, timeout) ERR_clear_error(); while( (ret = SSL_connect(conn)) < 0 ) TIMER_WAIT(fd, conn, ret, timeout) TIMER_FREE(fd) } return ret; } static int openssl_read_peek (int fd, char *buf, int bufsize, void *arg, double timeout, ssl_fn_t fn) { struct openssl_transport_context *ctx = arg; int ret = SSL_pending (ctx->conn); if (timeout == -1) timeout = opt.read_timeout; /* If we have data available for immediate read, simply return that, or do blocked read when timeout == 0 */ if (ret || timeout == 0) do { ret = fn (ctx->conn, buf, (ret ? MIN (bufsize, ret) : bufsize)); } while (ret == -1 && SSL_get_error (ctx->conn, ret) == SSL_ERROR_SYSCALL && errno == EINTR); else { TIMER_INIT(fd, ret, timeout) while( (ret = fn (ctx->conn, buf, bufsize)) <= 0 ) TIMER_WAIT(fd, ctx->conn, ret, timeout) TIMER_FREE(fd) } return ret; } #endif /* OPENSSL_RUN_WITHTIMEOUT */ static int openssl_read (int fd, char *buf, int bufsize, void *arg, double timeout) { return openssl_read_peek (fd, buf, bufsize, arg, timeout, SSL_read); } static int openssl_write (int fd _GL_UNUSED, char *buf, int bufsize, void *arg) { int ret = 0; struct openssl_transport_context *ctx = arg; SSL *conn = ctx->conn; do ret = SSL_write (conn, buf, bufsize); while (ret == -1 && SSL_get_error (conn, ret) == SSL_ERROR_SYSCALL && errno == EINTR); return ret; } static int openssl_poll (int fd, double timeout, int wait_for, void *arg) { struct openssl_transport_context *ctx = arg; SSL *conn = ctx->conn; if ((wait_for & WAIT_FOR_READ) && SSL_pending (conn)) return 1; /* if (timeout == 0) return 1; */ if (timeout == -1) timeout = opt.read_timeout; return select_fd (fd, timeout, wait_for); } static int openssl_peek (int fd, char *buf, int bufsize, void *arg, double timeout) { return openssl_read_peek (fd, buf, bufsize, arg, timeout, SSL_peek); } static const char * openssl_errstr (int fd _GL_UNUSED, void *arg) { struct openssl_transport_context *ctx = arg; unsigned long errcode; char *errmsg = NULL; int msglen = 0; /* If there are no SSL-specific errors, just return NULL. */ if ((errcode = ERR_get_error ()) == 0) return NULL; /* Get rid of previous contents of ctx->last_error, if any. */ xfree (ctx->last_error); /* Iterate over OpenSSL's error stack and accumulate errors in the last_error buffer, separated by "; ". This is better than using a static buffer, which *always* takes up space (and has to be large, to fit more than one error message), whereas these allocations are only performed when there is an actual error. */ for (;;) { const char *str = ERR_error_string (errcode, NULL); int len = strlen (str); /* Allocate space for the existing message, plus two more chars for the "; " separator and one for the terminating \0. */ errmsg = xrealloc (errmsg, msglen + len + 2 + 1); memcpy (errmsg + msglen, str, len); msglen += len; /* Get next error and bail out if there are no more. */ errcode = ERR_get_error (); if (errcode == 0) break; errmsg[msglen++] = ';'; errmsg[msglen++] = ' '; } errmsg[msglen] = '\0'; /* Store the error in ctx->last_error where openssl_close will eventually find it and free it. */ ctx->last_error = errmsg; return errmsg; } static void openssl_close (int fd, void *arg) { struct openssl_transport_context *ctx = arg; SSL *conn = ctx->conn; SSL_shutdown (conn); SSL_free (conn); xfree (ctx->last_error); xfree (ctx); close (fd); DEBUGP (("Closed %d/SSL 0x%0*lx\n", fd, PTR_FORMAT (conn))); } /* openssl_transport is the singleton that describes the SSL transport methods provided by this file. */ static struct transport_implementation openssl_transport = { openssl_read, openssl_write, openssl_poll, openssl_peek, openssl_errstr, openssl_close }; static const char * _sni_hostname(const char *hostname) { size_t len = strlen(hostname); char *sni_hostname = xmemdup(hostname, len + 1); /* Remove trailing dot(s) to fix #47408. * Regarding RFC 6066 (SNI): The hostname is represented as a byte * string using ASCII encoding without a trailing dot. */ while (len && sni_hostname[--len] == '.') sni_hostname[len] = 0; return sni_hostname; } /* Perform the SSL handshake on file descriptor FD, which is assumed to be connected to an SSL server. The SSL handle provided by OpenSSL is registered with the file descriptor FD using fd_register_transport, so that subsequent calls to fd_read, fd_write, etc., will use the corresponding SSL functions. Returns true on success, false on failure. */ bool ssl_connect_wget (int fd, const char *hostname, int *continue_session) { SSL *conn; struct openssl_transport_context *ctx; DEBUGP (("Initiating SSL handshake.\n")); assert (ssl_ctx != NULL); conn = SSL_new (ssl_ctx); if (!conn) goto error; #if OPENSSL_VERSION_NUMBER >= 0x0090806fL && !defined(OPENSSL_NO_TLSEXT) /* If the SSL library was built with support for ServerNameIndication then use it whenever we have a hostname. If not, don't, ever. */ if (! is_valid_ip_address (hostname)) { const char *sni_hostname = _sni_hostname(hostname); long rc = SSL_set_tlsext_host_name (conn, sni_hostname); xfree(sni_hostname); if (rc == 0) { DEBUGP (("Failed to set TLS server-name indication.")); goto error; } } #endif if (continue_session) { /* attempt to resume a previous SSL session */ ctx = (struct openssl_transport_context *) fd_transport_context (*continue_session); if (!ctx || !ctx->sess || !SSL_set_session (conn, ctx->sess)) goto error; } #ifndef FD_TO_SOCKET # define FD_TO_SOCKET(X) (X) #endif if (!SSL_set_fd (conn, FD_TO_SOCKET (fd))) goto error; SSL_set_connect_state (conn); /* Re-seed the PRNG before the SSL handshake */ init_prng (); if (RAND_status () != 1) { logprintf(LOG_NOTQUIET, _("WARNING: Could not seed PRNG. Consider using --random-file.\n")); goto error; } if (ssl_connect_with_timeout(fd, conn, opt.read_timeout) <= 0 || !SSL_is_init_finished(conn)) goto timedout; ctx = xnew0 (struct openssl_transport_context); ctx->conn = conn; ctx->sess = SSL_get0_session (conn); if (!ctx->sess) logprintf (LOG_NOTQUIET, "WARNING: Could not save SSL session data for socket %d\n", fd); /* Register FD with Wget's transport layer, i.e. arrange that our functions are used for reading, writing, and polling. */ fd_register_transport (fd, &openssl_transport, ctx); DEBUGP (("Handshake successful; connected socket %d to SSL handle 0x%0*lx\n", fd, PTR_FORMAT (conn))); ERR_clear_error (); return true; timedout: if (errno == ETIMEDOUT) DEBUGP (("SSL handshake timed out.\n")); else error: DEBUGP (("SSL handshake failed.\n")); print_errors (); if (conn) SSL_free (conn); return false; } #define ASTERISK_EXCLUDES_DOT /* mandated by rfc2818 */ /* Return true is STRING (case-insensitively) matches PATTERN, false otherwise. The recognized wildcard character is "*", which matches any character in STRING except ".". Any number of the "*" wildcard may be present in the pattern. This is used to match of hosts as indicated in rfc2818: "Names may contain the wildcard character * which is considered to match any single domain name component or component fragment. E.g., *.a.com matches foo.a.com but not bar.foo.a.com. f*.com matches foo.com but not bar.com [or foo.bar.com]." If the pattern contain no wildcards, pattern_match(a, b) is equivalent to !strcasecmp(a, b). */ static bool pattern_match (const char *pattern, const char *string) { const char *p = pattern, *n = string; char c; for (; (c = c_tolower (*p++)) != '\0'; n++) if (c == '*') { for (c = c_tolower (*p); c == '*'; c = c_tolower (*++p)) ; for (; *n != '\0'; n++) if (c_tolower (*n) == c && pattern_match (p, n)) return true; #ifdef ASTERISK_EXCLUDES_DOT else if (*n == '.') return false; #endif return c == '\0'; } else { if (c != c_tolower (*n)) return false; } return *n == '\0'; } static char *_get_rfc2253_formatted (X509_NAME *name) { int len; char *out = NULL; BIO* b; if ((b = BIO_new (BIO_s_mem ()))) { if (X509_NAME_print_ex (b, name, 0, XN_FLAG_RFC2253) >= 0 && (len = BIO_number_written (b)) > 0) { out = xmalloc (len + 1); BIO_read (b, out, len); out[len] = 0; } BIO_free (b); } return out ? out : xstrdup(""); } /* * Heavily modified from: * https://www.owasp.org/index.php/Certificate_and_Public_Key_Pinning#OpenSSL */ static bool pkp_pin_peer_pubkey (X509* cert, const char *pinnedpubkey) { /* Scratch */ int len1 = 0, len2 = 0; char *buff1 = NULL, *temp = NULL; /* Result is returned to caller */ bool result = false; /* if a path wasn't specified, don't pin */ if (!pinnedpubkey) return true; if (!cert) return result; /* Begin Gyrations to get the subjectPublicKeyInfo */ /* Thanks to Viktor Dukhovni on the OpenSSL mailing list */ /* https://groups.google.com/group/mailing.openssl.users/browse_thread /thread/d61858dae102c6c7 */ len1 = i2d_X509_PUBKEY (X509_get_X509_PUBKEY (cert), NULL); if (len1 < 1) goto cleanup; /* failed */ /* https://www.openssl.org/docs/crypto/buffer.html */ buff1 = temp = OPENSSL_malloc (len1); if (!buff1) goto cleanup; /* failed */ /* https://www.openssl.org/docs/crypto/d2i_X509.html */ len2 = i2d_X509_PUBKEY (X509_get_X509_PUBKEY (cert), (unsigned char **) &temp); /* * These checks are verifying we got back the same values as when we * sized the buffer. It's pretty weak since they should always be the * same. But it gives us something to test. */ if ((len1 != len2) || !temp || ((temp - buff1) != len1)) goto cleanup; /* failed */ /* End Gyrations */ /* The one good exit point */ result = wg_pin_peer_pubkey (pinnedpubkey, buff1, len1); cleanup: /* https://www.openssl.org/docs/crypto/buffer.html */ if (NULL != buff1) OPENSSL_free (buff1); return result; } /* Verify the validity of the certificate presented by the server. Also check that the "common name" of the server, as presented by its certificate, corresponds to HOST. (HOST typically comes from the URL and is what the user thinks he's connecting to.) This assumes that ssl_connect_wget has successfully finished, i.e. that the SSL handshake has been performed and that FD is connected to an SSL handle. If opt.check_cert is true (the default), this returns 1 if the certificate is valid, 0 otherwise. If opt.check_cert is 0, the function always returns 1, but should still be called because it warns the user about any problems with the certificate. */ bool ssl_check_certificate (int fd, const char *host) { X509 *cert; GENERAL_NAMES *subjectAltNames; char common_name[256]; long vresult; bool success = true; bool alt_name_checked = false; bool pinsuccess = opt.pinnedpubkey == NULL; /* If the user has specified --no-check-cert, we still want to warn him about problems with the server's certificate. */ const char *severity = opt.check_cert ? _("ERROR") : _("WARNING"); struct openssl_transport_context *ctx = fd_transport_context (fd); SSL *conn = ctx->conn; assert (conn != NULL); /* The user explicitly said to not check for the certificate. */ if (opt.check_cert == CHECK_CERT_QUIET && pinsuccess) return success; cert = SSL_get_peer_certificate (conn); if (!cert) { logprintf (LOG_NOTQUIET, _("%s: No certificate presented by %s.\n"), severity, quotearg_style (escape_quoting_style, host)); success = false; goto no_cert; /* must bail out since CERT is NULL */ } IF_DEBUG { char *subject = _get_rfc2253_formatted (X509_get_subject_name (cert)); char *issuer = _get_rfc2253_formatted (X509_get_issuer_name (cert)); DEBUGP (("certificate:\n subject: %s\n issuer: %s\n", quotearg_n_style (0, escape_quoting_style, subject), quotearg_n_style (1, escape_quoting_style, issuer))); xfree (subject); xfree (issuer); } vresult = SSL_get_verify_result (conn); if (vresult != X509_V_OK) { char *issuer = _get_rfc2253_formatted (X509_get_issuer_name (cert)); logprintf (LOG_NOTQUIET, _("%s: cannot verify %s's certificate, issued by %s:\n"), severity, quotearg_n_style (0, escape_quoting_style, host), quote_n (1, issuer)); xfree(issuer); /* Try to print more user-friendly (and translated) messages for the frequent verification errors. */ switch (vresult) { case X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY: logprintf (LOG_NOTQUIET, _(" Unable to locally verify the issuer's authority.\n")); break; case X509_V_ERR_SELF_SIGNED_CERT_IN_CHAIN: case X509_V_ERR_DEPTH_ZERO_SELF_SIGNED_CERT: logprintf (LOG_NOTQUIET, _(" Self-signed certificate encountered.\n")); break; case X509_V_ERR_CERT_NOT_YET_VALID: logprintf (LOG_NOTQUIET, _(" Issued certificate not yet valid.\n")); break; case X509_V_ERR_CERT_HAS_EXPIRED: logprintf (LOG_NOTQUIET, _(" Issued certificate has expired.\n")); break; default: /* For the less frequent error strings, simply provide the OpenSSL error message. */ logprintf (LOG_NOTQUIET, " %s\n", X509_verify_cert_error_string (vresult)); } success = false; /* Fall through, so that the user is warned about *all* issues with the cert (important with --no-check-certificate.) */ } /* Check that HOST matches the common name in the certificate. #### The following remains to be done: - When matching against common names, it should loop over all common names and choose the most specific one, i.e. the last one, not the first one, which the current code picks. - Ensure that ASN1 strings from the certificate are encoded as UTF-8 which can be meaningfully compared to HOST. */ subjectAltNames = X509_get_ext_d2i (cert, NID_subject_alt_name, NULL, NULL); if (subjectAltNames) { /* Test subject alternative names */ /* SNI hostname must not have a trailing dot */ const char *sni_hostname = _sni_hostname(host); /* Do we want to check for dNSNAmes or ipAddresses (see RFC 2818)? * Signal it by host_in_octet_string. */ ASN1_OCTET_STRING *host_in_octet_string = a2i_IPADDRESS (sni_hostname); int numaltnames = sk_GENERAL_NAME_num (subjectAltNames); int i; for (i=0; i < numaltnames; i++) { const GENERAL_NAME *name = sk_GENERAL_NAME_value (subjectAltNames, i); if (name) { if (host_in_octet_string) { if (name->type == GEN_IPADD) { /* Check for ipAddress */ /* TODO: Should we convert between IPv4-mapped IPv6 * addresses and IPv4 addresses? */ alt_name_checked = true; if (!ASN1_STRING_cmp (host_in_octet_string, name->d.iPAddress)) break; } } else if (name->type == GEN_DNS) { /* dNSName should be IA5String (i.e. ASCII), however who * does trust CA? Convert it into UTF-8 for sure. */ unsigned char *name_in_utf8 = NULL; /* Check for dNSName */ alt_name_checked = true; if (0 <= ASN1_STRING_to_UTF8 (&name_in_utf8, name->d.dNSName)) { /* Compare and check for NULL attack in ASN1_STRING */ if (pattern_match ((char *)name_in_utf8, sni_hostname) && (strlen ((char *)name_in_utf8) == (size_t) ASN1_STRING_length (name->d.dNSName))) { OPENSSL_free (name_in_utf8); break; } OPENSSL_free (name_in_utf8); } } } } sk_GENERAL_NAME_pop_free(subjectAltNames, GENERAL_NAME_free); if (host_in_octet_string) ASN1_OCTET_STRING_free(host_in_octet_string); if (alt_name_checked == true && i >= numaltnames) { logprintf (LOG_NOTQUIET, _("%s: no certificate subject alternative name matches\n" "\trequested host name %s.\n"), severity, quote_n (1, sni_hostname)); success = false; } xfree(sni_hostname); } if (alt_name_checked == false) { /* Test commomName */ X509_NAME *xname = X509_get_subject_name(cert); common_name[0] = '\0'; X509_NAME_get_text_by_NID (xname, NID_commonName, common_name, sizeof (common_name)); if (!pattern_match (common_name, host)) { logprintf (LOG_NOTQUIET, _("\ %s: certificate common name %s doesn't match requested host name %s.\n"), severity, quote_n (0, common_name), quote_n (1, host)); success = false; } else { /* We now determine the length of the ASN1 string. If it * differs from common_name's length, then there is a \0 * before the string terminates. This can be an instance of a * null-prefix attack. * * https://www.blackhat.com/html/bh-usa-09/bh-usa-09-archives.html#Marlinspike * */ int i = -1, j; X509_NAME_ENTRY *xentry; ASN1_STRING *sdata; if (xname) { for (;;) { j = X509_NAME_get_index_by_NID (xname, NID_commonName, i); if (j == -1) break; i = j; } } xentry = X509_NAME_get_entry(xname,i); sdata = X509_NAME_ENTRY_get_data(xentry); if (strlen (common_name) != (size_t) ASN1_STRING_length (sdata)) { logprintf (LOG_NOTQUIET, _("\ %s: certificate common name is invalid (contains a NUL character).\n\ This may be an indication that the host is not who it claims to be\n\ (that is, it is not the real %s).\n"), severity, quote (host)); success = false; } } } pinsuccess = pkp_pin_peer_pubkey (cert, opt.pinnedpubkey); if (!pinsuccess) { logprintf (LOG_ALWAYS, _("The public key does not match pinned public key!\n")); success = false; } if (success) DEBUGP (("X509 certificate successfully verified and matches host %s\n", quotearg_style (escape_quoting_style, host))); X509_free (cert); no_cert: if (opt.check_cert == CHECK_CERT_ON && !success) logprintf (LOG_NOTQUIET, _("\ To connect to %s insecurely, use `--no-check-certificate'.\n"), quotearg_style (escape_quoting_style, host)); /* never return true if pinsuccess fails */ return !pinsuccess ? false : (opt.check_cert == CHECK_CERT_ON ? success : true); } /* * vim: tabstop=2 shiftwidth=2 softtabstop=2 */ wget-1.21.2/src/hsts.h0000644000000000000000000000344714115732710011364 00000000000000/* Declarations for hsts.c Copyright (C) 1996-2012, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #ifndef WGET_HSTS_H #define WGET_HSTS_H #ifdef HAVE_HSTS #include "wget.h" #include "url.h" typedef struct hsts_store *hsts_store_t; hsts_store_t hsts_store_open (const char *); void hsts_store_save (hsts_store_t, const char *); void hsts_store_close (hsts_store_t); bool hsts_store_has_changed (hsts_store_t); bool hsts_store_entry (hsts_store_t, enum url_scheme, const char *, int, time_t, bool); bool hsts_match (hsts_store_t, struct url *); #endif /* HAVE_HSTS */ #endif /* WGET_HSTS_H */ wget-1.21.2/src/main.c0000644000000000000000000022530414115732710011320 00000000000000/* Command line parsing. Copyright (C) 1996-2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #include "wget.h" #include #include #include #include #include #include #if defined(ENABLE_NLS) || defined(WINDOWS) # include #endif #include #include #include #include "exits.h" #include "utils.h" #include "init.h" #include "retr.h" #include "recur.h" #include "host.h" #include "url.h" #include "progress.h" /* for progress_handle_sigwinch */ #include "convert.h" #include "spider.h" #include "http.h" /* for save_cookies */ #include "hsts.h" /* for initializing hsts_store to NULL */ #include "ptimer.h" #include "warc.h" #include "version.h" #include "c-strcase.h" #include "dirname.h" #include "xmemdup0.h" #include #include #include #ifdef TESTING /* Rename the main function so we can have a main() in fuzzing code and call the original main. */ # define main main_wget #endif #ifdef HAVE_METALINK # include # include "metalink.h" #endif #ifdef WINDOWS # include # include #ifndef ENABLE_NLS # include #endif #endif #ifdef __VMS # include "vms.h" #endif /* __VMS */ #ifndef PATH_SEPARATOR # define PATH_SEPARATOR '/' #endif #ifndef ENABLE_IRI struct iri dummy_iri; #endif #ifdef HAVE_LIBCARES #include ares_channel ares; #else void *ares; #endif struct options opt; /* defined in version.c */ extern char *system_getrc; /* Used for --version output in print_version */ #define MAX_CHARS_PER_LINE 72 #define TABULATION 4 const char *exec_name; /* Number of successfully downloaded URLs */ int numurls = 0; /* Initialize I18N/L10N. That amounts to invoking setlocale, and setting up gettext's message catalog using bindtextdomain and textdomain. Does nothing if NLS is disabled or missing. */ #if defined(SIGHUP) || defined(SIGUSR1) /* Hangup signal handler. When wget receives SIGHUP or SIGUSR1, it will proceed operation as usual, trying to write into a log file. If that is impossible, the output will be turned off. */ static void redirect_output_signal (int sig) { const char *signal_name = "WTF?!"; #ifdef SIGHUP if (sig == SIGHUP) signal_name = "SIGHUP"; #endif #ifdef SIGUSR1 if (sig == SIGUSR1) signal_name = "SIGUSR1"; #endif redirect_output (true,signal_name); progress_schedule_redirect (); signal (sig, redirect_output_signal); } #endif /* defined(SIGHUP) || defined(SIGUSR1) */ static void i18n_initialize (void) { /* ENABLE_NLS implies existence of functions invoked here. */ #ifdef ENABLE_NLS /* Set the current locale. */ setlocale (LC_ALL, ""); /* Set the text message domain. */ bindtextdomain ("wget", LOCALEDIR); bindtextdomain ("wget-gnulib", LOCALEDIR); textdomain ("wget"); #elif defined WINDOWS char MBCP[16] = ""; int CP; CP = _getmbcp(); /* Consider it's different from default. */ if (CP > 0) snprintf(MBCP, sizeof(MBCP), ".%d", CP); setlocale(LC_ALL, MBCP); #endif /* ENABLE_NLS */ } #ifdef HAVE_HSTS /* make the HSTS store global */ hsts_store_t hsts_store; static char* get_hsts_database (void) { if (opt.hsts_file) return xstrdup (opt.hsts_file); if (opt.homedir) { char *dir = ajoin_dir_file(opt.homedir, ".wget-hsts"); return dir; } return NULL; } static void load_hsts (void) { if (!hsts_store) { char *filename = get_hsts_database (); if (filename) { DEBUGP (("Reading HSTS entries from %s\n", filename)); hsts_store = hsts_store_open (filename); if (!hsts_store) logprintf (LOG_NOTQUIET, "ERROR: could not open HSTS store at '%s'. " "HSTS will be disabled.\n", filename); } else logprintf (LOG_NOTQUIET, "ERROR: could not open HSTS store. HSTS will be disabled.\n"); xfree (filename); } } static void save_hsts (void) { if (hsts_store) { char *filename = get_hsts_database (); if (filename && hsts_store_has_changed (hsts_store)) { DEBUGP (("Saving HSTS entries to %s\n", filename)); hsts_store_save (hsts_store, filename); } hsts_store_close (hsts_store); xfree (hsts_store); xfree (filename); } } #endif /* Definition of command-line options. */ _Noreturn static void print_help (void); _Noreturn static void print_version (void); #ifdef HAVE_SSL # define IF_SSL(a,b,c,d,e) { a, b, c, d , e }, #else # define IF_SSL(a,b,c,d,e) #endif struct cmdline_option { char long_name[MAX_LONGOPTION]; char short_name; enum { OPT_VALUE, OPT_BOOLEAN, OPT_FUNCALL, /* Non-standard options that have to be handled specially in main(). */ OPT__APPEND_OUTPUT, OPT__CLOBBER, OPT__DONT_REMOVE_LISTING, OPT__EXECUTE, OPT__NO, OPT__PARENT } type; const void *data; /* for standard options */ int argtype; /* for non-standard options */ }; static struct cmdline_option option_data[] = { { "accept", 'A', OPT_VALUE, "accept", -1 }, { "accept-regex", 0, OPT_VALUE, "acceptregex", -1 }, { "adjust-extension", 'E', OPT_BOOLEAN, "adjustextension", -1 }, { "append-output", 'a', OPT__APPEND_OUTPUT, NULL, required_argument }, { "ask-password", 0, OPT_BOOLEAN, "askpassword", -1 }, { "auth-no-challenge", 0, OPT_BOOLEAN, "authnochallenge", -1 }, { "background", 'b', OPT_BOOLEAN, "background", -1 }, { "backup-converted", 'K', OPT_BOOLEAN, "backupconverted", -1 }, { "backups", 0, OPT_BOOLEAN, "backups", -1 }, { "base", 'B', OPT_VALUE, "base", -1 }, { "bind-address", 0, OPT_VALUE, "bindaddress", -1 }, #ifdef HAVE_LIBCARES { "bind-dns-address", 0, OPT_VALUE, "binddnsaddress", -1 }, #endif { "body-data", 0, OPT_VALUE, "bodydata", -1 }, { "body-file", 0, OPT_VALUE, "bodyfile", -1 }, IF_SSL ( "ca-certificate", 0, OPT_VALUE, "cacertificate", -1 ) IF_SSL ( "ca-directory", 0, OPT_VALUE, "cadirectory", -1 ) { "cache", 0, OPT_BOOLEAN, "cache", -1 }, IF_SSL ( "certificate", 0, OPT_VALUE, "certificate", -1 ) IF_SSL ( "certificate-type", 0, OPT_VALUE, "certificatetype", -1 ) IF_SSL ( "check-certificate", 0, OPT_BOOLEAN, "checkcertificate", -1 ) { "clobber", 0, OPT__CLOBBER, NULL, optional_argument }, #ifdef HAVE_LIBZ { "compression", 0, OPT_VALUE, "compression", -1 }, #endif { "config", 0, OPT_VALUE, "chooseconfig", -1 }, { "connect-timeout", 0, OPT_VALUE, "connecttimeout", -1 }, { "continue", 'c', OPT_BOOLEAN, "continue", -1 }, { "convert-file-only", 0, OPT_BOOLEAN, "convertfileonly", -1 }, { "convert-links", 'k', OPT_BOOLEAN, "convertlinks", -1 }, { "content-disposition", 0, OPT_BOOLEAN, "contentdisposition", -1 }, { "content-on-error", 0, OPT_BOOLEAN, "contentonerror", -1 }, { "cookies", 0, OPT_BOOLEAN, "cookies", -1 }, IF_SSL ( "crl-file", 0, OPT_VALUE, "crlfile", -1 ) { "cut-dirs", 0, OPT_VALUE, "cutdirs", -1 }, { "debug", 'd', OPT_BOOLEAN, "debug", -1 }, { "default-page", 0, OPT_VALUE, "defaultpage", -1 }, { "delete-after", 0, OPT_BOOLEAN, "deleteafter", -1 }, { "directories", 0, OPT_BOOLEAN, "dirstruct", -1 }, { "directory-prefix", 'P', OPT_VALUE, "dirprefix", -1 }, { "dns-cache", 0, OPT_BOOLEAN, "dnscache", -1 }, #ifdef HAVE_LIBCARES { "dns-servers", 0, OPT_VALUE, "dnsservers", -1 }, #endif { "dns-timeout", 0, OPT_VALUE, "dnstimeout", -1 }, { "domains", 'D', OPT_VALUE, "domains", -1 }, { "dont-remove-listing", 0, OPT__DONT_REMOVE_LISTING, NULL, no_argument }, { "dot-style", 0, OPT_VALUE, "dotstyle", -1 }, /* deprecated */ { "egd-file", 0, OPT_VALUE, "egdfile", -1 }, { "exclude-directories", 'X', OPT_VALUE, "excludedirectories", -1 }, { "exclude-domains", 0, OPT_VALUE, "excludedomains", -1 }, { "execute", 'e', OPT__EXECUTE, NULL, required_argument }, { "follow-ftp", 0, OPT_BOOLEAN, "followftp", -1 }, { "follow-tags", 0, OPT_VALUE, "followtags", -1 }, { "force-directories", 'x', OPT_BOOLEAN, "dirstruct", -1 }, { "force-html", 'F', OPT_BOOLEAN, "forcehtml", -1 }, { "ftp-password", 0, OPT_VALUE, "ftppassword", -1 }, #ifdef __VMS { "ftp-stmlf", 0, OPT_BOOLEAN, "ftpstmlf", -1 }, #endif /* def __VMS */ { "ftp-user", 0, OPT_VALUE, "ftpuser", -1 }, IF_SSL ( "ftps-clear-data-connection", 0, OPT_BOOLEAN, "ftpscleardataconnection", -1 ) IF_SSL ( "ftps-fallback-to-ftp", 0, OPT_BOOLEAN, "ftpsfallbacktoftp", -1 ) IF_SSL ( "ftps-implicit", 0, OPT_BOOLEAN, "ftpsimplicit", -1 ) IF_SSL ( "ftps-resume-ssl", 0, OPT_BOOLEAN, "ftpsresumessl", -1 ) { "glob", 0, OPT_BOOLEAN, "glob", -1 }, { "header", 0, OPT_VALUE, "header", -1 }, { "help", 'h', OPT_FUNCALL, (void *)print_help, no_argument }, { "host-directories", 0, OPT_BOOLEAN, "addhostdir", -1 }, #ifdef HAVE_HSTS { "hsts", 0, OPT_BOOLEAN, "hsts", -1}, { "hsts-file", 0, OPT_VALUE, "hstsfile", -1 }, #endif { "html-extension", 'E', OPT_BOOLEAN, "adjustextension", -1 }, /* deprecated */ { "htmlify", 0, OPT_BOOLEAN, "htmlify", -1 }, { "http-keep-alive", 0, OPT_BOOLEAN, "httpkeepalive", -1 }, { "http-passwd", 0, OPT_VALUE, "httppassword", -1 }, /* deprecated */ { "http-password", 0, OPT_VALUE, "httppassword", -1 }, { "http-user", 0, OPT_VALUE, "httpuser", -1 }, IF_SSL ( "https-only", 0, OPT_BOOLEAN, "httpsonly", -1 ) { "ignore-case", 0, OPT_BOOLEAN, "ignorecase", -1 }, { "ignore-length", 0, OPT_BOOLEAN, "ignorelength", -1 }, { "ignore-tags", 0, OPT_VALUE, "ignoretags", -1 }, { "include-directories", 'I', OPT_VALUE, "includedirectories", -1 }, #ifdef ENABLE_IPV6 { "inet4-only", '4', OPT_BOOLEAN, "inet4only", -1 }, { "inet6-only", '6', OPT_BOOLEAN, "inet6only", -1 }, #endif { "input-file", 'i', OPT_VALUE, "input", -1 }, #ifdef HAVE_METALINK { "input-metalink", 0, OPT_VALUE, "inputmetalink", -1 }, #endif { "iri", 0, OPT_BOOLEAN, "iri", -1 }, { "keep-badhash", 0, OPT_BOOLEAN, "keepbadhash", -1 }, { "keep-session-cookies", 0, OPT_BOOLEAN, "keepsessioncookies", -1 }, { "level", 'l', OPT_VALUE, "reclevel", -1 }, { "limit-rate", 0, OPT_VALUE, "limitrate", -1 }, { "load-cookies", 0, OPT_VALUE, "loadcookies", -1 }, { "local-encoding", 0, OPT_VALUE, "localencoding", -1 }, { "rejected-log", 0, OPT_VALUE, "rejectedlog", -1 }, { "max-redirect", 0, OPT_VALUE, "maxredirect", -1 }, #ifdef HAVE_METALINK { "metalink-index", 0, OPT_VALUE, "metalinkindex", -1 }, { "metalink-over-http", 0, OPT_BOOLEAN, "metalinkoverhttp", -1 }, #endif { "method", 0, OPT_VALUE, "method", -1 }, { "mirror", 'm', OPT_BOOLEAN, "mirror", -1 }, { "netrc", 0, OPT_BOOLEAN, "netrc", -1 }, { "no", 'n', OPT__NO, NULL, required_argument }, { "no-clobber", 0, OPT_BOOLEAN, "noclobber", -1 }, { "no-config", 0, OPT_BOOLEAN, "noconfig", -1}, { "no-parent", 0, OPT_BOOLEAN, "noparent", -1 }, { "output-document", 'O', OPT_VALUE, "outputdocument", -1 }, { "output-file", 'o', OPT_VALUE, "logfile", -1 }, { "page-requisites", 'p', OPT_BOOLEAN, "pagerequisites", -1 }, { "parent", 0, OPT__PARENT, NULL, optional_argument }, { "passive-ftp", 0, OPT_BOOLEAN, "passiveftp", -1 }, { "password", 0, OPT_VALUE, "password", -1 }, IF_SSL ( "pinnedpubkey", 0, OPT_VALUE, "pinnedpubkey", -1 ) { "post-data", 0, OPT_VALUE, "postdata", -1 }, { "post-file", 0, OPT_VALUE, "postfile", -1 }, { "prefer-family", 0, OPT_VALUE, "preferfamily", -1 }, #ifdef HAVE_METALINK { "preferred-location", 0, OPT_VALUE, "preferredlocation", -1 }, #endif { "preserve-permissions", 0, OPT_BOOLEAN, "preservepermissions", -1 }, IF_SSL ( "ciphers", 0, OPT_VALUE, "ciphers", -1 ) IF_SSL ( "private-key", 0, OPT_VALUE, "privatekey", -1 ) IF_SSL ( "private-key-type", 0, OPT_VALUE, "privatekeytype", -1 ) { "progress", 0, OPT_VALUE, "progress", -1 }, { "show-progress", 0, OPT_BOOLEAN, "showprogress", -1 }, { "protocol-directories", 0, OPT_BOOLEAN, "protocoldirectories", -1 }, { "proxy", 0, OPT_BOOLEAN, "useproxy", -1 }, { "proxy__compat", 'Y', OPT_VALUE, "useproxy", -1 }, /* back-compatible */ { "proxy-passwd", 0, OPT_VALUE, "proxypassword", -1 }, /* deprecated */ { "proxy-password", 0, OPT_VALUE, "proxypassword", -1 }, { "proxy-user", 0, OPT_VALUE, "proxyuser", -1 }, { "quiet", 'q', OPT_BOOLEAN, "quiet", -1 }, { "quota", 'Q', OPT_VALUE, "quota", -1 }, { "random-file", 0, OPT_VALUE, "randomfile", -1 }, { "random-wait", 0, OPT_BOOLEAN, "randomwait", -1 }, { "read-timeout", 0, OPT_VALUE, "readtimeout", -1 }, { "recursive", 'r', OPT_BOOLEAN, "recursive", -1 }, { "referer", 0, OPT_VALUE, "referer", -1 }, { "regex-type", 0, OPT_VALUE, "regextype", -1 }, { "reject", 'R', OPT_VALUE, "reject", -1 }, { "reject-regex", 0, OPT_VALUE, "rejectregex", -1 }, { "relative", 'L', OPT_BOOLEAN, "relativeonly", -1 }, { "remote-encoding", 0, OPT_VALUE, "remoteencoding", -1 }, { "remove-listing", 0, OPT_BOOLEAN, "removelisting", -1 }, { "report-speed", 0, OPT_BOOLEAN, "reportspeed", -1 }, { "restrict-file-names", 0, OPT_BOOLEAN, "restrictfilenames", -1 }, { "retr-symlinks", 0, OPT_BOOLEAN, "retrsymlinks", -1 }, { "retry-connrefused", 0, OPT_BOOLEAN, "retryconnrefused", -1 }, { "retry-on-host-error", 0, OPT_BOOLEAN, "retryonhosterror", -1 }, { "retry-on-http-error", 0, OPT_VALUE, "retryonhttperror", -1 }, { "save-cookies", 0, OPT_VALUE, "savecookies", -1 }, { "save-headers", 0, OPT_BOOLEAN, "saveheaders", -1 }, IF_SSL ( "secure-protocol", 0, OPT_VALUE, "secureprotocol", -1 ) { "server-response", 'S', OPT_BOOLEAN, "serverresponse", -1 }, { "span-hosts", 'H', OPT_BOOLEAN, "spanhosts", -1 }, { "spider", 0, OPT_BOOLEAN, "spider", -1 }, { "start-pos", 0, OPT_VALUE, "startpos", -1 }, { "strict-comments", 0, OPT_BOOLEAN, "strictcomments", -1 }, { "timeout", 'T', OPT_VALUE, "timeout", -1 }, { "timestamping", 'N', OPT_BOOLEAN, "timestamping", -1 }, { "if-modified-since", 0, OPT_BOOLEAN, "ifmodifiedsince", -1 }, { "tries", 't', OPT_VALUE, "tries", -1 }, { "unlink", 0, OPT_BOOLEAN, "unlink", -1 }, { "trust-server-names", 0, OPT_BOOLEAN, "trustservernames", -1 }, #ifndef __VMS { "use-askpass", 0, OPT_VALUE, "useaskpass", -1}, #endif { "use-server-timestamps", 0, OPT_BOOLEAN, "useservertimestamps", -1 }, { "user", 0, OPT_VALUE, "user", -1 }, { "user-agent", 'U', OPT_VALUE, "useragent", -1 }, { "verbose", 'v', OPT_BOOLEAN, "verbose", -1 }, { "version", 'V', OPT_FUNCALL, (void *) print_version, no_argument }, { "wait", 'w', OPT_VALUE, "wait", -1 }, { "waitretry", 0, OPT_VALUE, "waitretry", -1 }, { "warc-cdx", 0, OPT_BOOLEAN, "warccdx", -1 }, #ifdef HAVE_LIBZ { "warc-compression", 0, OPT_BOOLEAN, "warccompression", -1 }, #endif { "warc-dedup", 0, OPT_VALUE, "warccdxdedup", -1 }, { "warc-digests", 0, OPT_BOOLEAN, "warcdigests", -1 }, { "warc-file", 0, OPT_VALUE, "warcfile", -1 }, { "warc-header", 0, OPT_VALUE, "warcheader", -1 }, { "warc-keep-log", 0, OPT_BOOLEAN, "warckeeplog", -1 }, { "warc-max-size", 0, OPT_VALUE, "warcmaxsize", -1 }, { "warc-tempdir", 0, OPT_VALUE, "warctempdir", -1 }, #ifdef USE_WATT32 { "wdebug", 0, OPT_BOOLEAN, "wdebug", -1 }, #endif #ifdef ENABLE_XATTR { "xattr", 0, OPT_BOOLEAN, "xattr", -1 }, #endif }; #undef IF_SSL /* Return a string that contains S with "no-" prepended. The string is NUL-terminated and allocated off static storage at Wget startup. */ static char * no_prefix (const char *s) { static char buffer[2048]; static char *p = buffer; char *cp = p; int size = 3 + strlen (s) + 1; /* "no-STRING\0" */ assert(p + size <= buffer + sizeof (buffer)); cp[0] = 'n', cp[1] = 'o', cp[2] = '-'; strcpy (cp + 3, s); p += size; return cp; } /* The arguments that that main passes to getopt_long. */ static struct option long_options[2 * countof (option_data) + 1]; static char short_options[128]; /* Mapping between short option chars and option_data indices. */ static unsigned char optmap[96]; /* Marker for `--no-FOO' values in long_options. */ #define BOOLEAN_NEG_MARKER 1024 /* Initialize the long_options array used by getopt_long from the data in option_data. */ static void init_switches (void) { static bool initialized; char *p = short_options; size_t i, o = 0; if (initialized) return; initialized = 1; for (i = 0; i < countof (option_data); i++) { struct cmdline_option *cmdopt = &option_data[i]; struct option *longopt; longopt = &long_options[o++]; longopt->name = cmdopt->long_name; longopt->val = i; if (cmdopt->short_name) { *p++ = cmdopt->short_name; optmap[cmdopt->short_name - 32] = longopt - long_options; } switch (cmdopt->type) { case OPT_VALUE: longopt->has_arg = required_argument; if (cmdopt->short_name) *p++ = ':'; break; case OPT_BOOLEAN: /* Specify an optional argument for long options, so that --option=off works the same as --no-option, for compatibility with pre-1.10 Wget. However, don't specify optional arguments short-option booleans because they prevent combining of short options. */ longopt->has_arg = optional_argument; /* For Boolean options, add the "--no-FOO" variant, which is identical to "--foo", except it has opposite meaning and it doesn't allow an argument. */ longopt = &long_options[o++]; longopt->name = no_prefix (cmdopt->long_name); longopt->has_arg = no_argument; /* Mask the value so we'll be able to recognize that we're dealing with the false value. */ longopt->val = i | BOOLEAN_NEG_MARKER; break; default: assert (cmdopt->argtype != -1); longopt->has_arg = cmdopt->argtype; if (cmdopt->short_name) { if (longopt->has_arg == required_argument) *p++ = ':'; /* Don't handle optional_argument */ } } } /* Terminate short_options. */ *p = '\0'; /* No need for xzero(long_options[o]) because its storage is static and it will be zeroed by default. */ assert (o <= countof (long_options)); } /* Print the usage message. */ static int print_usage (_GL_UNUSED int error) { #ifndef TESTING return fprintf (error ? stderr : stdout, _("Usage: %s [OPTION]... [URL]...\n"), exec_name); #else return 0; #endif } /* Print the help message, describing all the available options. If you add an option, be sure to update this list. */ _Noreturn static void print_help (void) { #ifndef TESTING /* We split the help text this way to ease translation of individual entries. */ static const char *help[] = { "\n", N_("\ Mandatory arguments to long options are mandatory for short options too.\n\n"), N_("\ Startup:\n"), N_("\ -V, --version display the version of Wget and exit\n"), N_("\ -h, --help print this help\n"), N_("\ -b, --background go to background after startup\n"), N_("\ -e, --execute=COMMAND execute a `.wgetrc'-style command\n"), "\n", N_("\ Logging and input file:\n"), N_("\ -o, --output-file=FILE log messages to FILE\n"), N_("\ -a, --append-output=FILE append messages to FILE\n"), #ifdef ENABLE_DEBUG N_("\ -d, --debug print lots of debugging information\n"), #endif #ifdef USE_WATT32 N_("\ --wdebug print Watt-32 debug output\n"), #endif N_("\ -q, --quiet quiet (no output)\n"), N_("\ -v, --verbose be verbose (this is the default)\n"), N_("\ -nv, --no-verbose turn off verboseness, without being quiet\n"), N_("\ --report-speed=TYPE output bandwidth as TYPE. TYPE can be bits\n"), N_("\ -i, --input-file=FILE download URLs found in local or external FILE\n"), #ifdef HAVE_METALINK N_("\ --input-metalink=FILE download files covered in local Metalink FILE\n"), #endif N_("\ -F, --force-html treat input file as HTML\n"), N_("\ -B, --base=URL resolves HTML input-file links (-i -F)\n\ relative to URL\n"), N_("\ --config=FILE specify config file to use\n"), N_("\ --no-config do not read any config file\n"), N_("\ --rejected-log=FILE log reasons for URL rejection to FILE\n"), "\n", N_("\ Download:\n"), N_("\ -t, --tries=NUMBER set number of retries to NUMBER (0 unlimits)\n"), N_("\ --retry-connrefused retry even if connection is refused\n"), N_("\ --retry-on-http-error=ERRORS comma-separated list of HTTP errors to retry\n"), N_("\ -O, --output-document=FILE write documents to FILE\n"), N_("\ -nc, --no-clobber skip downloads that would download to\n\ existing files (overwriting them)\n"), N_("\ --no-netrc don't try to obtain credentials from .netrc\n"), N_("\ -c, --continue resume getting a partially-downloaded file\n"), N_("\ --start-pos=OFFSET start downloading from zero-based position OFFSET\n"), N_("\ --progress=TYPE select progress gauge type\n"), N_("\ --show-progress display the progress bar in any verbosity mode\n"), N_("\ -N, --timestamping don't re-retrieve files unless newer than\n\ local\n"), N_("\ --no-if-modified-since don't use conditional if-modified-since get\n\ requests in timestamping mode\n"), N_("\ --no-use-server-timestamps don't set the local file's timestamp by\n\ the one on the server\n"), N_("\ -S, --server-response print server response\n"), N_("\ --spider don't download anything\n"), N_("\ -T, --timeout=SECONDS set all timeout values to SECONDS\n"), #ifdef HAVE_LIBCARES N_("\ --dns-servers=ADDRESSES list of DNS servers to query (comma separated)\n"), N_("\ --bind-dns-address=ADDRESS bind DNS resolver to ADDRESS (hostname or IP) on local host\n"), #endif N_("\ --dns-timeout=SECS set the DNS lookup timeout to SECS\n"), N_("\ --connect-timeout=SECS set the connect timeout to SECS\n"), N_("\ --read-timeout=SECS set the read timeout to SECS\n"), N_("\ -w, --wait=SECONDS wait SECONDS between retrievals\n\ (applies if more then 1 URL is to be retrieved)\n"), N_("\ --waitretry=SECONDS wait 1..SECONDS between retries of a retrieval\n\ (applies if more then 1 URL is to be retrieved)\n"), N_("\ --random-wait wait from 0.5*WAIT...1.5*WAIT secs between retrievals\n\ (applies if more then 1 URL is to be retrieved)\n"), N_("\ --no-proxy explicitly turn off proxy\n"), N_("\ -Q, --quota=NUMBER set retrieval quota to NUMBER\n"), N_("\ --bind-address=ADDRESS bind to ADDRESS (hostname or IP) on local host\n"), N_("\ --limit-rate=RATE limit download rate to RATE\n"), N_("\ --no-dns-cache disable caching DNS lookups\n"), N_("\ --restrict-file-names=OS restrict chars in file names to ones OS allows\n"), N_("\ --ignore-case ignore case when matching files/directories\n"), #ifdef ENABLE_IPV6 N_("\ -4, --inet4-only connect only to IPv4 addresses\n"), N_("\ -6, --inet6-only connect only to IPv6 addresses\n"), N_("\ --prefer-family=FAMILY connect first to addresses of specified family,\n\ one of IPv6, IPv4, or none\n"), #endif N_("\ --user=USER set both ftp and http user to USER\n"), N_("\ --password=PASS set both ftp and http password to PASS\n"), N_("\ --ask-password prompt for passwords\n"), #ifndef __VMS N_("\ --use-askpass=COMMAND specify credential handler for requesting \n\ username and password. If no COMMAND is \n\ specified the WGET_ASKPASS or the SSH_ASKPASS \n\ environment variable is used.\n"), #endif N_("\ --no-iri turn off IRI support\n"), N_("\ --local-encoding=ENC use ENC as the local encoding for IRIs\n"), N_("\ --remote-encoding=ENC use ENC as the default remote encoding\n"), N_("\ --unlink remove file before clobber\n"), #ifdef HAVE_METALINK N_("\ --keep-badhash keep files with checksum mismatch (append .badhash)\n"), N_("\ --metalink-index=NUMBER Metalink application/metalink4+xml metaurl ordinal NUMBER\n"), N_("\ --metalink-over-http use Metalink metadata from HTTP response headers\n"), N_("\ --preferred-location preferred location for Metalink resources\n"), #endif #ifdef ENABLE_XATTR N_("\ --xattr turn on storage of metadata in extended file attributes\n"), #endif "\n", N_("\ Directories:\n"), N_("\ -nd, --no-directories don't create directories\n"), N_("\ -x, --force-directories force creation of directories\n"), N_("\ -nH, --no-host-directories don't create host directories\n"), N_("\ --protocol-directories use protocol name in directories\n"), N_("\ -P, --directory-prefix=PREFIX save files to PREFIX/..\n"), N_("\ --cut-dirs=NUMBER ignore NUMBER remote directory components\n"), "\n", N_("\ HTTP options:\n"), N_("\ --http-user=USER set http user to USER\n"), N_("\ --http-password=PASS set http password to PASS\n"), N_("\ --no-cache disallow server-cached data\n"), N_ ("\ --default-page=NAME change the default page name (normally\n\ this is 'index.html'.)\n"), N_("\ -E, --adjust-extension save HTML/CSS documents with proper extensions\n"), N_("\ --ignore-length ignore 'Content-Length' header field\n"), N_("\ --header=STRING insert STRING among the headers\n"), #ifdef HAVE_LIBZ N_("\ --compression=TYPE choose compression, one of auto, gzip and none. (default: none)\n"), #endif N_("\ --max-redirect maximum redirections allowed per page\n"), N_("\ --proxy-user=USER set USER as proxy username\n"), N_("\ --proxy-password=PASS set PASS as proxy password\n"), N_("\ --referer=URL include 'Referer: URL' header in HTTP request\n"), N_("\ --save-headers save the HTTP headers to file\n"), N_("\ -U, --user-agent=AGENT identify as AGENT instead of Wget/VERSION\n"), N_("\ --no-http-keep-alive disable HTTP keep-alive (persistent connections)\n"), N_("\ --no-cookies don't use cookies\n"), N_("\ --load-cookies=FILE load cookies from FILE before session\n"), N_("\ --save-cookies=FILE save cookies to FILE after session\n"), N_("\ --keep-session-cookies load and save session (non-permanent) cookies\n"), N_("\ --post-data=STRING use the POST method; send STRING as the data\n"), N_("\ --post-file=FILE use the POST method; send contents of FILE\n"), N_("\ --method=HTTPMethod use method \"HTTPMethod\" in the request\n"), N_("\ --body-data=STRING send STRING as data. --method MUST be set\n"), N_("\ --body-file=FILE send contents of FILE. --method MUST be set\n"), N_("\ --content-disposition honor the Content-Disposition header when\n\ choosing local file names (EXPERIMENTAL)\n"), N_("\ --content-on-error output the received content on server errors\n"), N_("\ --auth-no-challenge send Basic HTTP authentication information\n\ without first waiting for the server's\n\ challenge\n"), "\n", #ifdef HAVE_SSL N_("\ HTTPS (SSL/TLS) options:\n"), N_("\ --secure-protocol=PR choose secure protocol, one of auto, SSLv2,\n\ SSLv3, TLSv1, TLSv1_1, TLSv1_2 and PFS\n"), N_("\ --https-only only follow secure HTTPS links\n"), N_("\ --no-check-certificate don't validate the server's certificate\n"), N_("\ --certificate=FILE client certificate file\n"), N_("\ --certificate-type=TYPE client certificate type, PEM or DER\n"), N_("\ --private-key=FILE private key file\n"), N_("\ --private-key-type=TYPE private key type, PEM or DER\n"), N_("\ --ca-certificate=FILE file with the bundle of CAs\n"), N_("\ --ca-directory=DIR directory where hash list of CAs is stored\n"), N_("\ --crl-file=FILE file with bundle of CRLs\n"), N_("\ --pinnedpubkey=FILE/HASHES Public key (PEM/DER) file, or any number\n\ of base64 encoded sha256 hashes preceded by\n\ \'sha256//\' and separated by \';\', to verify\n\ peer against\n"), #if defined(HAVE_LIBSSL) || defined(HAVE_LIBSSL32) N_("\ --random-file=FILE file with random data for seeding the SSL PRNG\n"), #endif #if (defined(HAVE_LIBSSL) || defined(HAVE_LIBSSL32)) && defined(HAVE_RAND_EGD) N_("\ --egd-file=FILE file naming the EGD socket with random data\n"), #endif "\n", N_("\ --ciphers=STR Set the priority string (GnuTLS) or cipher list string (OpenSSL) directly.\n\ Use with care. This option overrides --secure-protocol.\n\ The format and syntax of this string depend on the specific SSL/TLS engine.\n"), #endif /* HAVE_SSL */ #ifdef HAVE_HSTS N_("\ HSTS options:\n"), N_("\ --no-hsts disable HSTS\n"), N_("\ --hsts-file path of HSTS database (will override default)\n"), "\n", #endif N_("\ FTP options:\n"), #ifdef __VMS N_("\ --ftp-stmlf use Stream_LF format for all binary FTP files\n"), #endif /* def __VMS */ N_("\ --ftp-user=USER set ftp user to USER\n"), N_("\ --ftp-password=PASS set ftp password to PASS\n"), N_("\ --no-remove-listing don't remove '.listing' files\n"), N_("\ --no-glob turn off FTP file name globbing\n"), N_("\ --no-passive-ftp disable the \"passive\" transfer mode\n"), N_("\ --preserve-permissions preserve remote file permissions\n"), N_("\ --retr-symlinks when recursing, get linked-to files (not dir)\n"), "\n", #ifdef HAVE_SSL N_("\ FTPS options:\n"), N_("\ --ftps-implicit use implicit FTPS (default port is 990)\n"), N_("\ --ftps-resume-ssl resume the SSL/TLS session started in the control connection when\n" " opening a data connection\n"), N_("\ --ftps-clear-data-connection cipher the control channel only; all the data will be in plaintext\n"), N_("\ --ftps-fallback-to-ftp fall back to FTP if FTPS is not supported in the target server\n"), #endif N_("\ WARC options:\n"), N_("\ --warc-file=FILENAME save request/response data to a .warc.gz file\n"), N_("\ --warc-header=STRING insert STRING into the warcinfo record\n"), N_("\ --warc-max-size=NUMBER set maximum size of WARC files to NUMBER\n"), N_("\ --warc-cdx write CDX index files\n"), N_("\ --warc-dedup=FILENAME do not store records listed in this CDX file\n"), #ifdef HAVE_LIBZ N_("\ --no-warc-compression do not compress WARC files with GZIP\n"), #endif N_("\ --no-warc-digests do not calculate SHA1 digests\n"), N_("\ --no-warc-keep-log do not store the log file in a WARC record\n"), N_("\ --warc-tempdir=DIRECTORY location for temporary files created by the\n\ WARC writer\n"), "\n", N_("\ Recursive download:\n"), N_("\ -r, --recursive specify recursive download\n"), N_("\ -l, --level=NUMBER maximum recursion depth (inf or 0 for infinite)\n"), N_("\ --delete-after delete files locally after downloading them\n"), N_("\ -k, --convert-links make links in downloaded HTML or CSS point to\n\ local files\n"), N_("\ --convert-file-only convert the file part of the URLs only (usually known as the basename)\n"), N_("\ --backups=N before writing file X, rotate up to N backup files\n"), #ifdef __VMS N_("\ -K, --backup-converted before converting file X, back up as X_orig\n"), #else /* def __VMS */ N_("\ -K, --backup-converted before converting file X, back up as X.orig\n"), #endif /* def __VMS [else] */ N_("\ -m, --mirror shortcut for -N -r -l inf --no-remove-listing\n"), N_("\ -p, --page-requisites get all images, etc. needed to display HTML page\n"), N_("\ --strict-comments turn on strict (SGML) handling of HTML comments\n"), "\n", N_("\ Recursive accept/reject:\n"), N_("\ -A, --accept=LIST comma-separated list of accepted extensions\n"), N_("\ -R, --reject=LIST comma-separated list of rejected extensions\n"), N_("\ --accept-regex=REGEX regex matching accepted URLs\n"), N_("\ --reject-regex=REGEX regex matching rejected URLs\n"), #if defined HAVE_LIBPCRE || defined HAVE_LIBPCRE2 N_("\ --regex-type=TYPE regex type (posix|pcre)\n"), #else N_("\ --regex-type=TYPE regex type (posix)\n"), #endif N_("\ -D, --domains=LIST comma-separated list of accepted domains\n"), N_("\ --exclude-domains=LIST comma-separated list of rejected domains\n"), N_("\ --follow-ftp follow FTP links from HTML documents\n"), N_("\ --follow-tags=LIST comma-separated list of followed HTML tags\n"), N_("\ --ignore-tags=LIST comma-separated list of ignored HTML tags\n"), N_("\ -H, --span-hosts go to foreign hosts when recursive\n"), N_("\ -L, --relative follow relative links only\n"), N_("\ -I, --include-directories=LIST list of allowed directories\n"), N_("\ --trust-server-names use the name specified by the redirection\n\ URL's last component\n"), N_("\ -X, --exclude-directories=LIST list of excluded directories\n"), N_("\ -np, --no-parent don't ascend to the parent directory\n"), "\n", N_("Email bug reports, questions, discussions to \n" "and/or open issues at https://savannah.gnu.org/bugs/?func=additem&group=wget.\n") }; size_t i; if (printf (_("GNU Wget %s, a non-interactive network retriever.\n"), version_string) < 0) exit (WGET_EXIT_IO_FAIL); if (print_usage (0) < 0) exit (WGET_EXIT_IO_FAIL); for (i = 0; i < countof (help); i++) if (fputs (_(help[i]), stdout) < 0) exit (WGET_EXIT_IO_FAIL); #endif /* TESTING */ exit (WGET_EXIT_SUCCESS); } /* Return a human-readable printed representation of INTERVAL, measured in seconds. */ static char * secs_to_human_time (double interval) { static char buf[32]; int secs = (int) (interval + 0.5); int hours, mins, days; days = secs / 86400, secs %= 86400; hours = secs / 3600, secs %= 3600; mins = secs / 60, secs %= 60; if (days) sprintf (buf, "%dd %dh %dm %ds", days, hours, mins, secs); else if (hours) sprintf (buf, "%dh %dm %ds", hours, mins, secs); else if (mins) sprintf (buf, "%dm %ds", mins, secs); else sprintf (buf, "%ss", print_decimal (interval)); return buf; } static char * prompt_for_password (void) { if (opt.user) fprintf (stderr, _("Password for user %s: "), quote (opt.user)); else fprintf (stderr, _("Password: ")); #ifndef TESTING /* gnulib's getpass() uses static variables internally, bad for fuzing */ return getpass(""); #else return xstrdup(""); #endif } /* Execute external application opt.use_askpass */ static void run_use_askpass (char *question, char **answer) { char tmp[1024]; pid_t pid; int status; int com[2]; ssize_t bytes = 0; char *argv[3], *p; posix_spawn_file_actions_t fa; if (pipe (com) == -1) { fprintf (stderr, _("Cannot create pipe\n")); exit (WGET_EXIT_GENERIC_ERROR); } status = posix_spawn_file_actions_init (&fa); if (status) { fprintf (stderr, _("Error initializing spawn file actions for use-askpass: %d\n"), status); exit (WGET_EXIT_GENERIC_ERROR); } status = posix_spawn_file_actions_adddup2 (&fa, com[1], STDOUT_FILENO); if (status) { fprintf (stderr, _("Error setting spawn file actions for use-askpass: %d\n"), status); exit (WGET_EXIT_GENERIC_ERROR); } /* C89 initializer lists must be computable at load time, * thus this explicit initialization. */ argv[0] = opt.use_askpass; argv[1] = question; argv[2] = NULL; status = posix_spawnp (&pid, opt.use_askpass, &fa, NULL, argv, environ); if (status) { fprintf (stderr, "Error spawning %s: %d\n", opt.use_askpass, status); exit (WGET_EXIT_GENERIC_ERROR); } /* Parent process reads from child. */ close (com[1]); bytes = read (com[0], tmp, sizeof (tmp) - 1); if (bytes <= 0) { fprintf (stderr, _("Error reading response from command \"%s %s\": %s\n"), opt.use_askpass, question, strerror (errno)); exit (WGET_EXIT_GENERIC_ERROR); } /* Make sure there is a trailing 0 */ tmp[bytes] = '\0'; /* Remove a possible new line */ if ((p = strpbrk (tmp, "\r\n"))) bytes = p - tmp; *answer = xmemdup0 (tmp, bytes); } /* set the user name and password*/ static void use_askpass (struct url *u) { static char question[1024]; if (u->user == NULL || u->user[0] == '\0') { snprintf (question, sizeof (question), _("Username for '%s%s': "), scheme_leading_string(u->scheme), u->host); /* Prompt for username */ run_use_askpass (question, &u->user); if (opt.recursive) opt.user = xstrdup (u->user); } if (u->passwd == NULL || u->passwd[0] == '\0') { snprintf(question, sizeof (question), _("Password for '%s%s@%s': "), scheme_leading_string (u->scheme), u->user, u->host); /* Prompt for password */ run_use_askpass (question, &u->passwd); if (opt.recursive) opt.passwd = xstrdup (u->passwd); } } /* Function that prints the line argument while limiting it to at most line_length. prefix is printed on the first line and an appropriate number of spaces are added on subsequent lines.*/ static int format_and_print_line (const char *prefix, const char *line, int line_length) { int remaining_chars; char *line_dup, *token; assert (prefix != NULL); assert (line != NULL); assert (line_length > TABULATION); line_dup = xstrdup (line); if (printf ("%s", prefix) < 0) { xfree (line_dup); return -1; } /* Wrap to new line after prefix. */ remaining_chars = 0; /* We break on spaces. */ token = strtok (line_dup, " "); while (token != NULL) { /* If however a token is much larger than the maximum line length, all bets are off and we simply print the token on the next line. */ if (remaining_chars <= (int) strlen (token)) { if (printf ("\n%*c", TABULATION, ' ') < 0) { xfree (line_dup); return -1; } remaining_chars = line_length - TABULATION; } if (printf ("%s ", token) < 0) { xfree (line_dup); return -1; } remaining_chars -= strlen (token) + 1; /* account for " " */ token = strtok (NULL, " "); } if (printf ("\n") < 0) { xfree (line_dup); return -1; } xfree (line_dup); return 0; } _Noreturn static void print_version (void) { const char *wgetrc_title = _("Wgetrc: "); const char *locale_title = _("Locale: "); const char *compile_title = _("Compile: "); const char *link_title = _("Link: "); char *env_wgetrc, *user_wgetrc; int i; if (printf (_("GNU Wget %s built on %s.\n\n"), version_string, OS_TYPE) < 0) exit (WGET_EXIT_IO_FAIL); for (i = 0; compiled_features[i] != NULL; ) { int line_length = MAX_CHARS_PER_LINE; while ((line_length > 0) && (compiled_features[i] != NULL)) { if (printf ("%s ", compiled_features[i]) < 0) exit (WGET_EXIT_IO_FAIL); line_length -= (int) strlen (compiled_features[i]) + 2; i++; } if (printf ("\n") < 0) exit (WGET_EXIT_IO_FAIL); } if (printf ("\n") < 0) exit (WGET_EXIT_IO_FAIL); /* Print VMS-specific version info. */ #ifdef __VMS if (vms_version_supplement() < 0) exit (WGET_EXIT_IO_FAIL); #endif /* def __VMS */ /* Handle the case when $WGETRC is unset and $HOME/.wgetrc is absent. */ if (printf ("%s\n", wgetrc_title) < 0) exit (WGET_EXIT_IO_FAIL); env_wgetrc = wgetrc_env_file_name (); if (env_wgetrc && *env_wgetrc) { if (printf (_(" %s (env)\n"), env_wgetrc) < 0) exit (WGET_EXIT_IO_FAIL); xfree (env_wgetrc); } user_wgetrc = wgetrc_user_file_name (); if (user_wgetrc) { if (printf (_(" %s (user)\n"), user_wgetrc) < 0) exit (WGET_EXIT_IO_FAIL); xfree (user_wgetrc); } #ifdef SYSTEM_WGETRC if (printf (_(" %s (system)\n"), SYSTEM_WGETRC) < 0) exit (WGET_EXIT_IO_FAIL); #endif #ifdef ENABLE_NLS if (format_and_print_line (locale_title, LOCALEDIR, MAX_CHARS_PER_LINE) < 0) exit (WGET_EXIT_IO_FAIL); #endif /* def ENABLE_NLS */ if (compilation_string != NULL) if (format_and_print_line (compile_title, compilation_string, MAX_CHARS_PER_LINE) < 0) exit (WGET_EXIT_IO_FAIL); if (link_string != NULL) if (format_and_print_line (link_title, link_string, MAX_CHARS_PER_LINE) < 0) exit (WGET_EXIT_IO_FAIL); if (printf ("\n") < 0) exit (WGET_EXIT_IO_FAIL); /* TRANSLATORS: When available, an actual copyright character (circle-c) should be used in preference to "(C)". */ if (printf (_("\ Copyright (C) %s Free Software Foundation, Inc.\n"), "2015") < 0) exit (WGET_EXIT_IO_FAIL); if (fputs (_("\ License GPLv3+: GNU GPL version 3 or later\n\ .\n\ This is free software: you are free to change and redistribute it.\n\ There is NO WARRANTY, to the extent permitted by law.\n"), stdout) < 0) exit (WGET_EXIT_IO_FAIL); /* TRANSLATORS: When available, please use the proper diacritics for names such as this one. See en_US.po for reference. */ if (fputs (_("\nOriginally written by Hrvoje Niksic .\n"), stdout) < 0) exit (WGET_EXIT_IO_FAIL); if (fputs (_("Please send bug reports and questions to .\n"), stdout) < 0) exit (WGET_EXIT_IO_FAIL); exit (WGET_EXIT_SUCCESS); } const char *program_name; /* Needed by lib/error.c. */ const char *program_argstring; /* Needed by wget_warc.c. */ struct ptimer *timer; int cleaned_up; int main (int argc, char **argv) { char *p; int i, ret, longindex; int nurls; int retconf; int argstring_length; bool use_userconfig = false; bool noconfig = false; bool append_to_log = false; cleaned_up = 0; /* do cleanup later */ timer = ptimer_new (); double start_time = ptimer_measure (timer); total_downloaded_bytes = 0; program_name = argv[0]; i18n_initialize (); /* Construct the name of the executable, without the directory part. */ #ifdef __VMS /* On VMS, lose the "dev:[dir]" prefix and the ".EXE;nnn" suffix. */ exec_name = vms_basename (argv[0]); #else /* def __VMS */ exec_name = base_name (argv[0]); #endif /* def __VMS [else] */ #ifdef WINDOWS /* Drop extension (typically .EXE) from executable filename. */ windows_main ((char **) &exec_name); #endif /* Construct the arguments string. */ for (argstring_length = 1, i = 1; i < argc; i++) argstring_length += strlen (argv[i]) + 3 + 1; program_argstring = p = malloc (argstring_length); if (p == NULL) { fprintf (stderr, _("Memory allocation problem\n")); exit (WGET_EXIT_PARSE_ERROR); } for (i = 1; i < argc; i++) { int arglen; *p++ = '"'; arglen = strlen (argv[i]); memcpy (p, argv[i], arglen); p += arglen; *p++ = '"'; *p++ = ' '; } *p = '\0'; /* Load the hard-coded defaults. */ defaults (); opt.homedir = home_dir(); init_switches (); /* This separate getopt_long is needed to find the user config file option ("--config") and parse it before the other user options. */ longindex = -1; while ((getopt_long (argc, argv, short_options, long_options, &longindex)) != -1) { int confval; struct cmdline_option *config_opt; /* There is no short option for "--config". */ if (longindex >= 0) { confval = long_options[longindex].val; config_opt = &option_data[confval & ~BOOLEAN_NEG_MARKER]; if (strcmp (config_opt->long_name, "no-config") == 0) { noconfig = true; break; } else if (strcmp (config_opt->long_name, "config") == 0) { file_stats_t flstats; use_userconfig = true; memset(&flstats, 0, sizeof(flstats)); if (file_exists_p(optarg, &flstats) && run_wgetrc (optarg, &flstats)) break; else { fprintf (stderr, _("Exiting due to error in %s\n"), optarg); exit (WGET_EXIT_PARSE_ERROR); } } } } /* If the user did not specify a config, read the system wgetrc and ~/.wgetrc. */ if (noconfig == false && use_userconfig == false) if ((ret = initialize ())) return ret; opterr = 0; optind = 0; longindex = -1; while ((ret = getopt_long (argc, argv, short_options, long_options, &longindex)) != -1) { int val; struct cmdline_option *cmdopt; /* If LONGINDEX is unchanged, it means RET is referring a short option. */ if (longindex == -1) { if (ret == '?') { print_usage (1); fprintf (stderr, "\n"); fprintf (stderr, _("Try `%s --help' for more options.\n"), exec_name); exit (WGET_EXIT_PARSE_ERROR); } /* Find the short option character in the mapping. */ longindex = optmap[ret - 32]; } val = long_options[longindex].val; /* Use the retrieved value to locate the option in the option_data array, and to see if we're dealing with the negated "--no-FOO" variant of the boolean option "--foo". */ cmdopt = &option_data[val & ~BOOLEAN_NEG_MARKER]; switch (cmdopt->type) { case OPT_VALUE: setoptval (cmdopt->data, optarg, cmdopt->long_name); break; case OPT_BOOLEAN: if (optarg) /* The user has specified a value -- use it. */ setoptval (cmdopt->data, optarg, cmdopt->long_name); else { /* NEG is true for `--no-FOO' style boolean options. */ bool neg = !!(val & BOOLEAN_NEG_MARKER); setoptval (cmdopt->data, neg ? "0" : "1", cmdopt->long_name); } break; case OPT_FUNCALL: { void (*func) (void) = (void (*) (void)) cmdopt->data; func (); } break; case OPT__APPEND_OUTPUT: setoptval ("logfile", optarg, cmdopt->long_name); append_to_log = true; break; case OPT__EXECUTE: if (optarg) /* check silences static analyzer */ run_command (optarg); break; case OPT__NO: { /* We support real --no-FOO flags now, but keep these short options for convenience and backward compatibility. */ for (p = optarg; p && *p; p++) switch (*p) { case 'v': setoptval ("verbose", "0", cmdopt->long_name); break; case 'H': setoptval ("addhostdir", "0", cmdopt->long_name); break; case 'd': setoptval ("dirstruct", "0", cmdopt->long_name); break; case 'c': setoptval ("noclobber", "1", cmdopt->long_name); break; case 'p': setoptval ("noparent", "1", cmdopt->long_name); break; default: fprintf (stderr, _("%s: illegal option -- `-n%c'\n"), exec_name, *p); print_usage (1); fprintf (stderr, "\n"); fprintf (stderr, _("Try `%s --help' for more options.\n"), exec_name); exit (WGET_EXIT_GENERIC_ERROR); } break; } case OPT__PARENT: case OPT__CLOBBER: { /* The wgetrc commands are named noparent and noclobber, so we must revert the meaning of the cmdline options before passing the value to setoptval. */ bool flag = true; if (optarg) flag = (*optarg == '1' || c_tolower (*optarg) == 'y' || (c_tolower (optarg[0]) == 'o' && c_tolower (optarg[1]) == 'n')); setoptval (cmdopt->type == OPT__PARENT ? "noparent" : "noclobber", flag ? "0" : "1", cmdopt->long_name); break; } case OPT__DONT_REMOVE_LISTING: setoptval ("removelisting", "0", cmdopt->long_name); break; } longindex = -1; } nurls = argc - optind; /* Initialize logging ASAP. */ log_init (opt.lfilename, append_to_log); /* If we do not have Debug support compiled in AND Wget is invoked with the * --debug switch, instead of failing, we silently turn it into a no-op. For * this no-op, we explicitly set opt.debug to false and hence none of the * Debug output messages will be printed. */ #ifndef ENABLE_DEBUG if (opt.debug) { fprintf (stderr, _("Debugging support not compiled in. " "Ignoring --debug flag.\n")); opt.debug = false; } #endif /* All user options have now been processed, so it's now safe to do interoption dependency checks. */ if (opt.noclobber && (opt.convert_links || opt.convert_file_only)) { fprintf (stderr, opt.convert_links ? _("Both --no-clobber and --convert-links were specified," " only --convert-links will be used.\n") : _("Both --no-clobber and --convert-file-only were specified," " only --convert-file-only will be used.\n")); opt.noclobber = false; } if (opt.reclevel == 0) opt.reclevel = INFINITE_RECURSION; /* see recur.h for commentary */ if (opt.spider || opt.delete_after) opt.no_dirstruct = true; if (opt.page_requisites && !opt.recursive) { /* Don't set opt.recursive here because it would confuse the FTP code. Instead, call retrieve_tree below when either page_requisites or recursive is requested. */ opt.reclevel = 0; if (!opt.no_dirstruct) opt.dirstruct = 1; /* normally handled by cmd_spec_recursive() */ } if (opt.verbose == -1) opt.verbose = !opt.quiet; if (!opt.verbose && opt.show_progress == -1) opt.show_progress = false; if (opt.quiet && opt.show_progress == -1) opt.show_progress = false; /* Sanity checks. */ if (opt.verbose && opt.quiet) { fprintf (stderr, _("Can't be verbose and quiet at the same time.\n")); print_usage (1); exit (WGET_EXIT_GENERIC_ERROR); } if (opt.timestamping && opt.noclobber) { fprintf (stderr, _("\ Can't timestamp and not clobber old files at the same time.\n")); print_usage (1); exit (WGET_EXIT_GENERIC_ERROR); } #ifdef ENABLE_IPV6 if (opt.ipv4_only && opt.ipv6_only) { fprintf (stderr, _("Cannot specify both --inet4-only and --inet6-only.\n")); print_usage (1); exit (WGET_EXIT_GENERIC_ERROR); } #endif if (opt.output_document) { if ((opt.convert_links || opt.convert_file_only) && (nurls > 1 || opt.page_requisites || opt.recursive)) { fputs (_("\ Cannot specify both -k or --convert-file-only and -O if multiple URLs are given, or in combination\n\ with -p or -r. See the manual for details.\n\n"), stderr); print_usage (1); exit (WGET_EXIT_GENERIC_ERROR); } if (opt.page_requisites || opt.recursive) { logprintf (LOG_NOTQUIET, "%s", _("\ WARNING: combining -O with -r or -p will mean that all downloaded content\n\ will be placed in the single file you specified.\n\n")); } if (opt.timestamping) { logprintf (LOG_NOTQUIET, "%s", _("\ WARNING: timestamping does nothing in combination with -O. See the manual\n\ for details.\n\n")); opt.timestamping = false; } if (opt.noclobber && file_exists_p(opt.output_document, NULL)) { /* Check if output file exists; if it does, exit. */ logprintf (LOG_VERBOSE, _("File %s already there; not retrieving.\n"), quote (opt.output_document)); exit (WGET_EXIT_GENERIC_ERROR); } } if (opt.warc_filename != 0) { if (opt.noclobber) { fprintf (stderr, _("WARC output does not work with --no-clobber, " "--no-clobber will be disabled.\n")); opt.noclobber = false; } if (opt.timestamping) { fprintf (stderr, _("WARC output does not work with timestamping, " "timestamping will be disabled.\n")); opt.timestamping = false; } if (opt.spider) { fprintf (stderr, _("WARC output does not work with --spider.\n")); exit (WGET_EXIT_GENERIC_ERROR); } if (opt.always_rest || opt.start_pos >= 0) { fprintf (stderr, _("WARC output does not work with --continue or" " --start-pos, they will be disabled.\n")); opt.always_rest = false; opt.start_pos = -1; } if (opt.warc_cdx_dedup_filename != 0 && !opt.warc_digests_enabled) { fprintf (stderr, _("Digests are disabled; WARC deduplication will " "not find duplicate records.\n")); } if (opt.warc_keep_log) { opt.progress_type = xstrdup ("dot"); } } #ifdef HAVE_LIBZ if (opt.always_rest || opt.start_pos >= 0) { if (opt.compression == compression_auto) { /* Compression does not work with --continue or --start-pos. Since compression was not explicitly set, it will be disabled. */ opt.compression = compression_none; } else if (opt.compression != compression_none) { fprintf (stderr, _("Compression does not work with --continue or" " --start-pos, they will be disabled.\n")); opt.always_rest = false; opt.start_pos = -1; } } #endif if (opt.ask_passwd && opt.passwd) { fprintf (stderr, _("Cannot specify both --ask-password and --password.\n")); print_usage (1); exit (WGET_EXIT_GENERIC_ERROR); } if (opt.ask_passwd && !(opt.user || opt.http_user || opt.ftp_user)) { fprintf(stderr, _("WARNING: No username set with --ask-password. This is usually not what you want.\n")); } if (opt.start_pos >= 0 && opt.always_rest) { fprintf (stderr, _("Specifying both --start-pos and --continue is not " "recommended; --continue will be disabled.\n")); opt.always_rest = false; } if (!nurls && !opt.input_filename #ifdef HAVE_METALINK && !opt.input_metalink #endif ) { /* No URL specified. */ #ifndef TESTING fprintf (stderr, _("%s: missing URL\n"), exec_name); print_usage (1); fprintf (stderr, "\n"); /* #### Something nicer should be printed here -- similar to the pre-1.5 `--help' page. */ fprintf (stderr, _("Try `%s --help' for more options.\n"), exec_name); #endif exit (WGET_EXIT_GENERIC_ERROR); } /* Compile the regular expressions. */ switch (opt.regex_type) { #ifdef HAVE_LIBPCRE2 case regex_type_pcre: opt.regex_compile_fun = compile_pcre2_regex; opt.regex_match_fun = match_pcre2_regex; break; #endif #ifdef HAVE_LIBPCRE case regex_type_pcre: opt.regex_compile_fun = compile_pcre_regex; opt.regex_match_fun = match_pcre_regex; break; #endif case regex_type_posix: default: opt.regex_compile_fun = compile_posix_regex; opt.regex_match_fun = match_posix_regex; break; } if (opt.acceptregex_s) { opt.acceptregex = opt.regex_compile_fun (opt.acceptregex_s); if (!opt.acceptregex) exit (WGET_EXIT_GENERIC_ERROR); } if (opt.rejectregex_s) { opt.rejectregex = opt.regex_compile_fun (opt.rejectregex_s); if (!opt.rejectregex) exit (WGET_EXIT_GENERIC_ERROR); } if (opt.post_data || opt.post_file_name) { if (opt.post_data && opt.post_file_name) { fprintf (stderr, _("You cannot specify both --post-data and --post-file.\n")); exit (WGET_EXIT_GENERIC_ERROR); } else if (opt.method) { fprintf (stderr, _("You cannot use --post-data or --post-file along with --method. " "--method expects data through --body-data and --body-file options\n")); exit (WGET_EXIT_GENERIC_ERROR); } } if (opt.body_data || opt.body_file) { if (!opt.method) { fprintf (stderr, _("You must specify a method through --method=HTTPMethod " "to use with --body-data or --body-file.\n")); exit (WGET_EXIT_GENERIC_ERROR); } else if (opt.body_data && opt.body_file) { fprintf (stderr, _("You cannot specify both --body-data and --body-file.\n")); exit (WGET_EXIT_GENERIC_ERROR); } } /* Set various options as required for opt.method. */ /* When user specifies HEAD as the method, we do not wish to download any files. Hence, set wget to run in spider mode. */ if (opt.method && c_strcasecmp (opt.method, "HEAD") == 0) setoptval ("spider", "1", "spider"); /* Convert post_data to body-data and post_file_name to body-file options. This is required so as to remove redundant code later on in gethttp(). The --post-data and --post-file options may also be removed in the future hence it makes sense to convert them to aliases for the more generic --method options. This MUST occur only after the sanity checks so as to prevent the user from setting both post and body options simultaneously. */ if (opt.post_data || opt.post_file_name) { setoptval ("method", "POST", "method"); if (opt.post_data) { setoptval ("bodydata", opt.post_data, "body-data"); xfree(opt.post_data); } else { setoptval ("bodyfile", opt.post_file_name, "body-file"); xfree(opt.post_file_name); } } #ifdef ENABLE_IRI if (opt.enable_iri) { if (opt.locale && !check_encoding_name (opt.locale)) xfree (opt.locale); if (!opt.locale) opt.locale = find_locale (); if (opt.encoding_remote && !check_encoding_name (opt.encoding_remote)) xfree (opt.encoding_remote); } #else memset (&dummy_iri, 0, sizeof (dummy_iri)); if (opt.enable_iri || opt.locale || opt.encoding_remote) { /* sXXXav : be more specific... */ fprintf (stderr, _("This version does not have support for IRIs\n")); exit (WGET_EXIT_GENERIC_ERROR); } #endif if (opt.ask_passwd) { opt.passwd = prompt_for_password (); if (opt.passwd == NULL || opt.passwd[0] == '\0') exit (WGET_EXIT_GENERIC_ERROR); } if (opt.use_askpass) { if (opt.use_askpass[0] == '\0') { fprintf (stderr, _("use-askpass requires a string or either environment variable WGET_ASKPASS or SSH_ASKPASS to be set.\n")); exit (WGET_EXIT_GENERIC_ERROR); } } #ifdef USE_WATT32 if (opt.wdebug) dbug_init(); sock_init(); #elif ! defined TESTING if (opt.background) { bool logfile_changed = fork_to_background (); if (logfile_changed) log_init (opt.lfilename, append_to_log); } #endif /* Initialize progress. Have to do this after the options are processed so we know where the log file is. */ if (opt.show_progress) set_progress_implementation (opt.progress_type); /* Open WARC file. */ if (opt.warc_filename != 0) warc_init (); DEBUGP (("DEBUG output created by Wget %s on %s.\n\n", version_string, OS_TYPE)); /* Open the output filename if necessary. */ /* 2005-04-17 SMS. Note that having the output_stream ("-O") file opened here for an FTP URL rather than in getftp() (ftp.c) (and the http equivalent) rather limits the ability in VMS to open the file differently for ASCII versus binary FTP there. (Of course, doing it here allows a open failure to be detected immediately, without first connecting to the server.) */ if (opt.output_document) { if (HYPHENP (opt.output_document)) { #ifdef WINDOWS _setmode (_fileno (stdout), _O_BINARY); #endif output_stream = stdout; } else { struct stat st; #ifdef __VMS /* Common fopen() optional arguments: sequential access only, access callback function. */ # define FOPEN_OPT_ARGS , "fop=sqo", "acc", acc_cb, &open_id int open_id = 7; #else /* def __VMS */ # define FOPEN_OPT_ARGS #endif /* def __VMS [else] */ output_stream = fopen (opt.output_document, opt.always_rest ? "ab" : "wb" FOPEN_OPT_ARGS); if (output_stream == NULL) { perror (opt.output_document); exit (WGET_EXIT_GENERIC_ERROR); } if (fstat (fileno (output_stream), &st) == 0 && S_ISREG (st.st_mode)) output_stream_regular = true; } if (!output_stream_regular && (opt.convert_links || opt.recursive)) { fprintf (stderr, _("-k or -r can be used together with -O only if \ outputting to a regular file.\n")); exit (WGET_EXIT_GENERIC_ERROR); } if (!output_stream_regular && (opt.convert_links || opt.convert_file_only)) { fprintf (stderr, _("--convert-links or --convert-file-only can be used together \ only if outputting to a regular file.\n")); exit (WGET_EXIT_GENERIC_ERROR); } } #ifdef HAVE_LIBCARES if (opt.bind_dns_address || opt.dns_servers) { if (ares_library_init (ARES_LIB_INIT_ALL)) { fprintf (stderr, _("Failed to init libcares\n")); exit (WGET_EXIT_GENERIC_ERROR); } if (ares_init (&ares) != ARES_SUCCESS) { fprintf (stderr, _("Failed to init c-ares channel\n")); exit (WGET_EXIT_GENERIC_ERROR); } if (opt.bind_dns_address) { struct in_addr a4; #ifdef ENABLE_IPV6 struct in6_addr a6; #endif if (inet_pton (AF_INET, opt.bind_dns_address, &a4) == 1) { ares_set_local_ip4 (ares, ntohl (a4.s_addr)); } #ifdef ENABLE_IPV6 else if (inet_pton (AF_INET6, opt.bind_dns_address, &a6) == 1) { ares_set_local_ip6 (ares, (unsigned char *) &a6); } #endif else { fprintf (stderr, _("Failed to parse IP address '%s'\n"), opt.bind_dns_address); exit (WGET_EXIT_GENERIC_ERROR); } } if (opt.dns_servers) { int result; if ((result = ares_set_servers_csv (ares, opt.dns_servers)) != ARES_SUCCESS) { fprintf (stderr, _("Failed to set DNS server(s) '%s' (%d)\n"), opt.dns_servers, result); exit (WGET_EXIT_GENERIC_ERROR); } } } #endif #ifdef __VMS /* Set global ODS5 flag according to the specified destination (if any), otherwise according to the current default device. */ if (output_stream == NULL) set_ods5_dest( "SYS$DISK"); else if (output_stream != stdout) set_ods5_dest( opt.output_document); #endif /* def __VMS */ #ifdef WINDOWS ws_startup (); #endif #ifdef SIGHUP /* Setup the signal handler to redirect output when hangup is received. */ if (signal(SIGHUP, SIG_IGN) != SIG_IGN) signal(SIGHUP, redirect_output_signal); #endif /* ...and do the same for SIGUSR1. */ #ifdef SIGUSR1 signal (SIGUSR1, redirect_output_signal); #endif #ifdef SIGPIPE /* Writing to a closed socket normally signals SIGPIPE, and the process exits. What we want is to ignore SIGPIPE and just check for the return value of write(). */ signal (SIGPIPE, SIG_IGN); #endif #ifdef SIGWINCH signal (SIGWINCH, progress_handle_sigwinch); #endif #ifdef HAVE_HSTS /* Load the HSTS database. Maybe all the URLs are FTP(S), in which case HSTS would not be needed, but this is the best place to do it, and it shouldn't be a critical performance hit. */ if (opt.hsts) load_hsts (); #endif /* Retrieve the URLs from argument list. */ for (i = 0; i < nurls; i++, optind++) { char *t; char *filename = NULL, *redirected_URL = NULL; int dt = 0, url_err; /* Need to do a new struct iri every time, because * retrieve_url may modify it in some circumstances, * currently. */ struct iri *iri = iri_new (); struct url *url_parsed; t = rewrite_shorthand_url (argv[optind]); if (!t) t = argv[optind]; set_uri_encoding (iri, opt.locale, true); url_parsed = url_parse (t, &url_err, iri, true); if (!url_parsed) { char *error = url_error (t, url_err); logprintf (LOG_NOTQUIET, "%s: %s.\n",t, error); xfree (error); inform_exit_status (URLERROR); } else { /* Request credentials if use_askpass is set. */ if (opt.use_askpass) use_askpass (url_parsed); if ((opt.recursive || opt.page_requisites) && ((url_scheme (t) != SCHEME_FTP #ifdef HAVE_SSL && url_scheme (t) != SCHEME_FTPS #endif ) || url_uses_proxy (url_parsed))) { int old_follow_ftp = opt.follow_ftp; /* Turn opt.follow_ftp on in case of recursive FTP retrieval */ if (url_scheme (t) == SCHEME_FTP #ifdef HAVE_SSL || url_scheme (t) == SCHEME_FTPS #endif ) opt.follow_ftp = 1; retrieve_tree (url_parsed, NULL); opt.follow_ftp = old_follow_ftp; } else { retrieve_url (url_parsed, t, &filename, &redirected_URL, NULL, &dt, opt.recursive, iri, true); } if (opt.delete_after && filename != NULL && file_exists_p (filename, NULL)) { DEBUGP (("Removing file due to --delete-after in main():\n")); logprintf (LOG_VERBOSE, _("Removing %s.\n"), filename); if (unlink (filename)) logprintf (LOG_NOTQUIET, "unlink: %s\n", strerror (errno)); } xfree (redirected_URL); xfree (filename); url_free (url_parsed); } iri_free (iri); if (t != argv[optind]) xfree (t); } /* And then from the input file, if any. */ if (opt.input_filename) { int count; int status; status = retrieve_from_file (opt.input_filename, opt.force_html, &count); inform_exit_status (status); if (!count) logprintf (LOG_NOTQUIET, _("No URLs found in %s.\n"), opt.input_filename); } #ifdef HAVE_METALINK /* Finally, from metlink file, if any. */ if (opt.input_metalink) { metalink_error_t meta_err; uerr_t retr_err; metalink_t *metalink; meta_err = metalink_parse_file (opt.input_metalink, &metalink); if (meta_err) { logprintf (LOG_NOTQUIET, _("Unable to parse metalink file %s.\n"), opt.input_metalink); retr_err = METALINK_PARSE_ERROR; } else { /* We need to sort the resources if preferred location was specified by the user. */ if (opt.preferred_location && opt.preferred_location[0]) { metalink_file_t **mfile_ptr; for (mfile_ptr = metalink->files; *mfile_ptr; mfile_ptr++) { metalink_resource_t **mres_ptr; metalink_file_t *mfile = *mfile_ptr; size_t mres_count = 0; for (mres_ptr = mfile->resources; *mres_ptr; mres_ptr++) mres_count++; stable_sort (mfile->resources, mres_count, sizeof (metalink_resource_t *), metalink_res_cmp); } } retr_err = retrieve_from_metalink (metalink); if (retr_err != RETROK) { logprintf (LOG_NOTQUIET, _("Could not download all resources from %s.\n"), quote (opt.input_metalink)); } metalink_delete (metalink); } inform_exit_status (retr_err); } #endif /* HAVE_METALINK */ /* Print broken links. */ if (opt.recursive && opt.spider) print_broken_links (); /* Print the downloaded sum. */ if ((opt.recursive || opt.page_requisites || nurls > 1 || (opt.input_filename && total_downloaded_bytes != 0)) && total_downloaded_bytes != 0) { double end_time = ptimer_measure (timer); char *wall_time = xstrdup (secs_to_human_time (end_time - start_time)); char *download_time = xstrdup (secs_to_human_time (total_download_time)); ptimer_destroy (timer); timer = NULL; logprintf (LOG_NOTQUIET, _("FINISHED --%s--\nTotal wall clock time: %s\n" "Downloaded: %d files, %s in %s (%s)\n"), datetime_str (time (NULL)), wall_time, numurls, human_readable (total_downloaded_bytes, 10, 1), download_time, retr_rate (total_downloaded_bytes, total_download_time)); xfree (wall_time); xfree (download_time); /* Print quota warning, if exceeded. */ if (opt.quota && total_downloaded_bytes > opt.quota) logprintf (LOG_NOTQUIET, _("Download quota of %s EXCEEDED!\n"), human_readable (opt.quota, 10, 1)); } if (opt.cookies_output) save_cookies (); #ifdef HAVE_HSTS if (opt.hsts && hsts_store) save_hsts (); #endif if ((opt.convert_links || opt.convert_file_only) && !opt.delete_after) convert_all_links (); cleanup (); exit (get_exit_status ()); } /* * vim: et ts=2 sw=2 */ wget-1.21.2/src/netrc.h0000644000000000000000000000271014115732710011506 00000000000000/* Declarations for netrc.c Copyright (C) 1996, 1996-1997, 2007-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #ifndef NETRC_H #define NETRC_H #include void search_netrc (const char *, const char **, const char **, int, FILE *); void netrc_cleanup(void); #endif /* NETRC_H */ wget-1.21.2/src/gnutls.c0000644000000000000000000007632414115732710011716 00000000000000/* SSL support via GnuTLS library. Copyright (C) 2005-2012, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #include "wget.h" #include #include #include #include #include #include #include #include #include #include #include #include "utils.h" #include "connect.h" #include "url.h" #include "ptimer.h" #include "hash.h" #include "ssl.h" #include #ifdef WIN32 # include "w32sock.h" #endif #include "host.h" struct st_read_timer { double timeout; double next_timeout; struct ptimer *timer; int timed_out; }; static int _do_handshake (gnutls_session_t session, int fd, struct st_read_timer *timeout); #if GNUTLS_VERSION_NUMBER >= 0x030604 static int _do_reauth (gnutls_session_t session, int fd, struct st_read_timer *timeout); #endif static int key_type_to_gnutls_type (enum keyfile_type type) { switch (type) { case keyfile_pem: return GNUTLS_X509_FMT_PEM; case keyfile_asn1: return GNUTLS_X509_FMT_DER; default: abort (); } } /* Note: some of the functions private to this file have names that begin with "wgnutls_" (e.g. wgnutls_read) so that they wouldn't be confused with actual gnutls functions -- such as the gnutls_read preprocessor macro. */ static bool ssl_initialized = false; static gnutls_certificate_credentials_t credentials; bool ssl_init (void) { fprintf(stderr,"SSL_INIT\n"); /* Becomes true if GnuTLS is initialized. */ const char *ca_directory; DIR *dir; int ncerts = -1; int rc; /* GnuTLS should be initialized only once. */ if (ssl_initialized) return true; gnutls_global_init (); gnutls_certificate_allocate_credentials (&credentials); gnutls_certificate_set_verify_flags (credentials, GNUTLS_VERIFY_ALLOW_X509_V1_CA_CRT); #if GNUTLS_VERSION_MAJOR >= 3 if (!opt.ca_directory) ncerts = gnutls_certificate_set_x509_system_trust (credentials); #endif /* If GnuTLS version is too old or CA loading failed, fallback to old behaviour. * Also use old behaviour if the CA directory is user-provided. */ if (ncerts <= 0) { ncerts = 0; ca_directory = opt.ca_directory ? opt.ca_directory : "/etc/ssl/certs"; if ((dir = opendir (ca_directory)) == NULL) { if (opt.ca_directory && *opt.ca_directory) logprintf (LOG_NOTQUIET, _("ERROR: Cannot open directory %s.\n"), opt.ca_directory); } else { struct hash_table *inode_map = hash_table_new (196, NULL, NULL); struct dirent *dent; while ((dent = readdir (dir)) != NULL) { struct stat st; char ca_file[1024]; if (((unsigned) snprintf (ca_file, sizeof (ca_file), "%s/%s", ca_directory, dent->d_name)) >= sizeof (ca_file)) continue; // overflow if (stat (ca_file, &st) != 0) continue; if (! S_ISREG (st.st_mode)) continue; /* avoid loading the same file twice by checking the inode. */ if (hash_table_contains (inode_map, (void *)(intptr_t) st.st_ino)) continue; hash_table_put (inode_map, (void *)(intptr_t) st.st_ino, NULL); if ((rc = gnutls_certificate_set_x509_trust_file (credentials, ca_file, GNUTLS_X509_FMT_PEM)) <= 0) DEBUGP (("WARNING: Failed to open cert %s: (%d).\n", ca_file, rc)); else ncerts += rc; } hash_table_destroy (inode_map); closedir (dir); } } if (opt.ca_cert) { if (ncerts < 0) ncerts = 0; if ((rc = gnutls_certificate_set_x509_trust_file (credentials, opt.ca_cert, GNUTLS_X509_FMT_PEM)) <= 0) logprintf (LOG_NOTQUIET, _("ERROR: Failed to open cert %s: (%d).\n"), opt.ca_cert, rc); else { ncerts += rc; logprintf (LOG_VERBOSE, _("Loaded CA certificate '%s'\n"), opt.ca_cert); } } if (opt.crl_file) { if ((rc = gnutls_certificate_set_x509_crl_file (credentials, opt.crl_file, GNUTLS_X509_FMT_PEM)) <= 0) { logprintf (LOG_NOTQUIET, _("ERROR: Failed to load CRL file '%s': (%d)\n"), opt.crl_file, rc); return false; } logprintf (LOG_VERBOSE, _("Loaded CRL file '%s'\n"), opt.crl_file); } DEBUGP (("Certificates loaded: %d\n", ncerts)); /* Use the private key from the cert file unless otherwise specified. */ if (opt.cert_file && !opt.private_key) { opt.private_key = xstrdup (opt.cert_file); opt.private_key_type = opt.cert_type; } /* Use the cert from the private key file unless otherwise specified. */ if (!opt.cert_file && opt.private_key) { opt.cert_file = xstrdup (opt.private_key); opt.cert_type = opt.private_key_type; } if (opt.cert_file && opt.private_key) { int type; if (opt.private_key_type != opt.cert_type) { /* GnuTLS can't handle this */ logprintf (LOG_NOTQUIET, _("ERROR: GnuTLS requires the key and the \ cert to be of the same type.\n")); } type = key_type_to_gnutls_type (opt.private_key_type); gnutls_certificate_set_x509_key_file (credentials, opt.cert_file, opt.private_key, type); } ssl_initialized = true; return true; } void ssl_cleanup (void) { fprintf(stderr,"SSL_CLEANUP\n"); if (!ssl_initialized) return; if (credentials) gnutls_certificate_free_credentials(credentials); gnutls_global_deinit(); ssl_initialized = false; } struct wgnutls_transport_context { gnutls_session_t session; /* GnuTLS session handle */ gnutls_datum_t *session_data; int last_error; /* last error returned by read/write/... */ /* Since GnuTLS doesn't support the equivalent to recv(..., MSG_PEEK) or SSL_peek(), we have to do it ourselves. Peeked data is stored to PEEKBUF, and wgnutls_read checks that buffer before actually reading. */ char peekbuf[512]; int peeklen; }; static int wgnutls_read_timeout (int fd, char *buf, int bufsize, void *arg, double timeout) { #ifdef F_GETFL int flags = 0; #endif struct wgnutls_transport_context *ctx = arg; int ret = gnutls_record_check_pending (ctx->session); struct st_read_timer read_timer = {(timeout == -1 ? opt.read_timeout : timeout), 0, NULL, 0}; if (ret) return gnutls_record_recv (ctx->session, buf, MIN (ret, bufsize)); if (read_timer.timeout) { #ifdef F_GETFL flags = fcntl (fd, F_GETFL, 0); if (flags < 0) return flags; if (fcntl (fd, F_SETFL, flags | O_NONBLOCK)) return -1; #else /* XXX: Assume it was blocking before. */ const int one = 1; if (ioctl (fd, FIONBIO, &one) < 0) return -1; #endif read_timer.timer = ptimer_new (); if (read_timer.timer == NULL) { ret = -1; goto timer_err; } read_timer.next_timeout = read_timer.timeout; } ret = ctx->last_error; do { if (ret == GNUTLS_E_REHANDSHAKE) { int err; DEBUGP (("GnuTLS: *** REHANDSHAKE while reading\n")); if ((err = _do_handshake (ctx->session, fd, &read_timer)) != 0) { ret = err; break; } } #if GNUTLS_VERSION_NUMBER >= 0x030604 else if (ret == GNUTLS_E_REAUTH_REQUEST) { int err; DEBUGP (("GnuTLS: *** re-authentication while reading\n")); if ((err = _do_reauth (ctx->session, fd, &read_timer)) != 0) { ret = err; break; } } #endif do { ret = gnutls_record_recv (ctx->session, buf, bufsize); if (ret == GNUTLS_E_AGAIN && read_timer.timer) { int err = select_fd_nb (fd, read_timer.next_timeout, WAIT_FOR_READ); if (err <= 0) { if (err == 0) read_timer.timed_out = 1; goto break_all; } if ( (read_timer.next_timeout = read_timer.timeout - ptimer_measure (read_timer.timer)) <= 0 ) { read_timer.timed_out = 1; goto break_all; } } } while (ret == GNUTLS_E_AGAIN || ret == GNUTLS_E_INTERRUPTED); } while (ret == GNUTLS_E_REHANDSHAKE #if GNUTLS_VERSION_NUMBER >= 0x030604 || ret == GNUTLS_E_REAUTH_REQUEST #endif ); break_all: if (read_timer.timer) { ptimer_destroy (read_timer.timer); timer_err: ; #ifdef F_GETFL if (fcntl (fd, F_SETFL, flags) < 0) return -1; #else { const int zero = 0; if (ioctl (fd, FIONBIO, &zero) < 0) return -1; } #endif if (read_timer.timed_out) errno = ETIMEDOUT; } return ret; } static int wgnutls_read (int fd, char *buf, int bufsize, void *arg, double timeout) { int ret; struct wgnutls_transport_context *ctx = arg; if (ctx->peeklen) { /* If we have any peek data, simply return that. */ int copysize = MIN (bufsize, ctx->peeklen); memcpy (buf, ctx->peekbuf, copysize); ctx->peeklen -= copysize; if (ctx->peeklen != 0) memmove (ctx->peekbuf, ctx->peekbuf + copysize, ctx->peeklen); return copysize; } ret = wgnutls_read_timeout (fd, buf, bufsize, arg, timeout); ctx->last_error = ret; return ret; } static int wgnutls_write (int fd _GL_UNUSED, char *buf, int bufsize, void *arg) { struct wgnutls_transport_context *ctx = arg; int ret = ctx->last_error; /* it should never happen, placed here only for debug msg. */ if (ret == GNUTLS_E_REHANDSHAKE) { DEBUGP (("GnuTLS: *** REHANDSHAKE while writing\n")); if ((ret = _do_handshake (ctx->session, fd, NULL)) != 0) goto ext; } #if GNUTLS_VERSION_NUMBER >= 0x030604 else if (ret == GNUTLS_E_REAUTH_REQUEST) { DEBUGP (("GnuTLS: *** re-authentication while writing\n")); if ((ret = _do_reauth (ctx->session, fd, NULL)) != 0) goto ext; } #endif do ret = gnutls_record_send (ctx->session, buf, bufsize); while (ret == GNUTLS_E_INTERRUPTED || ret == GNUTLS_E_AGAIN); ext: ctx->last_error = ret; return ret; } static int wgnutls_poll (int fd, double timeout, int wait_for, void *arg) { struct wgnutls_transport_context *ctx = arg; if ((wait_for & WAIT_FOR_READ) && (ctx->peeklen || gnutls_record_check_pending (ctx->session))) return 1; if (timeout == -1) timeout = opt.read_timeout; return select_fd (fd, timeout, wait_for); } static int wgnutls_peek (int fd, char *buf, int bufsize, void *arg, double timeout) { int read = 0; struct wgnutls_transport_context *ctx = arg; int offset = MIN (bufsize, ctx->peeklen); if (ctx->peeklen) { memcpy (buf, ctx->peekbuf, offset); return offset; } if (bufsize > (int) sizeof ctx->peekbuf) bufsize = sizeof ctx->peekbuf; if (bufsize > offset) { /* let wgnutls_read_timeout() take care about timeout */ /*if (timeout && gnutls_record_check_pending (ctx->session) == 0 && select_fd (fd, 0.0, WAIT_FOR_READ) <= 0) read = 0; else*/ read = wgnutls_read_timeout (fd, buf + offset, bufsize - offset, ctx, timeout); ctx->last_error = read; if (read < 0) { if (offset) read = 0; else return read; } if (read > 0) { memcpy (ctx->peekbuf + offset, buf + offset, read); ctx->peeklen += read; } } return offset + read; } static const char * wgnutls_errstr (int fd _GL_UNUSED, void *arg) { struct wgnutls_transport_context *ctx = arg; if (ctx->last_error > 0 || ((ctx->last_error == GNUTLS_E_AGAIN || ctx->last_error == GNUTLS_E_REHANDSHAKE #if GNUTLS_VERSION_NUMBER >= 0x030604 || ctx->last_error == GNUTLS_E_REAUTH_REQUEST #endif ) && errno == ETIMEDOUT)) return NULL; return gnutls_strerror (ctx->last_error); } static void wgnutls_close (int fd, void *arg) { struct wgnutls_transport_context *ctx = arg; /*gnutls_bye (ctx->session, GNUTLS_SHUT_RDWR);*/ if (ctx->session_data) { gnutls_free (ctx->session_data->data); gnutls_free (ctx->session_data); } gnutls_deinit (ctx->session); xfree (ctx); close (fd); } /* gnutls_transport is the singleton that describes the SSL transport methods provided by this file. */ static struct transport_implementation wgnutls_transport = { wgnutls_read, wgnutls_write, wgnutls_poll, wgnutls_peek, wgnutls_errstr, wgnutls_close }; static int _do_handshake (gnutls_session_t session, int fd, struct st_read_timer *read_timer) { #ifdef F_GETFL int flags = 0; #endif int err; double next_timeout = (read_timer ? read_timer->next_timeout : opt.read_timeout); /* if (read_timer != NULL) - fd is already non blocking */ if (!read_timer && next_timeout) { #ifdef F_GETFL flags = fcntl (fd, F_GETFL, 0); if (flags < 0) return flags; if (fcntl (fd, F_SETFL, flags | O_NONBLOCK)) return -1; #else /* XXX: Assume it was blocking before. */ const int one = 1; if (ioctl (fd, FIONBIO, &one) < 0) return -1; #endif } /* We don't stop the handshake process for non-fatal errors */ do { err = gnutls_handshake (session); if (err == GNUTLS_E_AGAIN && next_timeout) { int sel; if (gnutls_record_get_direction (session)) { /* wait for writeability */ sel = WAIT_FOR_WRITE; } else { /* wait for readability */ sel = WAIT_FOR_READ; } sel = select_fd_nb (fd, next_timeout, sel); if (sel <= 0) { if (sel == 0) { if (read_timer) goto read_timedout; else { errno = ETIMEDOUT; err = -1; } } break; } if (read_timer) { if ( (read_timer->next_timeout = read_timer->timeout - ptimer_measure (read_timer->timer)) <= 0 ) { read_timedout: /* return GNUTLS_E_REHANDSHAKE for gnutls_read */ err = GNUTLS_E_REHANDSHAKE; read_timer->timed_out = 1; break; } next_timeout = read_timer->next_timeout; } } else if (err < 0) { logprintf (LOG_NOTQUIET, "GnuTLS: %s\n", gnutls_strerror (err)); if (err == GNUTLS_E_WARNING_ALERT_RECEIVED || err == GNUTLS_E_FATAL_ALERT_RECEIVED) { gnutls_alert_description_t alert = gnutls_alert_get (session); const char *str = gnutls_alert_get_name (alert); logprintf (LOG_NOTQUIET, "GnuTLS: received alert [%u]: %s\n", alert, str ? str : "(unknown)"); } } } while (err && gnutls_error_is_fatal (err) == 0); if (!read_timer && next_timeout) { #ifdef F_GETFL if (fcntl (fd, F_SETFL, flags) < 0) return -1; #else const int zero = 0; if (ioctl (fd, FIONBIO, &zero) < 0) return -1; #endif } return err; } #if GNUTLS_VERSION_NUMBER >= 0x030604 static int _do_reauth (gnutls_session_t session, int fd, struct st_read_timer *read_timer) { #ifdef F_GETFL int flags = 0; #endif int err; double next_timeout = (read_timer ? read_timer->next_timeout : opt.read_timeout); /* if (read_timer != NULL) - fd is already non blocking */ if (!read_timer && next_timeout) { #ifdef F_GETFL flags = fcntl (fd, F_GETFL, 0); if (flags < 0) return flags; if (fcntl (fd, F_SETFL, flags | O_NONBLOCK)) return -1; #else /* XXX: Assume it was blocking before. */ const int one = 1; if (ioctl (fd, FIONBIO, &one) < 0) return -1; #endif } /* We don't stop the handshake process for non-fatal errors */ do { err = gnutls_reauth (session, 0); if (err == GNUTLS_E_AGAIN && next_timeout) { int sel; if (gnutls_record_get_direction (session)) { /* wait for writeability */ sel = WAIT_FOR_WRITE; } else { /* wait for readability */ sel = WAIT_FOR_READ; } sel = select_fd_nb (fd, next_timeout, sel); if (sel <= 0) { if (sel == 0) { if (read_timer) goto read_timedout; else { errno = ETIMEDOUT; err = -1; } } break; } if (read_timer) { if ( (read_timer->next_timeout = read_timer->timeout - ptimer_measure (read_timer->timer)) <= 0 ) { read_timedout: /* return GNUTLS_E_REAUTH_REQUEST for gnutls_read */ err = GNUTLS_E_REAUTH_REQUEST; read_timer->timed_out = 1; break; } next_timeout = read_timer->next_timeout; } } else if (err < 0) { logprintf (LOG_NOTQUIET, "GnuTLS: %s\n", gnutls_strerror (err)); } } while (err && gnutls_error_is_fatal (err) == 0); if (!read_timer && next_timeout) { #ifdef F_GETFL if (fcntl (fd, F_SETFL, flags) < 0) return -1; #else const int zero = 0; if (ioctl (fd, FIONBIO, &zero) < 0) return -1; #endif } return err; } #endif static const char * _sni_hostname(const char *hostname) { size_t len = strlen(hostname); char *sni_hostname = xmemdup(hostname, len + 1); /* Remove trailing dot(s) to fix #47408. * Regarding RFC 6066 (SNI): The hostname is represented as a byte * string using ASCII encoding without a trailing dot. */ while (len && sni_hostname[--len] == '.') sni_hostname[len] = 0; return sni_hostname; } static int set_prio_default (gnutls_session_t session) { int err = -1; #if HAVE_GNUTLS_PRIORITY_SET_DIRECT switch (opt.secure_protocol) { case secure_protocol_auto: err = gnutls_set_default_priority (session); gnutls_session_enable_compatibility_mode(session); break; case secure_protocol_sslv2: case secure_protocol_sslv3: err = gnutls_priority_set_direct (session, "NORMAL:-VERS-TLS-ALL:+VERS-SSL3.0", NULL); break; case secure_protocol_tlsv1: err = gnutls_priority_set_direct (session, "NORMAL:-VERS-SSL3.0", NULL); break; case secure_protocol_tlsv1_1: err = gnutls_priority_set_direct (session, "NORMAL:-VERS-SSL3.0:-VERS-TLS1.0", NULL); break; case secure_protocol_tlsv1_2: err = gnutls_priority_set_direct (session, "NORMAL:-VERS-SSL3.0:-VERS-TLS1.0:-VERS-TLS1.1", NULL); break; case secure_protocol_tlsv1_3: #if GNUTLS_VERSION_NUMBER >= 0x030603 err = gnutls_priority_set_direct (session, "NORMAL:-VERS-SSL3.0:+VERS-TLS1.3:-VERS-TLS1.0:-VERS-TLS1.1:-VERS-TLS1.2", NULL); break; #else logprintf (LOG_NOTQUIET, _("Your GnuTLS version is too old to support TLS 1.3\n")); return -1; #endif case secure_protocol_pfs: err = gnutls_priority_set_direct (session, "PFS:-VERS-SSL3.0", NULL); if (err != GNUTLS_E_SUCCESS) /* fallback if PFS is not available */ err = gnutls_priority_set_direct (session, "NORMAL:-RSA:-VERS-SSL3.0", NULL); break; default: logprintf (LOG_NOTQUIET, _("GnuTLS: unimplemented 'secure-protocol' option value %u\n"), (unsigned) opt.secure_protocol); logprintf (LOG_NOTQUIET, _("Please report this issue to bug-wget@gnu.org\n")); abort (); } #else int allowed_protocols[4] = {0, 0, 0, 0}; switch (opt.secure_protocol) { case secure_protocol_auto: err = gnutls_set_default_priority (session); break; case secure_protocol_sslv2: case secure_protocol_sslv3: allowed_protocols[0] = GNUTLS_SSL3; err = gnutls_protocol_set_priority (session, allowed_protocols); break; case secure_protocol_tlsv1: allowed_protocols[0] = GNUTLS_TLS1_0; allowed_protocols[1] = GNUTLS_TLS1_1; allowed_protocols[2] = GNUTLS_TLS1_2; #if GNUTLS_VERSION_NUMBER >= 0x030603 allowed_protocols[3] = GNUTLS_TLS1_3; #endif err = gnutls_protocol_set_priority (session, allowed_protocols); break; case secure_protocol_tlsv1_1: allowed_protocols[0] = GNUTLS_TLS1_1; allowed_protocols[1] = GNUTLS_TLS1_2; #if GNUTLS_VERSION_NUMBER >= 0x030603 allowed_protocols[2] = GNUTLS_TLS1_3; #endif err = gnutls_protocol_set_priority (session, allowed_protocols); break; case secure_protocol_tlsv1_2: allowed_protocols[0] = GNUTLS_TLS1_2; #if GNUTLS_VERSION_NUMBER >= 0x030603 allowed_protocols[1] = GNUTLS_TLS1_3; #endif err = gnutls_protocol_set_priority (session, allowed_protocols); break; case secure_protocol_tlsv1_3: #if GNUTLS_VERSION_NUMBER >= 0x030603 allowed_protocols[0] = GNUTLS_TLS1_3; err = gnutls_protocol_set_priority (session, allowed_protocols); break; #else logprintf (LOG_NOTQUIET, _("Your GnuTLS version is too old to support TLS 1.3\n")); return -1; #endif default: logprintf (LOG_NOTQUIET, _("GnuTLS: unimplemented 'secure-protocol' option value %d\n"), opt.secure_protocol); logprintf (LOG_NOTQUIET, _("Please report this issue to bug-wget@gnu.org\n")); abort (); } #endif return err; } bool ssl_connect_wget (int fd, const char *hostname, int *continue_session) { struct wgnutls_transport_context *ctx; gnutls_session_t session; int err; #if GNUTLS_VERSION_NUMBER >= 0x030604 // enable support of TLS1.3 post-handshake authentication gnutls_init (&session, GNUTLS_CLIENT | GNUTLS_POST_HANDSHAKE_AUTH); #else gnutls_init (&session, GNUTLS_CLIENT); #endif /* We set the server name but only if it's not an IP address. */ if (! is_valid_ip_address (hostname)) { /* GnuTLS 3.4.x (x<=10) disrespects the length parameter, we have to construct a new string */ /* see https://gitlab.com/gnutls/gnutls/issues/78 */ const char *sni_hostname = _sni_hostname(hostname); gnutls_server_name_set (session, GNUTLS_NAME_DNS, sni_hostname, strlen(sni_hostname)); xfree(sni_hostname); } gnutls_credentials_set (session, GNUTLS_CRD_CERTIFICATE, credentials); #ifndef FD_TO_SOCKET # define FD_TO_SOCKET(X) (X) #endif #ifdef HAVE_INTPTR_T gnutls_transport_set_ptr (session, (gnutls_transport_ptr_t) (intptr_t) FD_TO_SOCKET (fd)); #else gnutls_transport_set_ptr (session, (gnutls_transport_ptr_t) FD_TO_SOCKET (fd)); #endif if (!opt.tls_ciphers_string) { err = set_prio_default (session); } else { #if HAVE_GNUTLS_PRIORITY_SET_DIRECT err = gnutls_priority_set_direct (session, opt.tls_ciphers_string, NULL); #else logprintf (LOG_NOTQUIET, _("GnuTLS: Cannot set prio string directly. Falling back to default priority.\n")); err = gnutls_set_default_priority (session); #endif } if (err < 0) { logprintf (LOG_NOTQUIET, "GnuTLS: %s\n", gnutls_strerror (err)); gnutls_deinit (session); return false; } if (continue_session) { ctx = (struct wgnutls_transport_context *) fd_transport_context (*continue_session); if (!gnutls_session_is_resumed (session)) { if (!ctx || !ctx->session_data || gnutls_session_set_data (session, ctx->session_data->data, ctx->session_data->size)) { if (ctx && ctx->session_data) { /* server does not want to continue the session */ if (ctx->session_data->data) gnutls_free (ctx->session_data->data); gnutls_free (ctx->session_data); } gnutls_deinit (session); return false; } } else { logputs (LOG_ALWAYS, "SSL session has already been resumed. Continuing.\n"); continue_session = NULL; } } err = _do_handshake (session, fd, NULL); if (err < 0) { gnutls_deinit (session); return false; } ctx = xnew0 (struct wgnutls_transport_context); ctx->session_data = xnew0 (gnutls_datum_t); ctx->session = session; if (gnutls_session_get_data2 (session, ctx->session_data)) { xfree (ctx->session_data); logprintf (LOG_NOTQUIET, "WARNING: Could not save SSL session data for socket %d\n", fd); } fd_register_transport (fd, &wgnutls_transport, ctx); return true; } static bool pkp_pin_peer_pubkey (gnutls_x509_crt_t cert, const char *pinnedpubkey) { /* Scratch */ size_t len1 = 0, len2 = 0; char *buff1 = NULL; gnutls_pubkey_t key = NULL; /* Result is returned to caller */ int ret = 0; bool result = false; /* if a path wasn't specified, don't pin */ if (NULL == pinnedpubkey) return true; if (NULL == cert) return result; /* Begin Gyrations to get the public key */ gnutls_pubkey_init (&key); ret = gnutls_pubkey_import_x509 (key, cert, 0); if (ret < 0) goto cleanup; /* failed */ ret = gnutls_pubkey_export (key, GNUTLS_X509_FMT_DER, NULL, &len1); if (ret != GNUTLS_E_SHORT_MEMORY_BUFFER || len1 == 0) goto cleanup; /* failed */ buff1 = xmalloc (len1); len2 = len1; ret = gnutls_pubkey_export (key, GNUTLS_X509_FMT_DER, buff1, &len2); if (ret < 0 || len1 != len2) goto cleanup; /* failed */ /* End Gyrations */ /* The one good exit point */ result = wg_pin_peer_pubkey (pinnedpubkey, buff1, len1); cleanup: if (NULL != key) gnutls_pubkey_deinit (key); xfree (buff1); return result; } #define _CHECK_CERT(flag,msg) \ if (status & (flag))\ {\ logprintf (LOG_NOTQUIET, (msg),\ severity, quote (host));\ success = false;\ } bool ssl_check_certificate (int fd, const char *host) { struct wgnutls_transport_context *ctx = fd_transport_context (fd); unsigned int status; int err; /* If the user has specified --no-check-cert, we still want to warn him about problems with the server's certificate. */ const char *severity = opt.check_cert ? _("ERROR") : _("WARNING"); bool success = true; bool pinsuccess = opt.pinnedpubkey == NULL; /* The user explicitly said to not check for the certificate. */ if (opt.check_cert == CHECK_CERT_QUIET && pinsuccess) return success; err = gnutls_certificate_verify_peers2 (ctx->session, &status); if (err < 0) { logprintf (LOG_NOTQUIET, _("%s: No certificate presented by %s.\n"), severity, quotearg_style (escape_quoting_style, host)); success = false; goto out; } _CHECK_CERT (GNUTLS_CERT_INVALID, _("%s: The certificate of %s is not trusted.\n")); _CHECK_CERT (GNUTLS_CERT_SIGNER_NOT_FOUND, _("%s: The certificate of %s doesn't have a known issuer.\n")); _CHECK_CERT (GNUTLS_CERT_REVOKED, _("%s: The certificate of %s has been revoked.\n")); _CHECK_CERT (GNUTLS_CERT_SIGNER_NOT_CA, _("%s: The certificate signer of %s was not a CA.\n")); _CHECK_CERT (GNUTLS_CERT_INSECURE_ALGORITHM, _("%s: The certificate of %s was signed using an insecure algorithm.\n")); _CHECK_CERT (GNUTLS_CERT_NOT_ACTIVATED, _("%s: The certificate of %s is not yet activated.\n")); _CHECK_CERT (GNUTLS_CERT_EXPIRED, _("%s: The certificate of %s has expired.\n")); if (gnutls_certificate_type_get (ctx->session) == GNUTLS_CRT_X509) { time_t now = time (NULL); gnutls_x509_crt_t cert; const gnutls_datum_t *cert_list; unsigned int cert_list_size; const char *sni_hostname; if ((err = gnutls_x509_crt_init (&cert)) < 0) { logprintf (LOG_NOTQUIET, _("Error initializing X509 certificate: %s\n"), gnutls_strerror (err)); success = false; goto out; } cert_list = gnutls_certificate_get_peers (ctx->session, &cert_list_size); if (!cert_list) { logprintf (LOG_NOTQUIET, _("No certificate found\n")); success = false; goto crt_deinit; } err = gnutls_x509_crt_import (cert, cert_list, GNUTLS_X509_FMT_DER); if (err < 0) { logprintf (LOG_NOTQUIET, _("Error parsing certificate: %s\n"), gnutls_strerror (err)); success = false; goto crt_deinit; } if (now < gnutls_x509_crt_get_activation_time (cert)) { logprintf (LOG_NOTQUIET, _("The certificate has not yet been activated\n")); success = false; } if (now >= gnutls_x509_crt_get_expiration_time (cert)) { logprintf (LOG_NOTQUIET, _("The certificate has expired\n")); success = false; } sni_hostname = _sni_hostname(host); if (!gnutls_x509_crt_check_hostname (cert, sni_hostname)) { logprintf (LOG_NOTQUIET, _("The certificate's owner does not match hostname %s\n"), quote (sni_hostname)); success = false; } xfree(sni_hostname); pinsuccess = pkp_pin_peer_pubkey (cert, opt.pinnedpubkey); if (!pinsuccess) { logprintf (LOG_ALWAYS, _("The public key does not match pinned public key!\n")); success = false; } crt_deinit: gnutls_x509_crt_deinit (cert); } else { logprintf (LOG_NOTQUIET, _("Certificate must be X.509\n")); success = false; } out: /* never return true if pinsuccess fails */ return !pinsuccess ? false : (opt.check_cert == CHECK_CERT_ON ? success : true); } wget-1.21.2/src/css-url.h0000644000000000000000000000267614115732710011776 00000000000000/* Declarations for css-url.c. Copyright (C) 2006, 2009-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #ifndef CSS_URL_H #define CSS_URL_H void get_urls_css (struct map_context *, int, int); struct urlpos *get_urls_css_file (const char *, const char *); #endif /* CSS_URL_H */ wget-1.21.2/src/utils.h0000644000000000000000000001330614115732710011536 00000000000000/* Declarations for utils.c. Copyright (C) 1996-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #ifndef UTILS_H #define UTILS_H #include #include /* Constant is using when we don`t know attempted size exactly */ #define UNKNOWN_ATTEMPTED_SIZE -3 #ifndef MAX_PINNED_PUBKEY_SIZE #define MAX_PINNED_PUBKEY_SIZE 1048576 /* 1MB */ #endif /* Macros that interface to malloc, but know about type sizes, and cast the result to the appropriate type. The casts are not necessary in standard C, but Wget performs them anyway for the sake of pre-standard environments and possibly C++. */ #define xnew(type) (xmalloc (sizeof (type))) #define xnew0(type) (xcalloc (1, sizeof (type))) #define xnew_array(type, len) (xmalloc ((len) * sizeof (type))) #define xnew0_array(type, len) (xcalloc ((len), sizeof (type))) #define xfree(p) do { free ((void *) (p)); p = NULL; } while (0) struct hash_table; struct file_memory { char *content; long length; int mmap_p; }; #define HYPHENP(x) (*(x) == '-' && !*((x) + 1)) char *time_str (time_t); char *datetime_str (time_t); char *xstrdup_lower (const char *); char *strdupdelim (const char *, const char *); char **sepstring (const char *); bool subdir_p (const char *, const char *); bool fork_to_background (void); char *aprintf (const char *, ...) GCC_FORMAT_ATTR (1, 2); char *concat_strings (const char *, ...); typedef struct file_stat_s { int access_err; /* Error in accecssing file : Not present vs permission */ ino_t st_ino; /* st_ino from stats() on the file before open() */ dev_t st_dev; /* st_dev from stats() on the file before open() */ } file_stats_t; void touch (const char *, time_t); int remove_link (const char *); bool file_exists_p (const char *, file_stats_t *); bool file_non_directory_p (const char *); wgint file_size (const char *); int make_directory (const char *); char *unique_name_passthrough (const char *); char *unique_name (const char *); FILE *unique_create (const char *, bool, char **); FILE *fopen_excl (const char *, int); FILE *fopen_stat (const char *, const char *, file_stats_t *); int open_stat (const char *, int, mode_t, file_stats_t *); char *file_merge (const char *, const char *); int fnmatch_nocase (const char *, const char *, int); bool acceptable (const char *); bool accept_url (const char *); bool accdir (const char *s); char *suffix (const char *s); bool match_tail (const char *, const char *, bool); bool has_wildcards_p (const char *); bool has_html_suffix_p (const char *); struct file_memory *wget_read_file (const char *); void wget_read_file_free (struct file_memory *); void free_vec (char **); char **merge_vecs (char **, char **); char **vec_append (char **, const char *); void string_set_add (struct hash_table *, const char *); int string_set_contains (struct hash_table *, const char *); void string_set_to_array (struct hash_table *, char **); void string_set_free (struct hash_table *); void free_keys_and_values (struct hash_table *); const char *with_thousand_seps (wgint); /* human_readable must be able to accept wgint arguments. */ char *human_readable (wgint, const int, const int); int numdigit (wgint); char *number_to_string (char *, wgint); char *number_to_static_string (wgint); wgint convert_to_bits (wgint); int determine_screen_width (void); int random_number (int); double random_float (void); bool run_with_timeout (double, void (*) (void *), void *); void xsleep (double); /* How many bytes it will take to store LEN bytes in base64. */ #define BASE64_LENGTH(len) (4 * (((len) + 2) / 3)) size_t wget_base64_encode (const void *, size_t, char *); ssize_t wget_base64_decode (const char *, void *, size_t); #ifdef HAVE_LIBPCRE2 void *compile_pcre2_regex (const char *); bool match_pcre2_regex (const void *, const char *); #endif #ifdef HAVE_LIBPCRE void *compile_pcre_regex (const char *); bool match_pcre_regex (const void *, const char *); #endif void *compile_posix_regex (const char *); bool match_posix_regex (const void *, const char *); void stable_sort (void *, size_t, size_t, int (*) (const void *, const void *)); const char *print_decimal (double); long get_max_length (const char *path, int length, int name); #ifndef HAVE_STRLCPY size_t strlcpy (char *dst, const char *src, size_t size); #endif void wg_hex_to_string (char *str_buffer, const char *hex_buffer, size_t hex_len); extern unsigned char char_prop[]; #ifdef HAVE_SSL /* Check pinned public key. */ bool wg_pin_peer_pubkey (const char *pinnedpubkey, const char *pubkey, size_t pubkeylen); #endif #endif /* UTILS_H */ wget-1.21.2/src/ftp-opie.c0000644000000000000000000016440014115732710012116 00000000000000/* Opie (s/key) support for FTP. Copyright (C) 1998-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #include "wget.h" #include #include #include #include "md5.h" #include "ftp.h" /* Dictionary for integer-word translations. Available in appendix D of rfc2289. */ static char Wp[2048][4] = { { 'A', '\0', '\0', '\0' }, { 'A', 'B', 'E', '\0' }, { 'A', 'C', 'E', '\0' }, { 'A', 'C', 'T', '\0' }, { 'A', 'D', '\0', '\0' }, { 'A', 'D', 'A', '\0' }, { 'A', 'D', 'D', '\0' }, { 'A', 'G', 'O', '\0' }, { 'A', 'I', 'D', '\0' }, { 'A', 'I', 'M', '\0' }, { 'A', 'I', 'R', '\0' }, { 'A', 'L', 'L', '\0' }, { 'A', 'L', 'P', '\0' }, { 'A', 'M', '\0', '\0' }, { 'A', 'M', 'Y', '\0' }, { 'A', 'N', '\0', '\0' }, { 'A', 'N', 'A', '\0' }, { 'A', 'N', 'D', '\0' }, { 'A', 'N', 'N', '\0' }, { 'A', 'N', 'T', '\0' }, { 'A', 'N', 'Y', '\0' }, { 'A', 'P', 'E', '\0' }, { 'A', 'P', 'S', '\0' }, { 'A', 'P', 'T', '\0' }, { 'A', 'R', 'C', '\0' }, { 'A', 'R', 'E', '\0' }, { 'A', 'R', 'K', '\0' }, { 'A', 'R', 'M', '\0' }, { 'A', 'R', 'T', '\0' }, { 'A', 'S', '\0', '\0' }, { 'A', 'S', 'H', '\0' }, { 'A', 'S', 'K', '\0' }, { 'A', 'T', '\0', '\0' }, { 'A', 'T', 'E', '\0' }, { 'A', 'U', 'G', '\0' }, { 'A', 'U', 'K', '\0' }, { 'A', 'V', 'E', '\0' }, { 'A', 'W', 'E', '\0' }, { 'A', 'W', 'K', '\0' }, { 'A', 'W', 'L', '\0' }, { 'A', 'W', 'N', '\0' }, { 'A', 'X', '\0', '\0' }, { 'A', 'Y', 'E', '\0' }, { 'B', 'A', 'D', '\0' }, { 'B', 'A', 'G', '\0' }, { 'B', 'A', 'H', '\0' }, { 'B', 'A', 'M', '\0' }, { 'B', 'A', 'N', '\0' }, { 'B', 'A', 'R', '\0' }, { 'B', 'A', 'T', '\0' }, { 'B', 'A', 'Y', '\0' }, { 'B', 'E', '\0', '\0' }, { 'B', 'E', 'D', '\0' }, { 'B', 'E', 'E', '\0' }, { 'B', 'E', 'G', '\0' }, { 'B', 'E', 'N', '\0' }, { 'B', 'E', 'T', '\0' }, { 'B', 'E', 'Y', '\0' }, { 'B', 'I', 'B', '\0' }, { 'B', 'I', 'D', '\0' }, { 'B', 'I', 'G', '\0' }, { 'B', 'I', 'N', '\0' }, { 'B', 'I', 'T', '\0' }, { 'B', 'O', 'B', '\0' }, { 'B', 'O', 'G', '\0' }, { 'B', 'O', 'N', '\0' }, { 'B', 'O', 'O', '\0' }, { 'B', 'O', 'P', '\0' }, { 'B', 'O', 'W', '\0' }, { 'B', 'O', 'Y', '\0' }, { 'B', 'U', 'B', '\0' }, { 'B', 'U', 'D', '\0' }, { 'B', 'U', 'G', '\0' }, { 'B', 'U', 'M', '\0' }, { 'B', 'U', 'N', '\0' }, { 'B', 'U', 'S', '\0' }, { 'B', 'U', 'T', '\0' }, { 'B', 'U', 'Y', '\0' }, { 'B', 'Y', '\0', '\0' }, { 'B', 'Y', 'E', '\0' }, { 'C', 'A', 'B', '\0' }, { 'C', 'A', 'L', '\0' }, { 'C', 'A', 'M', '\0' }, { 'C', 'A', 'N', '\0' }, { 'C', 'A', 'P', '\0' }, { 'C', 'A', 'R', '\0' }, { 'C', 'A', 'T', '\0' }, { 'C', 'A', 'W', '\0' }, { 'C', 'O', 'D', '\0' }, { 'C', 'O', 'G', '\0' }, { 'C', 'O', 'L', '\0' }, { 'C', 'O', 'N', '\0' }, { 'C', 'O', 'O', '\0' }, { 'C', 'O', 'P', '\0' }, { 'C', 'O', 'T', '\0' }, { 'C', 'O', 'W', '\0' }, { 'C', 'O', 'Y', '\0' }, { 'C', 'R', 'Y', '\0' }, { 'C', 'U', 'B', '\0' }, { 'C', 'U', 'E', '\0' }, { 'C', 'U', 'P', '\0' }, { 'C', 'U', 'R', '\0' }, { 'C', 'U', 'T', '\0' }, { 'D', 'A', 'B', '\0' }, { 'D', 'A', 'D', '\0' }, { 'D', 'A', 'M', '\0' }, { 'D', 'A', 'N', '\0' }, { 'D', 'A', 'R', '\0' }, { 'D', 'A', 'Y', '\0' }, { 'D', 'E', 'E', '\0' }, { 'D', 'E', 'L', '\0' }, { 'D', 'E', 'N', '\0' }, { 'D', 'E', 'S', '\0' }, { 'D', 'E', 'W', '\0' }, { 'D', 'I', 'D', '\0' }, { 'D', 'I', 'E', '\0' }, { 'D', 'I', 'G', '\0' }, { 'D', 'I', 'N', '\0' }, { 'D', 'I', 'P', '\0' }, { 'D', 'O', '\0', '\0' }, { 'D', 'O', 'E', '\0' }, { 'D', 'O', 'G', '\0' }, { 'D', 'O', 'N', '\0' }, { 'D', 'O', 'T', '\0' }, { 'D', 'O', 'W', '\0' }, { 'D', 'R', 'Y', '\0' }, { 'D', 'U', 'B', '\0' }, { 'D', 'U', 'D', '\0' }, { 'D', 'U', 'E', '\0' }, { 'D', 'U', 'G', '\0' }, { 'D', 'U', 'N', '\0' }, { 'E', 'A', 'R', '\0' }, { 'E', 'A', 'T', '\0' }, { 'E', 'D', '\0', '\0' }, { 'E', 'E', 'L', '\0' }, { 'E', 'G', 'G', '\0' }, { 'E', 'G', 'O', '\0' }, { 'E', 'L', 'I', '\0' }, { 'E', 'L', 'K', '\0' }, { 'E', 'L', 'M', '\0' }, { 'E', 'L', 'Y', '\0' }, { 'E', 'M', '\0', '\0' }, { 'E', 'N', 'D', '\0' }, { 'E', 'S', 'T', '\0' }, { 'E', 'T', 'C', '\0' }, { 'E', 'V', 'A', '\0' }, { 'E', 'V', 'E', '\0' }, { 'E', 'W', 'E', '\0' }, { 'E', 'Y', 'E', '\0' }, { 'F', 'A', 'D', '\0' }, { 'F', 'A', 'N', '\0' }, { 'F', 'A', 'R', '\0' }, { 'F', 'A', 'T', '\0' }, { 'F', 'A', 'Y', '\0' }, { 'F', 'E', 'D', '\0' }, { 'F', 'E', 'E', '\0' }, { 'F', 'E', 'W', '\0' }, { 'F', 'I', 'B', '\0' }, { 'F', 'I', 'G', '\0' }, { 'F', 'I', 'N', '\0' }, { 'F', 'I', 'R', '\0' }, { 'F', 'I', 'T', '\0' }, { 'F', 'L', 'O', '\0' }, { 'F', 'L', 'Y', '\0' }, { 'F', 'O', 'E', '\0' }, { 'F', 'O', 'G', '\0' }, { 'F', 'O', 'R', '\0' }, { 'F', 'R', 'Y', '\0' }, { 'F', 'U', 'M', '\0' }, { 'F', 'U', 'N', '\0' }, { 'F', 'U', 'R', '\0' }, { 'G', 'A', 'B', '\0' }, { 'G', 'A', 'D', '\0' }, { 'G', 'A', 'G', '\0' }, { 'G', 'A', 'L', '\0' }, { 'G', 'A', 'M', '\0' }, { 'G', 'A', 'P', '\0' }, { 'G', 'A', 'S', '\0' }, { 'G', 'A', 'Y', '\0' }, { 'G', 'E', 'E', '\0' }, { 'G', 'E', 'L', '\0' }, { 'G', 'E', 'M', '\0' }, { 'G', 'E', 'T', '\0' }, { 'G', 'I', 'G', '\0' }, { 'G', 'I', 'L', '\0' }, { 'G', 'I', 'N', '\0' }, { 'G', 'O', '\0', '\0' }, { 'G', 'O', 'T', '\0' }, { 'G', 'U', 'M', '\0' }, { 'G', 'U', 'N', '\0' }, { 'G', 'U', 'S', '\0' }, { 'G', 'U', 'T', '\0' }, { 'G', 'U', 'Y', '\0' }, { 'G', 'Y', 'M', '\0' }, { 'G', 'Y', 'P', '\0' }, { 'H', 'A', '\0', '\0' }, { 'H', 'A', 'D', '\0' }, { 'H', 'A', 'L', '\0' }, { 'H', 'A', 'M', '\0' }, { 'H', 'A', 'N', '\0' }, { 'H', 'A', 'P', '\0' }, { 'H', 'A', 'S', '\0' }, { 'H', 'A', 'T', '\0' }, { 'H', 'A', 'W', '\0' }, { 'H', 'A', 'Y', '\0' }, { 'H', 'E', '\0', '\0' }, { 'H', 'E', 'M', '\0' }, { 'H', 'E', 'N', '\0' }, { 'H', 'E', 'R', '\0' }, { 'H', 'E', 'W', '\0' }, { 'H', 'E', 'Y', '\0' }, { 'H', 'I', '\0', '\0' }, { 'H', 'I', 'D', '\0' }, { 'H', 'I', 'M', '\0' }, { 'H', 'I', 'P', '\0' }, { 'H', 'I', 'S', '\0' }, { 'H', 'I', 'T', '\0' }, { 'H', 'O', '\0', '\0' }, { 'H', 'O', 'B', '\0' }, { 'H', 'O', 'C', '\0' }, { 'H', 'O', 'E', '\0' }, { 'H', 'O', 'G', '\0' }, { 'H', 'O', 'P', '\0' }, { 'H', 'O', 'T', '\0' }, { 'H', 'O', 'W', '\0' }, { 'H', 'U', 'B', '\0' }, { 'H', 'U', 'E', '\0' }, { 'H', 'U', 'G', '\0' }, { 'H', 'U', 'H', '\0' }, { 'H', 'U', 'M', '\0' }, { 'H', 'U', 'T', '\0' }, { 'I', '\0', '\0', '\0' }, { 'I', 'C', 'Y', '\0' }, { 'I', 'D', 'A', '\0' }, { 'I', 'F', '\0', '\0' }, { 'I', 'K', 'E', '\0' }, { 'I', 'L', 'L', '\0' }, { 'I', 'N', 'K', '\0' }, { 'I', 'N', 'N', '\0' }, { 'I', 'O', '\0', '\0' }, { 'I', 'O', 'N', '\0' }, { 'I', 'Q', '\0', '\0' }, { 'I', 'R', 'A', '\0' }, { 'I', 'R', 'E', '\0' }, { 'I', 'R', 'K', '\0' }, { 'I', 'S', '\0', '\0' }, { 'I', 'T', '\0', '\0' }, { 'I', 'T', 'S', '\0' }, { 'I', 'V', 'Y', '\0' }, { 'J', 'A', 'B', '\0' }, { 'J', 'A', 'G', '\0' }, { 'J', 'A', 'M', '\0' }, { 'J', 'A', 'N', '\0' }, { 'J', 'A', 'R', '\0' }, { 'J', 'A', 'W', '\0' }, { 'J', 'A', 'Y', '\0' }, { 'J', 'E', 'T', '\0' }, { 'J', 'I', 'G', '\0' }, { 'J', 'I', 'M', '\0' }, { 'J', 'O', '\0', '\0' }, { 'J', 'O', 'B', '\0' }, { 'J', 'O', 'E', '\0' }, { 'J', 'O', 'G', '\0' }, { 'J', 'O', 'T', '\0' }, { 'J', 'O', 'Y', '\0' }, { 'J', 'U', 'G', '\0' }, { 'J', 'U', 'T', '\0' }, { 'K', 'A', 'Y', '\0' }, { 'K', 'E', 'G', '\0' }, { 'K', 'E', 'N', '\0' }, { 'K', 'E', 'Y', '\0' }, { 'K', 'I', 'D', '\0' }, { 'K', 'I', 'M', '\0' }, { 'K', 'I', 'N', '\0' }, { 'K', 'I', 'T', '\0' }, { 'L', 'A', '\0', '\0' }, { 'L', 'A', 'B', '\0' }, { 'L', 'A', 'C', '\0' }, { 'L', 'A', 'D', '\0' }, { 'L', 'A', 'G', '\0' }, { 'L', 'A', 'M', '\0' }, { 'L', 'A', 'P', '\0' }, { 'L', 'A', 'W', '\0' }, { 'L', 'A', 'Y', '\0' }, { 'L', 'E', 'A', '\0' }, { 'L', 'E', 'D', '\0' }, { 'L', 'E', 'E', '\0' }, { 'L', 'E', 'G', '\0' }, { 'L', 'E', 'N', '\0' }, { 'L', 'E', 'O', '\0' }, { 'L', 'E', 'T', '\0' }, { 'L', 'E', 'W', '\0' }, { 'L', 'I', 'D', '\0' }, { 'L', 'I', 'E', '\0' }, { 'L', 'I', 'N', '\0' }, { 'L', 'I', 'P', '\0' }, { 'L', 'I', 'T', '\0' }, { 'L', 'O', '\0', '\0' }, { 'L', 'O', 'B', '\0' }, { 'L', 'O', 'G', '\0' }, { 'L', 'O', 'P', '\0' }, { 'L', 'O', 'S', '\0' }, { 'L', 'O', 'T', '\0' }, { 'L', 'O', 'U', '\0' }, { 'L', 'O', 'W', '\0' }, { 'L', 'O', 'Y', '\0' }, { 'L', 'U', 'G', '\0' }, { 'L', 'Y', 'E', '\0' }, { 'M', 'A', '\0', '\0' }, { 'M', 'A', 'C', '\0' }, { 'M', 'A', 'D', '\0' }, { 'M', 'A', 'E', '\0' }, { 'M', 'A', 'N', '\0' }, { 'M', 'A', 'O', '\0' }, { 'M', 'A', 'P', '\0' }, { 'M', 'A', 'T', '\0' }, { 'M', 'A', 'W', '\0' }, { 'M', 'A', 'Y', '\0' }, { 'M', 'E', '\0', '\0' }, { 'M', 'E', 'G', '\0' }, { 'M', 'E', 'L', '\0' }, { 'M', 'E', 'N', '\0' }, { 'M', 'E', 'T', '\0' }, { 'M', 'E', 'W', '\0' }, { 'M', 'I', 'D', '\0' }, { 'M', 'I', 'N', '\0' }, { 'M', 'I', 'T', '\0' }, { 'M', 'O', 'B', '\0' }, { 'M', 'O', 'D', '\0' }, { 'M', 'O', 'E', '\0' }, { 'M', 'O', 'O', '\0' }, { 'M', 'O', 'P', '\0' }, { 'M', 'O', 'S', '\0' }, { 'M', 'O', 'T', '\0' }, { 'M', 'O', 'W', '\0' }, { 'M', 'U', 'D', '\0' }, { 'M', 'U', 'G', '\0' }, { 'M', 'U', 'M', '\0' }, { 'M', 'Y', '\0', '\0' }, { 'N', 'A', 'B', '\0' }, { 'N', 'A', 'G', '\0' }, { 'N', 'A', 'N', '\0' }, { 'N', 'A', 'P', '\0' }, { 'N', 'A', 'T', '\0' }, { 'N', 'A', 'Y', '\0' }, { 'N', 'E', '\0', '\0' }, { 'N', 'E', 'D', '\0' }, { 'N', 'E', 'E', '\0' }, { 'N', 'E', 'T', '\0' }, { 'N', 'E', 'W', '\0' }, { 'N', 'I', 'B', '\0' }, { 'N', 'I', 'L', '\0' }, { 'N', 'I', 'P', '\0' }, { 'N', 'I', 'T', '\0' }, { 'N', 'O', '\0', '\0' }, { 'N', 'O', 'B', '\0' }, { 'N', 'O', 'D', '\0' }, { 'N', 'O', 'N', '\0' }, { 'N', 'O', 'R', '\0' }, { 'N', 'O', 'T', '\0' }, { 'N', 'O', 'V', '\0' }, { 'N', 'O', 'W', '\0' }, { 'N', 'U', '\0', '\0' }, { 'N', 'U', 'N', '\0' }, { 'N', 'U', 'T', '\0' }, { 'O', '\0', '\0', '\0' }, { 'O', 'A', 'F', '\0' }, { 'O', 'A', 'K', '\0' }, { 'O', 'A', 'R', '\0' }, { 'O', 'A', 'T', '\0' }, { 'O', 'D', 'D', '\0' }, { 'O', 'D', 'E', '\0' }, { 'O', 'F', '\0', '\0' }, { 'O', 'F', 'F', '\0' }, { 'O', 'F', 'T', '\0' }, { 'O', 'H', '\0', '\0' }, { 'O', 'I', 'L', '\0' }, { 'O', 'K', '\0', '\0' }, { 'O', 'L', 'D', '\0' }, { 'O', 'N', '\0', '\0' }, { 'O', 'N', 'E', '\0' }, { 'O', 'R', '\0', '\0' }, { 'O', 'R', 'B', '\0' }, { 'O', 'R', 'E', '\0' }, { 'O', 'R', 'R', '\0' }, { 'O', 'S', '\0', '\0' }, { 'O', 'T', 'T', '\0' }, { 'O', 'U', 'R', '\0' }, { 'O', 'U', 'T', '\0' }, { 'O', 'V', 'A', '\0' }, { 'O', 'W', '\0', '\0' }, { 'O', 'W', 'E', '\0' }, { 'O', 'W', 'L', '\0' }, { 'O', 'W', 'N', '\0' }, { 'O', 'X', '\0', '\0' }, { 'P', 'A', '\0', '\0' }, { 'P', 'A', 'D', '\0' }, { 'P', 'A', 'L', '\0' }, { 'P', 'A', 'M', '\0' }, { 'P', 'A', 'N', '\0' }, { 'P', 'A', 'P', '\0' }, { 'P', 'A', 'R', '\0' }, { 'P', 'A', 'T', '\0' }, { 'P', 'A', 'W', '\0' }, { 'P', 'A', 'Y', '\0' }, { 'P', 'E', 'A', '\0' }, { 'P', 'E', 'G', '\0' }, { 'P', 'E', 'N', '\0' }, { 'P', 'E', 'P', '\0' }, { 'P', 'E', 'R', '\0' }, { 'P', 'E', 'T', '\0' }, { 'P', 'E', 'W', '\0' }, { 'P', 'H', 'I', '\0' }, { 'P', 'I', '\0', '\0' }, { 'P', 'I', 'E', '\0' }, { 'P', 'I', 'N', '\0' }, { 'P', 'I', 'T', '\0' }, { 'P', 'L', 'Y', '\0' }, { 'P', 'O', '\0', '\0' }, { 'P', 'O', 'D', '\0' }, { 'P', 'O', 'E', '\0' }, { 'P', 'O', 'P', '\0' }, { 'P', 'O', 'T', '\0' }, { 'P', 'O', 'W', '\0' }, { 'P', 'R', 'O', '\0' }, { 'P', 'R', 'Y', '\0' }, { 'P', 'U', 'B', '\0' }, { 'P', 'U', 'G', '\0' }, { 'P', 'U', 'N', '\0' }, { 'P', 'U', 'P', '\0' }, { 'P', 'U', 'T', '\0' }, { 'Q', 'U', 'O', '\0' }, { 'R', 'A', 'G', '\0' }, { 'R', 'A', 'M', '\0' }, { 'R', 'A', 'N', '\0' }, { 'R', 'A', 'P', '\0' }, { 'R', 'A', 'T', '\0' }, { 'R', 'A', 'W', '\0' }, { 'R', 'A', 'Y', '\0' }, { 'R', 'E', 'B', '\0' }, { 'R', 'E', 'D', '\0' }, { 'R', 'E', 'P', '\0' }, { 'R', 'E', 'T', '\0' }, { 'R', 'I', 'B', '\0' }, { 'R', 'I', 'D', '\0' }, { 'R', 'I', 'G', '\0' }, { 'R', 'I', 'M', '\0' }, { 'R', 'I', 'O', '\0' }, { 'R', 'I', 'P', '\0' }, { 'R', 'O', 'B', '\0' }, { 'R', 'O', 'D', '\0' }, { 'R', 'O', 'E', '\0' }, { 'R', 'O', 'N', '\0' }, { 'R', 'O', 'T', '\0' }, { 'R', 'O', 'W', '\0' }, { 'R', 'O', 'Y', '\0' }, { 'R', 'U', 'B', '\0' }, { 'R', 'U', 'E', '\0' }, { 'R', 'U', 'G', '\0' }, { 'R', 'U', 'M', '\0' }, { 'R', 'U', 'N', '\0' }, { 'R', 'Y', 'E', '\0' }, { 'S', 'A', 'C', '\0' }, { 'S', 'A', 'D', '\0' }, { 'S', 'A', 'G', '\0' }, { 'S', 'A', 'L', '\0' }, { 'S', 'A', 'M', '\0' }, { 'S', 'A', 'N', '\0' }, { 'S', 'A', 'P', '\0' }, { 'S', 'A', 'T', '\0' }, { 'S', 'A', 'W', '\0' }, { 'S', 'A', 'Y', '\0' }, { 'S', 'E', 'A', '\0' }, { 'S', 'E', 'C', '\0' }, { 'S', 'E', 'E', '\0' }, { 'S', 'E', 'N', '\0' }, { 'S', 'E', 'T', '\0' }, { 'S', 'E', 'W', '\0' }, { 'S', 'H', 'E', '\0' }, { 'S', 'H', 'Y', '\0' }, { 'S', 'I', 'N', '\0' }, { 'S', 'I', 'P', '\0' }, { 'S', 'I', 'R', '\0' }, { 'S', 'I', 'S', '\0' }, { 'S', 'I', 'T', '\0' }, { 'S', 'K', 'I', '\0' }, { 'S', 'K', 'Y', '\0' }, { 'S', 'L', 'Y', '\0' }, { 'S', 'O', '\0', '\0' }, { 'S', 'O', 'B', '\0' }, { 'S', 'O', 'D', '\0' }, { 'S', 'O', 'N', '\0' }, { 'S', 'O', 'P', '\0' }, { 'S', 'O', 'W', '\0' }, { 'S', 'O', 'Y', '\0' }, { 'S', 'P', 'A', '\0' }, { 'S', 'P', 'Y', '\0' }, { 'S', 'U', 'B', '\0' }, { 'S', 'U', 'D', '\0' }, { 'S', 'U', 'E', '\0' }, { 'S', 'U', 'M', '\0' }, { 'S', 'U', 'N', '\0' }, { 'S', 'U', 'P', '\0' }, { 'T', 'A', 'B', '\0' }, { 'T', 'A', 'D', '\0' }, { 'T', 'A', 'G', '\0' }, { 'T', 'A', 'N', '\0' }, { 'T', 'A', 'P', '\0' }, { 'T', 'A', 'R', '\0' }, { 'T', 'E', 'A', '\0' }, { 'T', 'E', 'D', '\0' }, { 'T', 'E', 'E', '\0' }, { 'T', 'E', 'N', '\0' }, { 'T', 'H', 'E', '\0' }, { 'T', 'H', 'Y', '\0' }, { 'T', 'I', 'C', '\0' }, { 'T', 'I', 'E', '\0' }, { 'T', 'I', 'M', '\0' }, { 'T', 'I', 'N', '\0' }, { 'T', 'I', 'P', '\0' }, { 'T', 'O', '\0', '\0' }, { 'T', 'O', 'E', '\0' }, { 'T', 'O', 'G', '\0' }, { 'T', 'O', 'M', '\0' }, { 'T', 'O', 'N', '\0' }, { 'T', 'O', 'O', '\0' }, { 'T', 'O', 'P', '\0' }, { 'T', 'O', 'W', '\0' }, { 'T', 'O', 'Y', '\0' }, { 'T', 'R', 'Y', '\0' }, { 'T', 'U', 'B', '\0' }, { 'T', 'U', 'G', '\0' }, { 'T', 'U', 'M', '\0' }, { 'T', 'U', 'N', '\0' }, { 'T', 'W', 'O', '\0' }, { 'U', 'N', '\0', '\0' }, { 'U', 'P', '\0', '\0' }, { 'U', 'S', '\0', '\0' }, { 'U', 'S', 'E', '\0' }, { 'V', 'A', 'N', '\0' }, { 'V', 'A', 'T', '\0' }, { 'V', 'E', 'T', '\0' }, { 'V', 'I', 'E', '\0' }, { 'W', 'A', 'D', '\0' }, { 'W', 'A', 'G', '\0' }, { 'W', 'A', 'R', '\0' }, { 'W', 'A', 'S', '\0' }, { 'W', 'A', 'Y', '\0' }, { 'W', 'E', '\0', '\0' }, { 'W', 'E', 'B', '\0' }, { 'W', 'E', 'D', '\0' }, { 'W', 'E', 'E', '\0' }, { 'W', 'E', 'T', '\0' }, { 'W', 'H', 'O', '\0' }, { 'W', 'H', 'Y', '\0' }, { 'W', 'I', 'N', '\0' }, { 'W', 'I', 'T', '\0' }, { 'W', 'O', 'K', '\0' }, { 'W', 'O', 'N', '\0' }, { 'W', 'O', 'O', '\0' }, { 'W', 'O', 'W', '\0' }, { 'W', 'R', 'Y', '\0' }, { 'W', 'U', '\0', '\0' }, { 'Y', 'A', 'M', '\0' }, { 'Y', 'A', 'P', '\0' }, { 'Y', 'A', 'W', '\0' }, { 'Y', 'E', '\0', '\0' }, { 'Y', 'E', 'A', '\0' }, { 'Y', 'E', 'S', '\0' }, { 'Y', 'E', 'T', '\0' }, { 'Y', 'O', 'U', '\0' }, { 'A', 'B', 'E', 'D' }, { 'A', 'B', 'E', 'L' }, { 'A', 'B', 'E', 'T' }, { 'A', 'B', 'L', 'E' }, { 'A', 'B', 'U', 'T' }, { 'A', 'C', 'H', 'E' }, { 'A', 'C', 'I', 'D' }, { 'A', 'C', 'M', 'E' }, { 'A', 'C', 'R', 'E' }, { 'A', 'C', 'T', 'A' }, { 'A', 'C', 'T', 'S' }, { 'A', 'D', 'A', 'M' }, { 'A', 'D', 'D', 'S' }, { 'A', 'D', 'E', 'N' }, { 'A', 'F', 'A', 'R' }, { 'A', 'F', 'R', 'O' }, { 'A', 'G', 'E', 'E' }, { 'A', 'H', 'E', 'M' }, { 'A', 'H', 'O', 'Y' }, { 'A', 'I', 'D', 'A' }, { 'A', 'I', 'D', 'E' }, { 'A', 'I', 'D', 'S' }, { 'A', 'I', 'R', 'Y' }, { 'A', 'J', 'A', 'R' }, { 'A', 'K', 'I', 'N' }, { 'A', 'L', 'A', 'N' }, { 'A', 'L', 'E', 'C' }, { 'A', 'L', 'G', 'A' }, { 'A', 'L', 'I', 'A' }, { 'A', 'L', 'L', 'Y' }, { 'A', 'L', 'M', 'A' }, { 'A', 'L', 'O', 'E' }, { 'A', 'L', 'S', 'O' }, { 'A', 'L', 'T', 'O' }, { 'A', 'L', 'U', 'M' }, { 'A', 'L', 'V', 'A' }, { 'A', 'M', 'E', 'N' }, { 'A', 'M', 'E', 'S' }, { 'A', 'M', 'I', 'D' }, { 'A', 'M', 'M', 'O' }, { 'A', 'M', 'O', 'K' }, { 'A', 'M', 'O', 'S' }, { 'A', 'M', 'R', 'A' }, { 'A', 'N', 'D', 'Y' }, { 'A', 'N', 'E', 'W' }, { 'A', 'N', 'N', 'A' }, { 'A', 'N', 'N', 'E' }, { 'A', 'N', 'T', 'E' }, { 'A', 'N', 'T', 'I' }, { 'A', 'Q', 'U', 'A' }, { 'A', 'R', 'A', 'B' }, { 'A', 'R', 'C', 'H' }, { 'A', 'R', 'E', 'A' }, { 'A', 'R', 'G', 'O' }, { 'A', 'R', 'I', 'D' }, { 'A', 'R', 'M', 'Y' }, { 'A', 'R', 'T', 'S' }, { 'A', 'R', 'T', 'Y' }, { 'A', 'S', 'I', 'A' }, { 'A', 'S', 'K', 'S' }, { 'A', 'T', 'O', 'M' }, { 'A', 'U', 'N', 'T' }, { 'A', 'U', 'R', 'A' }, { 'A', 'U', 'T', 'O' }, { 'A', 'V', 'E', 'R' }, { 'A', 'V', 'I', 'D' }, { 'A', 'V', 'I', 'S' }, { 'A', 'V', 'O', 'N' }, { 'A', 'V', 'O', 'W' }, { 'A', 'W', 'A', 'Y' }, { 'A', 'W', 'R', 'Y' }, { 'B', 'A', 'B', 'E' }, { 'B', 'A', 'B', 'Y' }, { 'B', 'A', 'C', 'H' }, { 'B', 'A', 'C', 'K' }, { 'B', 'A', 'D', 'E' }, { 'B', 'A', 'I', 'L' }, { 'B', 'A', 'I', 'T' }, { 'B', 'A', 'K', 'E' }, { 'B', 'A', 'L', 'D' }, { 'B', 'A', 'L', 'E' }, { 'B', 'A', 'L', 'I' }, { 'B', 'A', 'L', 'K' }, { 'B', 'A', 'L', 'L' }, { 'B', 'A', 'L', 'M' }, { 'B', 'A', 'N', 'D' }, { 'B', 'A', 'N', 'E' }, { 'B', 'A', 'N', 'G' }, { 'B', 'A', 'N', 'K' }, { 'B', 'A', 'R', 'B' }, { 'B', 'A', 'R', 'D' }, { 'B', 'A', 'R', 'E' }, { 'B', 'A', 'R', 'K' }, { 'B', 'A', 'R', 'N' }, { 'B', 'A', 'R', 'R' }, { 'B', 'A', 'S', 'E' }, { 'B', 'A', 'S', 'H' }, { 'B', 'A', 'S', 'K' }, { 'B', 'A', 'S', 'S' }, { 'B', 'A', 'T', 'E' }, { 'B', 'A', 'T', 'H' }, { 'B', 'A', 'W', 'D' }, { 'B', 'A', 'W', 'L' }, { 'B', 'E', 'A', 'D' }, { 'B', 'E', 'A', 'K' }, { 'B', 'E', 'A', 'M' }, { 'B', 'E', 'A', 'N' }, { 'B', 'E', 'A', 'R' }, { 'B', 'E', 'A', 'T' }, { 'B', 'E', 'A', 'U' }, { 'B', 'E', 'C', 'K' }, { 'B', 'E', 'E', 'F' }, { 'B', 'E', 'E', 'N' }, { 'B', 'E', 'E', 'R' }, { 'B', 'E', 'E', 'T' }, { 'B', 'E', 'L', 'A' }, { 'B', 'E', 'L', 'L' }, { 'B', 'E', 'L', 'T' }, { 'B', 'E', 'N', 'D' }, { 'B', 'E', 'N', 'T' }, { 'B', 'E', 'R', 'G' }, { 'B', 'E', 'R', 'N' }, { 'B', 'E', 'R', 'T' }, { 'B', 'E', 'S', 'S' }, { 'B', 'E', 'S', 'T' }, { 'B', 'E', 'T', 'A' }, { 'B', 'E', 'T', 'H' }, { 'B', 'H', 'O', 'Y' }, { 'B', 'I', 'A', 'S' }, { 'B', 'I', 'D', 'E' }, { 'B', 'I', 'E', 'N' }, { 'B', 'I', 'L', 'E' }, { 'B', 'I', 'L', 'K' }, { 'B', 'I', 'L', 'L' }, { 'B', 'I', 'N', 'D' }, { 'B', 'I', 'N', 'G' }, { 'B', 'I', 'R', 'D' }, { 'B', 'I', 'T', 'E' }, { 'B', 'I', 'T', 'S' }, { 'B', 'L', 'A', 'B' }, { 'B', 'L', 'A', 'T' }, { 'B', 'L', 'E', 'D' }, { 'B', 'L', 'E', 'W' }, { 'B', 'L', 'O', 'B' }, { 'B', 'L', 'O', 'C' }, { 'B', 'L', 'O', 'T' }, { 'B', 'L', 'O', 'W' }, { 'B', 'L', 'U', 'E' }, { 'B', 'L', 'U', 'M' }, { 'B', 'L', 'U', 'R' }, { 'B', 'O', 'A', 'R' }, { 'B', 'O', 'A', 'T' }, { 'B', 'O', 'C', 'A' }, { 'B', 'O', 'C', 'K' }, { 'B', 'O', 'D', 'E' }, { 'B', 'O', 'D', 'Y' }, { 'B', 'O', 'G', 'Y' }, { 'B', 'O', 'H', 'R' }, { 'B', 'O', 'I', 'L' }, { 'B', 'O', 'L', 'D' }, { 'B', 'O', 'L', 'O' }, { 'B', 'O', 'L', 'T' }, { 'B', 'O', 'M', 'B' }, { 'B', 'O', 'N', 'A' }, { 'B', 'O', 'N', 'D' }, { 'B', 'O', 'N', 'E' }, { 'B', 'O', 'N', 'G' }, { 'B', 'O', 'N', 'N' }, { 'B', 'O', 'N', 'Y' }, { 'B', 'O', 'O', 'K' }, { 'B', 'O', 'O', 'M' }, { 'B', 'O', 'O', 'N' }, { 'B', 'O', 'O', 'T' }, { 'B', 'O', 'R', 'E' }, { 'B', 'O', 'R', 'G' }, { 'B', 'O', 'R', 'N' }, { 'B', 'O', 'S', 'E' }, { 'B', 'O', 'S', 'S' }, { 'B', 'O', 'T', 'H' }, { 'B', 'O', 'U', 'T' }, { 'B', 'O', 'W', 'L' }, { 'B', 'O', 'Y', 'D' }, { 'B', 'R', 'A', 'D' }, { 'B', 'R', 'A', 'E' }, { 'B', 'R', 'A', 'G' }, { 'B', 'R', 'A', 'N' }, { 'B', 'R', 'A', 'Y' }, { 'B', 'R', 'E', 'D' }, { 'B', 'R', 'E', 'W' }, { 'B', 'R', 'I', 'G' }, { 'B', 'R', 'I', 'M' }, { 'B', 'R', 'O', 'W' }, { 'B', 'U', 'C', 'K' }, { 'B', 'U', 'D', 'D' }, { 'B', 'U', 'F', 'F' }, { 'B', 'U', 'L', 'B' }, { 'B', 'U', 'L', 'K' }, { 'B', 'U', 'L', 'L' }, { 'B', 'U', 'N', 'K' }, { 'B', 'U', 'N', 'T' }, { 'B', 'U', 'O', 'Y' }, { 'B', 'U', 'R', 'G' }, { 'B', 'U', 'R', 'L' }, { 'B', 'U', 'R', 'N' }, { 'B', 'U', 'R', 'R' }, { 'B', 'U', 'R', 'T' }, { 'B', 'U', 'R', 'Y' }, { 'B', 'U', 'S', 'H' }, { 'B', 'U', 'S', 'S' }, { 'B', 'U', 'S', 'T' }, { 'B', 'U', 'S', 'Y' }, { 'B', 'Y', 'T', 'E' }, { 'C', 'A', 'D', 'Y' }, { 'C', 'A', 'F', 'E' }, { 'C', 'A', 'G', 'E' }, { 'C', 'A', 'I', 'N' }, { 'C', 'A', 'K', 'E' }, { 'C', 'A', 'L', 'F' }, { 'C', 'A', 'L', 'L' }, { 'C', 'A', 'L', 'M' }, { 'C', 'A', 'M', 'E' }, { 'C', 'A', 'N', 'E' }, { 'C', 'A', 'N', 'T' }, { 'C', 'A', 'R', 'D' }, { 'C', 'A', 'R', 'E' }, { 'C', 'A', 'R', 'L' }, { 'C', 'A', 'R', 'R' }, { 'C', 'A', 'R', 'T' }, { 'C', 'A', 'S', 'E' }, { 'C', 'A', 'S', 'H' }, { 'C', 'A', 'S', 'K' }, { 'C', 'A', 'S', 'T' }, { 'C', 'A', 'V', 'E' }, { 'C', 'E', 'I', 'L' }, { 'C', 'E', 'L', 'L' }, { 'C', 'E', 'N', 'T' }, { 'C', 'E', 'R', 'N' }, { 'C', 'H', 'A', 'D' }, { 'C', 'H', 'A', 'R' }, { 'C', 'H', 'A', 'T' }, { 'C', 'H', 'A', 'W' }, { 'C', 'H', 'E', 'F' }, { 'C', 'H', 'E', 'N' }, { 'C', 'H', 'E', 'W' }, { 'C', 'H', 'I', 'C' }, { 'C', 'H', 'I', 'N' }, { 'C', 'H', 'O', 'U' }, { 'C', 'H', 'O', 'W' }, { 'C', 'H', 'U', 'B' }, { 'C', 'H', 'U', 'G' }, { 'C', 'H', 'U', 'M' }, { 'C', 'I', 'T', 'E' }, { 'C', 'I', 'T', 'Y' }, { 'C', 'L', 'A', 'D' }, { 'C', 'L', 'A', 'M' }, { 'C', 'L', 'A', 'N' }, { 'C', 'L', 'A', 'W' }, { 'C', 'L', 'A', 'Y' }, { 'C', 'L', 'O', 'D' }, { 'C', 'L', 'O', 'G' }, { 'C', 'L', 'O', 'T' }, { 'C', 'L', 'U', 'B' }, { 'C', 'L', 'U', 'E' }, { 'C', 'O', 'A', 'L' }, { 'C', 'O', 'A', 'T' }, { 'C', 'O', 'C', 'A' }, { 'C', 'O', 'C', 'K' }, { 'C', 'O', 'C', 'O' }, { 'C', 'O', 'D', 'A' }, { 'C', 'O', 'D', 'E' }, { 'C', 'O', 'D', 'Y' }, { 'C', 'O', 'E', 'D' }, { 'C', 'O', 'I', 'L' }, { 'C', 'O', 'I', 'N' }, { 'C', 'O', 'K', 'E' }, { 'C', 'O', 'L', 'A' }, { 'C', 'O', 'L', 'D' }, { 'C', 'O', 'L', 'T' }, { 'C', 'O', 'M', 'A' }, { 'C', 'O', 'M', 'B' }, { 'C', 'O', 'M', 'E' }, { 'C', 'O', 'O', 'K' }, { 'C', 'O', 'O', 'L' }, { 'C', 'O', 'O', 'N' }, { 'C', 'O', 'O', 'T' }, { 'C', 'O', 'R', 'D' }, { 'C', 'O', 'R', 'E' }, { 'C', 'O', 'R', 'K' }, { 'C', 'O', 'R', 'N' }, { 'C', 'O', 'S', 'T' }, { 'C', 'O', 'V', 'E' }, { 'C', 'O', 'W', 'L' }, { 'C', 'R', 'A', 'B' }, { 'C', 'R', 'A', 'G' }, { 'C', 'R', 'A', 'M' }, { 'C', 'R', 'A', 'Y' }, { 'C', 'R', 'E', 'W' }, { 'C', 'R', 'I', 'B' }, { 'C', 'R', 'O', 'W' }, { 'C', 'R', 'U', 'D' }, { 'C', 'U', 'B', 'A' }, { 'C', 'U', 'B', 'E' }, { 'C', 'U', 'F', 'F' }, { 'C', 'U', 'L', 'L' }, { 'C', 'U', 'L', 'T' }, { 'C', 'U', 'N', 'Y' }, { 'C', 'U', 'R', 'B' }, { 'C', 'U', 'R', 'D' }, { 'C', 'U', 'R', 'E' }, { 'C', 'U', 'R', 'L' }, { 'C', 'U', 'R', 'T' }, { 'C', 'U', 'T', 'S' }, { 'D', 'A', 'D', 'E' }, { 'D', 'A', 'L', 'E' }, { 'D', 'A', 'M', 'E' }, { 'D', 'A', 'N', 'A' }, { 'D', 'A', 'N', 'E' }, { 'D', 'A', 'N', 'G' }, { 'D', 'A', 'N', 'K' }, { 'D', 'A', 'R', 'E' }, { 'D', 'A', 'R', 'K' }, { 'D', 'A', 'R', 'N' }, { 'D', 'A', 'R', 'T' }, { 'D', 'A', 'S', 'H' }, { 'D', 'A', 'T', 'A' }, { 'D', 'A', 'T', 'E' }, { 'D', 'A', 'V', 'E' }, { 'D', 'A', 'V', 'Y' }, { 'D', 'A', 'W', 'N' }, { 'D', 'A', 'Y', 'S' }, { 'D', 'E', 'A', 'D' }, { 'D', 'E', 'A', 'F' }, { 'D', 'E', 'A', 'L' }, { 'D', 'E', 'A', 'N' }, { 'D', 'E', 'A', 'R' }, { 'D', 'E', 'B', 'T' }, { 'D', 'E', 'C', 'K' }, { 'D', 'E', 'E', 'D' }, { 'D', 'E', 'E', 'M' }, { 'D', 'E', 'E', 'R' }, { 'D', 'E', 'F', 'T' }, { 'D', 'E', 'F', 'Y' }, { 'D', 'E', 'L', 'L' }, { 'D', 'E', 'N', 'T' }, { 'D', 'E', 'N', 'Y' }, { 'D', 'E', 'S', 'K' }, { 'D', 'I', 'A', 'L' }, { 'D', 'I', 'C', 'E' }, { 'D', 'I', 'E', 'D' }, { 'D', 'I', 'E', 'T' }, { 'D', 'I', 'M', 'E' }, { 'D', 'I', 'N', 'E' }, { 'D', 'I', 'N', 'G' }, { 'D', 'I', 'N', 'T' }, { 'D', 'I', 'R', 'E' }, { 'D', 'I', 'R', 'T' }, { 'D', 'I', 'S', 'C' }, { 'D', 'I', 'S', 'H' }, { 'D', 'I', 'S', 'K' }, { 'D', 'I', 'V', 'E' }, { 'D', 'O', 'C', 'K' }, { 'D', 'O', 'E', 'S' }, { 'D', 'O', 'L', 'E' }, { 'D', 'O', 'L', 'L' }, { 'D', 'O', 'L', 'T' }, { 'D', 'O', 'M', 'E' }, { 'D', 'O', 'N', 'E' }, { 'D', 'O', 'O', 'M' }, { 'D', 'O', 'O', 'R' }, { 'D', 'O', 'R', 'A' }, { 'D', 'O', 'S', 'E' }, { 'D', 'O', 'T', 'E' }, { 'D', 'O', 'U', 'G' }, { 'D', 'O', 'U', 'R' }, { 'D', 'O', 'V', 'E' }, { 'D', 'O', 'W', 'N' }, { 'D', 'R', 'A', 'B' }, { 'D', 'R', 'A', 'G' }, { 'D', 'R', 'A', 'M' }, { 'D', 'R', 'A', 'W' }, { 'D', 'R', 'E', 'W' }, { 'D', 'R', 'U', 'B' }, { 'D', 'R', 'U', 'G' }, { 'D', 'R', 'U', 'M' }, { 'D', 'U', 'A', 'L' }, { 'D', 'U', 'C', 'K' }, { 'D', 'U', 'C', 'T' }, { 'D', 'U', 'E', 'L' }, { 'D', 'U', 'E', 'T' }, { 'D', 'U', 'K', 'E' }, { 'D', 'U', 'L', 'L' }, { 'D', 'U', 'M', 'B' }, { 'D', 'U', 'N', 'E' }, { 'D', 'U', 'N', 'K' }, { 'D', 'U', 'S', 'K' }, { 'D', 'U', 'S', 'T' }, { 'D', 'U', 'T', 'Y' }, { 'E', 'A', 'C', 'H' }, { 'E', 'A', 'R', 'L' }, { 'E', 'A', 'R', 'N' }, { 'E', 'A', 'S', 'E' }, { 'E', 'A', 'S', 'T' }, { 'E', 'A', 'S', 'Y' }, { 'E', 'B', 'E', 'N' }, { 'E', 'C', 'H', 'O' }, { 'E', 'D', 'D', 'Y' }, { 'E', 'D', 'E', 'N' }, { 'E', 'D', 'G', 'E' }, { 'E', 'D', 'G', 'Y' }, { 'E', 'D', 'I', 'T' }, { 'E', 'D', 'N', 'A' }, { 'E', 'G', 'A', 'N' }, { 'E', 'L', 'A', 'N' }, { 'E', 'L', 'B', 'A' }, { 'E', 'L', 'L', 'A' }, { 'E', 'L', 'S', 'E' }, { 'E', 'M', 'I', 'L' }, { 'E', 'M', 'I', 'T' }, { 'E', 'M', 'M', 'A' }, { 'E', 'N', 'D', 'S' }, { 'E', 'R', 'I', 'C' }, { 'E', 'R', 'O', 'S' }, { 'E', 'V', 'E', 'N' }, { 'E', 'V', 'E', 'R' }, { 'E', 'V', 'I', 'L' }, { 'E', 'Y', 'E', 'D' }, { 'F', 'A', 'C', 'E' }, { 'F', 'A', 'C', 'T' }, { 'F', 'A', 'D', 'E' }, { 'F', 'A', 'I', 'L' }, { 'F', 'A', 'I', 'N' }, { 'F', 'A', 'I', 'R' }, { 'F', 'A', 'K', 'E' }, { 'F', 'A', 'L', 'L' }, { 'F', 'A', 'M', 'E' }, { 'F', 'A', 'N', 'G' }, { 'F', 'A', 'R', 'M' }, { 'F', 'A', 'S', 'T' }, { 'F', 'A', 'T', 'E' }, { 'F', 'A', 'W', 'N' }, { 'F', 'E', 'A', 'R' }, { 'F', 'E', 'A', 'T' }, { 'F', 'E', 'E', 'D' }, { 'F', 'E', 'E', 'L' }, { 'F', 'E', 'E', 'T' }, { 'F', 'E', 'L', 'L' }, { 'F', 'E', 'L', 'T' }, { 'F', 'E', 'N', 'D' }, { 'F', 'E', 'R', 'N' }, { 'F', 'E', 'S', 'T' }, { 'F', 'E', 'U', 'D' }, { 'F', 'I', 'E', 'F' }, { 'F', 'I', 'G', 'S' }, { 'F', 'I', 'L', 'E' }, { 'F', 'I', 'L', 'L' }, { 'F', 'I', 'L', 'M' }, { 'F', 'I', 'N', 'D' }, { 'F', 'I', 'N', 'E' }, { 'F', 'I', 'N', 'K' }, { 'F', 'I', 'R', 'E' }, { 'F', 'I', 'R', 'M' }, { 'F', 'I', 'S', 'H' }, { 'F', 'I', 'S', 'K' }, { 'F', 'I', 'S', 'T' }, { 'F', 'I', 'T', 'S' }, { 'F', 'I', 'V', 'E' }, { 'F', 'L', 'A', 'G' }, { 'F', 'L', 'A', 'K' }, { 'F', 'L', 'A', 'M' }, { 'F', 'L', 'A', 'T' }, { 'F', 'L', 'A', 'W' }, { 'F', 'L', 'E', 'A' }, { 'F', 'L', 'E', 'D' }, { 'F', 'L', 'E', 'W' }, { 'F', 'L', 'I', 'T' }, { 'F', 'L', 'O', 'C' }, { 'F', 'L', 'O', 'G' }, { 'F', 'L', 'O', 'W' }, { 'F', 'L', 'U', 'B' }, { 'F', 'L', 'U', 'E' }, { 'F', 'O', 'A', 'L' }, { 'F', 'O', 'A', 'M' }, { 'F', 'O', 'G', 'Y' }, { 'F', 'O', 'I', 'L' }, { 'F', 'O', 'L', 'D' }, { 'F', 'O', 'L', 'K' }, { 'F', 'O', 'N', 'D' }, { 'F', 'O', 'N', 'T' }, { 'F', 'O', 'O', 'D' }, { 'F', 'O', 'O', 'L' }, { 'F', 'O', 'O', 'T' }, { 'F', 'O', 'R', 'D' }, { 'F', 'O', 'R', 'E' }, { 'F', 'O', 'R', 'K' }, { 'F', 'O', 'R', 'M' }, { 'F', 'O', 'R', 'T' }, { 'F', 'O', 'S', 'S' }, { 'F', 'O', 'U', 'L' }, { 'F', 'O', 'U', 'R' }, { 'F', 'O', 'W', 'L' }, { 'F', 'R', 'A', 'U' }, { 'F', 'R', 'A', 'Y' }, { 'F', 'R', 'E', 'D' }, { 'F', 'R', 'E', 'E' }, { 'F', 'R', 'E', 'T' }, { 'F', 'R', 'E', 'Y' }, { 'F', 'R', 'O', 'G' }, { 'F', 'R', 'O', 'M' }, { 'F', 'U', 'E', 'L' }, { 'F', 'U', 'L', 'L' }, { 'F', 'U', 'M', 'E' }, { 'F', 'U', 'N', 'D' }, { 'F', 'U', 'N', 'K' }, { 'F', 'U', 'R', 'Y' }, { 'F', 'U', 'S', 'E' }, { 'F', 'U', 'S', 'S' }, { 'G', 'A', 'F', 'F' }, { 'G', 'A', 'G', 'E' }, { 'G', 'A', 'I', 'L' }, { 'G', 'A', 'I', 'N' }, { 'G', 'A', 'I', 'T' }, { 'G', 'A', 'L', 'A' }, { 'G', 'A', 'L', 'E' }, { 'G', 'A', 'L', 'L' }, { 'G', 'A', 'L', 'T' }, { 'G', 'A', 'M', 'E' }, { 'G', 'A', 'N', 'G' }, { 'G', 'A', 'R', 'B' }, { 'G', 'A', 'R', 'Y' }, { 'G', 'A', 'S', 'H' }, { 'G', 'A', 'T', 'E' }, { 'G', 'A', 'U', 'L' }, { 'G', 'A', 'U', 'R' }, { 'G', 'A', 'V', 'E' }, { 'G', 'A', 'W', 'K' }, { 'G', 'E', 'A', 'R' }, { 'G', 'E', 'L', 'D' }, { 'G', 'E', 'N', 'E' }, { 'G', 'E', 'N', 'T' }, { 'G', 'E', 'R', 'M' }, { 'G', 'E', 'T', 'S' }, { 'G', 'I', 'B', 'E' }, { 'G', 'I', 'F', 'T' }, { 'G', 'I', 'L', 'D' }, { 'G', 'I', 'L', 'L' }, { 'G', 'I', 'L', 'T' }, { 'G', 'I', 'N', 'A' }, { 'G', 'I', 'R', 'D' }, { 'G', 'I', 'R', 'L' }, { 'G', 'I', 'S', 'T' }, { 'G', 'I', 'V', 'E' }, { 'G', 'L', 'A', 'D' }, { 'G', 'L', 'E', 'E' }, { 'G', 'L', 'E', 'N' }, { 'G', 'L', 'I', 'B' }, { 'G', 'L', 'O', 'B' }, { 'G', 'L', 'O', 'M' }, { 'G', 'L', 'O', 'W' }, { 'G', 'L', 'U', 'E' }, { 'G', 'L', 'U', 'M' }, { 'G', 'L', 'U', 'T' }, { 'G', 'O', 'A', 'D' }, { 'G', 'O', 'A', 'L' }, { 'G', 'O', 'A', 'T' }, { 'G', 'O', 'E', 'R' }, { 'G', 'O', 'E', 'S' }, { 'G', 'O', 'L', 'D' }, { 'G', 'O', 'L', 'F' }, { 'G', 'O', 'N', 'E' }, { 'G', 'O', 'N', 'G' }, { 'G', 'O', 'O', 'D' }, { 'G', 'O', 'O', 'F' }, { 'G', 'O', 'R', 'E' }, { 'G', 'O', 'R', 'Y' }, { 'G', 'O', 'S', 'H' }, { 'G', 'O', 'U', 'T' }, { 'G', 'O', 'W', 'N' }, { 'G', 'R', 'A', 'B' }, { 'G', 'R', 'A', 'D' }, { 'G', 'R', 'A', 'Y' }, { 'G', 'R', 'E', 'G' }, { 'G', 'R', 'E', 'W' }, { 'G', 'R', 'E', 'Y' }, { 'G', 'R', 'I', 'D' }, { 'G', 'R', 'I', 'M' }, { 'G', 'R', 'I', 'N' }, { 'G', 'R', 'I', 'T' }, { 'G', 'R', 'O', 'W' }, { 'G', 'R', 'U', 'B' }, { 'G', 'U', 'L', 'F' }, { 'G', 'U', 'L', 'L' }, { 'G', 'U', 'N', 'K' }, { 'G', 'U', 'R', 'U' }, { 'G', 'U', 'S', 'H' }, { 'G', 'U', 'S', 'T' }, { 'G', 'W', 'E', 'N' }, { 'G', 'W', 'Y', 'N' }, { 'H', 'A', 'A', 'G' }, { 'H', 'A', 'A', 'S' }, { 'H', 'A', 'C', 'K' }, { 'H', 'A', 'I', 'L' }, { 'H', 'A', 'I', 'R' }, { 'H', 'A', 'L', 'E' }, { 'H', 'A', 'L', 'F' }, { 'H', 'A', 'L', 'L' }, { 'H', 'A', 'L', 'O' }, { 'H', 'A', 'L', 'T' }, { 'H', 'A', 'N', 'D' }, { 'H', 'A', 'N', 'G' }, { 'H', 'A', 'N', 'K' }, { 'H', 'A', 'N', 'S' }, { 'H', 'A', 'R', 'D' }, { 'H', 'A', 'R', 'K' }, { 'H', 'A', 'R', 'M' }, { 'H', 'A', 'R', 'T' }, { 'H', 'A', 'S', 'H' }, { 'H', 'A', 'S', 'T' }, { 'H', 'A', 'T', 'E' }, { 'H', 'A', 'T', 'H' }, { 'H', 'A', 'U', 'L' }, { 'H', 'A', 'V', 'E' }, { 'H', 'A', 'W', 'K' }, { 'H', 'A', 'Y', 'S' }, { 'H', 'E', 'A', 'D' }, { 'H', 'E', 'A', 'L' }, { 'H', 'E', 'A', 'R' }, { 'H', 'E', 'A', 'T' }, { 'H', 'E', 'B', 'E' }, { 'H', 'E', 'C', 'K' }, { 'H', 'E', 'E', 'D' }, { 'H', 'E', 'E', 'L' }, { 'H', 'E', 'F', 'T' }, { 'H', 'E', 'L', 'D' }, { 'H', 'E', 'L', 'L' }, { 'H', 'E', 'L', 'M' }, { 'H', 'E', 'R', 'B' }, { 'H', 'E', 'R', 'D' }, { 'H', 'E', 'R', 'E' }, { 'H', 'E', 'R', 'O' }, { 'H', 'E', 'R', 'S' }, { 'H', 'E', 'S', 'S' }, { 'H', 'E', 'W', 'N' }, { 'H', 'I', 'C', 'K' }, { 'H', 'I', 'D', 'E' }, { 'H', 'I', 'G', 'H' }, { 'H', 'I', 'K', 'E' }, { 'H', 'I', 'L', 'L' }, { 'H', 'I', 'L', 'T' }, { 'H', 'I', 'N', 'D' }, { 'H', 'I', 'N', 'T' }, { 'H', 'I', 'R', 'E' }, { 'H', 'I', 'S', 'S' }, { 'H', 'I', 'V', 'E' }, { 'H', 'O', 'B', 'O' }, { 'H', 'O', 'C', 'K' }, { 'H', 'O', 'F', 'F' }, { 'H', 'O', 'L', 'D' }, { 'H', 'O', 'L', 'E' }, { 'H', 'O', 'L', 'M' }, { 'H', 'O', 'L', 'T' }, { 'H', 'O', 'M', 'E' }, { 'H', 'O', 'N', 'E' }, { 'H', 'O', 'N', 'K' }, { 'H', 'O', 'O', 'D' }, { 'H', 'O', 'O', 'F' }, { 'H', 'O', 'O', 'K' }, { 'H', 'O', 'O', 'T' }, { 'H', 'O', 'R', 'N' }, { 'H', 'O', 'S', 'E' }, { 'H', 'O', 'S', 'T' }, { 'H', 'O', 'U', 'R' }, { 'H', 'O', 'V', 'E' }, { 'H', 'O', 'W', 'E' }, { 'H', 'O', 'W', 'L' }, { 'H', 'O', 'Y', 'T' }, { 'H', 'U', 'C', 'K' }, { 'H', 'U', 'E', 'D' }, { 'H', 'U', 'F', 'F' }, { 'H', 'U', 'G', 'E' }, { 'H', 'U', 'G', 'H' }, { 'H', 'U', 'G', 'O' }, { 'H', 'U', 'L', 'K' }, { 'H', 'U', 'L', 'L' }, { 'H', 'U', 'N', 'K' }, { 'H', 'U', 'N', 'T' }, { 'H', 'U', 'R', 'D' }, { 'H', 'U', 'R', 'L' }, { 'H', 'U', 'R', 'T' }, { 'H', 'U', 'S', 'H' }, { 'H', 'Y', 'D', 'E' }, { 'H', 'Y', 'M', 'N' }, { 'I', 'B', 'I', 'S' }, { 'I', 'C', 'O', 'N' }, { 'I', 'D', 'E', 'A' }, { 'I', 'D', 'L', 'E' }, { 'I', 'F', 'F', 'Y' }, { 'I', 'N', 'C', 'A' }, { 'I', 'N', 'C', 'H' }, { 'I', 'N', 'T', 'O' }, { 'I', 'O', 'N', 'S' }, { 'I', 'O', 'T', 'A' }, { 'I', 'O', 'W', 'A' }, { 'I', 'R', 'I', 'S' }, { 'I', 'R', 'M', 'A' }, { 'I', 'R', 'O', 'N' }, { 'I', 'S', 'L', 'E' }, { 'I', 'T', 'C', 'H' }, { 'I', 'T', 'E', 'M' }, { 'I', 'V', 'A', 'N' }, { 'J', 'A', 'C', 'K' }, { 'J', 'A', 'D', 'E' }, { 'J', 'A', 'I', 'L' }, { 'J', 'A', 'K', 'E' }, { 'J', 'A', 'N', 'E' }, { 'J', 'A', 'V', 'A' }, { 'J', 'E', 'A', 'N' }, { 'J', 'E', 'F', 'F' }, { 'J', 'E', 'R', 'K' }, { 'J', 'E', 'S', 'S' }, { 'J', 'E', 'S', 'T' }, { 'J', 'I', 'B', 'E' }, { 'J', 'I', 'L', 'L' }, { 'J', 'I', 'L', 'T' }, { 'J', 'I', 'V', 'E' }, { 'J', 'O', 'A', 'N' }, { 'J', 'O', 'B', 'S' }, { 'J', 'O', 'C', 'K' }, { 'J', 'O', 'E', 'L' }, { 'J', 'O', 'E', 'Y' }, { 'J', 'O', 'H', 'N' }, { 'J', 'O', 'I', 'N' }, { 'J', 'O', 'K', 'E' }, { 'J', 'O', 'L', 'T' }, { 'J', 'O', 'V', 'E' }, { 'J', 'U', 'D', 'D' }, { 'J', 'U', 'D', 'E' }, { 'J', 'U', 'D', 'O' }, { 'J', 'U', 'D', 'Y' }, { 'J', 'U', 'J', 'U' }, { 'J', 'U', 'K', 'E' }, { 'J', 'U', 'L', 'Y' }, { 'J', 'U', 'N', 'E' }, { 'J', 'U', 'N', 'K' }, { 'J', 'U', 'N', 'O' }, { 'J', 'U', 'R', 'Y' }, { 'J', 'U', 'S', 'T' }, { 'J', 'U', 'T', 'E' }, { 'K', 'A', 'H', 'N' }, { 'K', 'A', 'L', 'E' }, { 'K', 'A', 'N', 'E' }, { 'K', 'A', 'N', 'T' }, { 'K', 'A', 'R', 'L' }, { 'K', 'A', 'T', 'E' }, { 'K', 'E', 'E', 'L' }, { 'K', 'E', 'E', 'N' }, { 'K', 'E', 'N', 'O' }, { 'K', 'E', 'N', 'T' }, { 'K', 'E', 'R', 'N' }, { 'K', 'E', 'R', 'R' }, { 'K', 'E', 'Y', 'S' }, { 'K', 'I', 'C', 'K' }, { 'K', 'I', 'L', 'L' }, { 'K', 'I', 'N', 'D' }, { 'K', 'I', 'N', 'G' }, { 'K', 'I', 'R', 'K' }, { 'K', 'I', 'S', 'S' }, { 'K', 'I', 'T', 'E' }, { 'K', 'L', 'A', 'N' }, { 'K', 'N', 'E', 'E' }, { 'K', 'N', 'E', 'W' }, { 'K', 'N', 'I', 'T' }, { 'K', 'N', 'O', 'B' }, { 'K', 'N', 'O', 'T' }, { 'K', 'N', 'O', 'W' }, { 'K', 'O', 'C', 'H' }, { 'K', 'O', 'N', 'G' }, { 'K', 'U', 'D', 'O' }, { 'K', 'U', 'R', 'D' }, { 'K', 'U', 'R', 'T' }, { 'K', 'Y', 'L', 'E' }, { 'L', 'A', 'C', 'E' }, { 'L', 'A', 'C', 'K' }, { 'L', 'A', 'C', 'Y' }, { 'L', 'A', 'D', 'Y' }, { 'L', 'A', 'I', 'D' }, { 'L', 'A', 'I', 'N' }, { 'L', 'A', 'I', 'R' }, { 'L', 'A', 'K', 'E' }, { 'L', 'A', 'M', 'B' }, { 'L', 'A', 'M', 'E' }, { 'L', 'A', 'N', 'D' }, { 'L', 'A', 'N', 'E' }, { 'L', 'A', 'N', 'G' }, { 'L', 'A', 'R', 'D' }, { 'L', 'A', 'R', 'K' }, { 'L', 'A', 'S', 'S' }, { 'L', 'A', 'S', 'T' }, { 'L', 'A', 'T', 'E' }, { 'L', 'A', 'U', 'D' }, { 'L', 'A', 'V', 'A' }, { 'L', 'A', 'W', 'N' }, { 'L', 'A', 'W', 'S' }, { 'L', 'A', 'Y', 'S' }, { 'L', 'E', 'A', 'D' }, { 'L', 'E', 'A', 'F' }, { 'L', 'E', 'A', 'K' }, { 'L', 'E', 'A', 'N' }, { 'L', 'E', 'A', 'R' }, { 'L', 'E', 'E', 'K' }, { 'L', 'E', 'E', 'R' }, { 'L', 'E', 'F', 'T' }, { 'L', 'E', 'N', 'D' }, { 'L', 'E', 'N', 'S' }, { 'L', 'E', 'N', 'T' }, { 'L', 'E', 'O', 'N' }, { 'L', 'E', 'S', 'K' }, { 'L', 'E', 'S', 'S' }, { 'L', 'E', 'S', 'T' }, { 'L', 'E', 'T', 'S' }, { 'L', 'I', 'A', 'R' }, { 'L', 'I', 'C', 'E' }, { 'L', 'I', 'C', 'K' }, { 'L', 'I', 'E', 'D' }, { 'L', 'I', 'E', 'N' }, { 'L', 'I', 'E', 'S' }, { 'L', 'I', 'E', 'U' }, { 'L', 'I', 'F', 'E' }, { 'L', 'I', 'F', 'T' }, { 'L', 'I', 'K', 'E' }, { 'L', 'I', 'L', 'A' }, { 'L', 'I', 'L', 'T' }, { 'L', 'I', 'L', 'Y' }, { 'L', 'I', 'M', 'A' }, { 'L', 'I', 'M', 'B' }, { 'L', 'I', 'M', 'E' }, { 'L', 'I', 'N', 'D' }, { 'L', 'I', 'N', 'E' }, { 'L', 'I', 'N', 'K' }, { 'L', 'I', 'N', 'T' }, { 'L', 'I', 'O', 'N' }, { 'L', 'I', 'S', 'A' }, { 'L', 'I', 'S', 'T' }, { 'L', 'I', 'V', 'E' }, { 'L', 'O', 'A', 'D' }, { 'L', 'O', 'A', 'F' }, { 'L', 'O', 'A', 'M' }, { 'L', 'O', 'A', 'N' }, { 'L', 'O', 'C', 'K' }, { 'L', 'O', 'F', 'T' }, { 'L', 'O', 'G', 'E' }, { 'L', 'O', 'I', 'S' }, { 'L', 'O', 'L', 'A' }, { 'L', 'O', 'N', 'E' }, { 'L', 'O', 'N', 'G' }, { 'L', 'O', 'O', 'K' }, { 'L', 'O', 'O', 'N' }, { 'L', 'O', 'O', 'T' }, { 'L', 'O', 'R', 'D' }, { 'L', 'O', 'R', 'E' }, { 'L', 'O', 'S', 'E' }, { 'L', 'O', 'S', 'S' }, { 'L', 'O', 'S', 'T' }, { 'L', 'O', 'U', 'D' }, { 'L', 'O', 'V', 'E' }, { 'L', 'O', 'W', 'E' }, { 'L', 'U', 'C', 'K' }, { 'L', 'U', 'C', 'Y' }, { 'L', 'U', 'G', 'E' }, { 'L', 'U', 'K', 'E' }, { 'L', 'U', 'L', 'U' }, { 'L', 'U', 'N', 'D' }, { 'L', 'U', 'N', 'G' }, { 'L', 'U', 'R', 'A' }, { 'L', 'U', 'R', 'E' }, { 'L', 'U', 'R', 'K' }, { 'L', 'U', 'S', 'H' }, { 'L', 'U', 'S', 'T' }, { 'L', 'Y', 'L', 'E' }, { 'L', 'Y', 'N', 'N' }, { 'L', 'Y', 'O', 'N' }, { 'L', 'Y', 'R', 'A' }, { 'M', 'A', 'C', 'E' }, { 'M', 'A', 'D', 'E' }, { 'M', 'A', 'G', 'I' }, { 'M', 'A', 'I', 'D' }, { 'M', 'A', 'I', 'L' }, { 'M', 'A', 'I', 'N' }, { 'M', 'A', 'K', 'E' }, { 'M', 'A', 'L', 'E' }, { 'M', 'A', 'L', 'I' }, { 'M', 'A', 'L', 'L' }, { 'M', 'A', 'L', 'T' }, { 'M', 'A', 'N', 'A' }, { 'M', 'A', 'N', 'N' }, { 'M', 'A', 'N', 'Y' }, { 'M', 'A', 'R', 'C' }, { 'M', 'A', 'R', 'E' }, { 'M', 'A', 'R', 'K' }, { 'M', 'A', 'R', 'S' }, { 'M', 'A', 'R', 'T' }, { 'M', 'A', 'R', 'Y' }, { 'M', 'A', 'S', 'H' }, { 'M', 'A', 'S', 'K' }, { 'M', 'A', 'S', 'S' }, { 'M', 'A', 'S', 'T' }, { 'M', 'A', 'T', 'E' }, { 'M', 'A', 'T', 'H' }, { 'M', 'A', 'U', 'L' }, { 'M', 'A', 'Y', 'O' }, { 'M', 'E', 'A', 'D' }, { 'M', 'E', 'A', 'L' }, { 'M', 'E', 'A', 'N' }, { 'M', 'E', 'A', 'T' }, { 'M', 'E', 'E', 'K' }, { 'M', 'E', 'E', 'T' }, { 'M', 'E', 'L', 'D' }, { 'M', 'E', 'L', 'T' }, { 'M', 'E', 'M', 'O' }, { 'M', 'E', 'N', 'D' }, { 'M', 'E', 'N', 'U' }, { 'M', 'E', 'R', 'T' }, { 'M', 'E', 'S', 'H' }, { 'M', 'E', 'S', 'S' }, { 'M', 'I', 'C', 'E' }, { 'M', 'I', 'K', 'E' }, { 'M', 'I', 'L', 'D' }, { 'M', 'I', 'L', 'E' }, { 'M', 'I', 'L', 'K' }, { 'M', 'I', 'L', 'L' }, { 'M', 'I', 'L', 'T' }, { 'M', 'I', 'M', 'I' }, { 'M', 'I', 'N', 'D' }, { 'M', 'I', 'N', 'E' }, { 'M', 'I', 'N', 'I' }, { 'M', 'I', 'N', 'K' }, { 'M', 'I', 'N', 'T' }, { 'M', 'I', 'R', 'E' }, { 'M', 'I', 'S', 'S' }, { 'M', 'I', 'S', 'T' }, { 'M', 'I', 'T', 'E' }, { 'M', 'I', 'T', 'T' }, { 'M', 'O', 'A', 'N' }, { 'M', 'O', 'A', 'T' }, { 'M', 'O', 'C', 'K' }, { 'M', 'O', 'D', 'E' }, { 'M', 'O', 'L', 'D' }, { 'M', 'O', 'L', 'E' }, { 'M', 'O', 'L', 'L' }, { 'M', 'O', 'L', 'T' }, { 'M', 'O', 'N', 'A' }, { 'M', 'O', 'N', 'K' }, { 'M', 'O', 'N', 'T' }, { 'M', 'O', 'O', 'D' }, { 'M', 'O', 'O', 'N' }, { 'M', 'O', 'O', 'R' }, { 'M', 'O', 'O', 'T' }, { 'M', 'O', 'R', 'E' }, { 'M', 'O', 'R', 'N' }, { 'M', 'O', 'R', 'T' }, { 'M', 'O', 'S', 'S' }, { 'M', 'O', 'S', 'T' }, { 'M', 'O', 'T', 'H' }, { 'M', 'O', 'V', 'E' }, { 'M', 'U', 'C', 'H' }, { 'M', 'U', 'C', 'K' }, { 'M', 'U', 'D', 'D' }, { 'M', 'U', 'F', 'F' }, { 'M', 'U', 'L', 'E' }, { 'M', 'U', 'L', 'L' }, { 'M', 'U', 'R', 'K' }, { 'M', 'U', 'S', 'H' }, { 'M', 'U', 'S', 'T' }, { 'M', 'U', 'T', 'E' }, { 'M', 'U', 'T', 'T' }, { 'M', 'Y', 'R', 'A' }, { 'M', 'Y', 'T', 'H' }, { 'N', 'A', 'G', 'Y' }, { 'N', 'A', 'I', 'L' }, { 'N', 'A', 'I', 'R' }, { 'N', 'A', 'M', 'E' }, { 'N', 'A', 'R', 'Y' }, { 'N', 'A', 'S', 'H' }, { 'N', 'A', 'V', 'E' }, { 'N', 'A', 'V', 'Y' }, { 'N', 'E', 'A', 'L' }, { 'N', 'E', 'A', 'R' }, { 'N', 'E', 'A', 'T' }, { 'N', 'E', 'C', 'K' }, { 'N', 'E', 'E', 'D' }, { 'N', 'E', 'I', 'L' }, { 'N', 'E', 'L', 'L' }, { 'N', 'E', 'O', 'N' }, { 'N', 'E', 'R', 'O' }, { 'N', 'E', 'S', 'S' }, { 'N', 'E', 'S', 'T' }, { 'N', 'E', 'W', 'S' }, { 'N', 'E', 'W', 'T' }, { 'N', 'I', 'B', 'S' }, { 'N', 'I', 'C', 'E' }, { 'N', 'I', 'C', 'K' }, { 'N', 'I', 'L', 'E' }, { 'N', 'I', 'N', 'A' }, { 'N', 'I', 'N', 'E' }, { 'N', 'O', 'A', 'H' }, { 'N', 'O', 'D', 'E' }, { 'N', 'O', 'E', 'L' }, { 'N', 'O', 'L', 'L' }, { 'N', 'O', 'N', 'E' }, { 'N', 'O', 'O', 'K' }, { 'N', 'O', 'O', 'N' }, { 'N', 'O', 'R', 'M' }, { 'N', 'O', 'S', 'E' }, { 'N', 'O', 'T', 'E' }, { 'N', 'O', 'U', 'N' }, { 'N', 'O', 'V', 'A' }, { 'N', 'U', 'D', 'E' }, { 'N', 'U', 'L', 'L' }, { 'N', 'U', 'M', 'B' }, { 'O', 'A', 'T', 'H' }, { 'O', 'B', 'E', 'Y' }, { 'O', 'B', 'O', 'E' }, { 'O', 'D', 'I', 'N' }, { 'O', 'H', 'I', 'O' }, { 'O', 'I', 'L', 'Y' }, { 'O', 'I', 'N', 'T' }, { 'O', 'K', 'A', 'Y' }, { 'O', 'L', 'A', 'F' }, { 'O', 'L', 'D', 'Y' }, { 'O', 'L', 'G', 'A' }, { 'O', 'L', 'I', 'N' }, { 'O', 'M', 'A', 'N' }, { 'O', 'M', 'E', 'N' }, { 'O', 'M', 'I', 'T' }, { 'O', 'N', 'C', 'E' }, { 'O', 'N', 'E', 'S' }, { 'O', 'N', 'L', 'Y' }, { 'O', 'N', 'T', 'O' }, { 'O', 'N', 'U', 'S' }, { 'O', 'R', 'A', 'L' }, { 'O', 'R', 'G', 'Y' }, { 'O', 'S', 'L', 'O' }, { 'O', 'T', 'I', 'S' }, { 'O', 'T', 'T', 'O' }, { 'O', 'U', 'C', 'H' }, { 'O', 'U', 'S', 'T' }, { 'O', 'U', 'T', 'S' }, { 'O', 'V', 'A', 'L' }, { 'O', 'V', 'E', 'N' }, { 'O', 'V', 'E', 'R' }, { 'O', 'W', 'L', 'Y' }, { 'O', 'W', 'N', 'S' }, { 'Q', 'U', 'A', 'D' }, { 'Q', 'U', 'I', 'T' }, { 'Q', 'U', 'O', 'D' }, { 'R', 'A', 'C', 'E' }, { 'R', 'A', 'C', 'K' }, { 'R', 'A', 'C', 'Y' }, { 'R', 'A', 'F', 'T' }, { 'R', 'A', 'G', 'E' }, { 'R', 'A', 'I', 'D' }, { 'R', 'A', 'I', 'L' }, { 'R', 'A', 'I', 'N' }, { 'R', 'A', 'K', 'E' }, { 'R', 'A', 'N', 'K' }, { 'R', 'A', 'N', 'T' }, { 'R', 'A', 'R', 'E' }, { 'R', 'A', 'S', 'H' }, { 'R', 'A', 'T', 'E' }, { 'R', 'A', 'V', 'E' }, { 'R', 'A', 'Y', 'S' }, { 'R', 'E', 'A', 'D' }, { 'R', 'E', 'A', 'L' }, { 'R', 'E', 'A', 'M' }, { 'R', 'E', 'A', 'R' }, { 'R', 'E', 'C', 'K' }, { 'R', 'E', 'E', 'D' }, { 'R', 'E', 'E', 'F' }, { 'R', 'E', 'E', 'K' }, { 'R', 'E', 'E', 'L' }, { 'R', 'E', 'I', 'D' }, { 'R', 'E', 'I', 'N' }, { 'R', 'E', 'N', 'A' }, { 'R', 'E', 'N', 'D' }, { 'R', 'E', 'N', 'T' }, { 'R', 'E', 'S', 'T' }, { 'R', 'I', 'C', 'E' }, { 'R', 'I', 'C', 'H' }, { 'R', 'I', 'C', 'K' }, { 'R', 'I', 'D', 'E' }, { 'R', 'I', 'F', 'T' }, { 'R', 'I', 'L', 'L' }, { 'R', 'I', 'M', 'E' }, { 'R', 'I', 'N', 'G' }, { 'R', 'I', 'N', 'K' }, { 'R', 'I', 'S', 'E' }, { 'R', 'I', 'S', 'K' }, { 'R', 'I', 'T', 'E' }, { 'R', 'O', 'A', 'D' }, { 'R', 'O', 'A', 'M' }, { 'R', 'O', 'A', 'R' }, { 'R', 'O', 'B', 'E' }, { 'R', 'O', 'C', 'K' }, { 'R', 'O', 'D', 'E' }, { 'R', 'O', 'I', 'L' }, { 'R', 'O', 'L', 'L' }, { 'R', 'O', 'M', 'E' }, { 'R', 'O', 'O', 'D' }, { 'R', 'O', 'O', 'F' }, { 'R', 'O', 'O', 'K' }, { 'R', 'O', 'O', 'M' }, { 'R', 'O', 'O', 'T' }, { 'R', 'O', 'S', 'A' }, { 'R', 'O', 'S', 'E' }, { 'R', 'O', 'S', 'S' }, { 'R', 'O', 'S', 'Y' }, { 'R', 'O', 'T', 'H' }, { 'R', 'O', 'U', 'T' }, { 'R', 'O', 'V', 'E' }, { 'R', 'O', 'W', 'E' }, { 'R', 'O', 'W', 'S' }, { 'R', 'U', 'B', 'E' }, { 'R', 'U', 'B', 'Y' }, { 'R', 'U', 'D', 'E' }, { 'R', 'U', 'D', 'Y' }, { 'R', 'U', 'I', 'N' }, { 'R', 'U', 'L', 'E' }, { 'R', 'U', 'N', 'G' }, { 'R', 'U', 'N', 'S' }, { 'R', 'U', 'N', 'T' }, { 'R', 'U', 'S', 'E' }, { 'R', 'U', 'S', 'H' }, { 'R', 'U', 'S', 'K' }, { 'R', 'U', 'S', 'S' }, { 'R', 'U', 'S', 'T' }, { 'R', 'U', 'T', 'H' }, { 'S', 'A', 'C', 'K' }, { 'S', 'A', 'F', 'E' }, { 'S', 'A', 'G', 'E' }, { 'S', 'A', 'I', 'D' }, { 'S', 'A', 'I', 'L' }, { 'S', 'A', 'L', 'E' }, { 'S', 'A', 'L', 'K' }, { 'S', 'A', 'L', 'T' }, { 'S', 'A', 'M', 'E' }, { 'S', 'A', 'N', 'D' }, { 'S', 'A', 'N', 'E' }, { 'S', 'A', 'N', 'G' }, { 'S', 'A', 'N', 'K' }, { 'S', 'A', 'R', 'A' }, { 'S', 'A', 'U', 'L' }, { 'S', 'A', 'V', 'E' }, { 'S', 'A', 'Y', 'S' }, { 'S', 'C', 'A', 'N' }, { 'S', 'C', 'A', 'R' }, { 'S', 'C', 'A', 'T' }, { 'S', 'C', 'O', 'T' }, { 'S', 'E', 'A', 'L' }, { 'S', 'E', 'A', 'M' }, { 'S', 'E', 'A', 'R' }, { 'S', 'E', 'A', 'T' }, { 'S', 'E', 'E', 'D' }, { 'S', 'E', 'E', 'K' }, { 'S', 'E', 'E', 'M' }, { 'S', 'E', 'E', 'N' }, { 'S', 'E', 'E', 'S' }, { 'S', 'E', 'L', 'F' }, { 'S', 'E', 'L', 'L' }, { 'S', 'E', 'N', 'D' }, { 'S', 'E', 'N', 'T' }, { 'S', 'E', 'T', 'S' }, { 'S', 'E', 'W', 'N' }, { 'S', 'H', 'A', 'G' }, { 'S', 'H', 'A', 'M' }, { 'S', 'H', 'A', 'W' }, { 'S', 'H', 'A', 'Y' }, { 'S', 'H', 'E', 'D' }, { 'S', 'H', 'I', 'M' }, { 'S', 'H', 'I', 'N' }, { 'S', 'H', 'O', 'D' }, { 'S', 'H', 'O', 'E' }, { 'S', 'H', 'O', 'T' }, { 'S', 'H', 'O', 'W' }, { 'S', 'H', 'U', 'N' }, { 'S', 'H', 'U', 'T' }, { 'S', 'I', 'C', 'K' }, { 'S', 'I', 'D', 'E' }, { 'S', 'I', 'F', 'T' }, { 'S', 'I', 'G', 'H' }, { 'S', 'I', 'G', 'N' }, { 'S', 'I', 'L', 'K' }, { 'S', 'I', 'L', 'L' }, { 'S', 'I', 'L', 'O' }, { 'S', 'I', 'L', 'T' }, { 'S', 'I', 'N', 'E' }, { 'S', 'I', 'N', 'G' }, { 'S', 'I', 'N', 'K' }, { 'S', 'I', 'R', 'E' }, { 'S', 'I', 'T', 'E' }, { 'S', 'I', 'T', 'S' }, { 'S', 'I', 'T', 'U' }, { 'S', 'K', 'A', 'T' }, { 'S', 'K', 'E', 'W' }, { 'S', 'K', 'I', 'D' }, { 'S', 'K', 'I', 'M' }, { 'S', 'K', 'I', 'N' }, { 'S', 'K', 'I', 'T' }, { 'S', 'L', 'A', 'B' }, { 'S', 'L', 'A', 'M' }, { 'S', 'L', 'A', 'T' }, { 'S', 'L', 'A', 'Y' }, { 'S', 'L', 'E', 'D' }, { 'S', 'L', 'E', 'W' }, { 'S', 'L', 'I', 'D' }, { 'S', 'L', 'I', 'M' }, { 'S', 'L', 'I', 'T' }, { 'S', 'L', 'O', 'B' }, { 'S', 'L', 'O', 'G' }, { 'S', 'L', 'O', 'T' }, { 'S', 'L', 'O', 'W' }, { 'S', 'L', 'U', 'G' }, { 'S', 'L', 'U', 'M' }, { 'S', 'L', 'U', 'R' }, { 'S', 'M', 'O', 'G' }, { 'S', 'M', 'U', 'G' }, { 'S', 'N', 'A', 'G' }, { 'S', 'N', 'O', 'B' }, { 'S', 'N', 'O', 'W' }, { 'S', 'N', 'U', 'B' }, { 'S', 'N', 'U', 'G' }, { 'S', 'O', 'A', 'K' }, { 'S', 'O', 'A', 'R' }, { 'S', 'O', 'C', 'K' }, { 'S', 'O', 'D', 'A' }, { 'S', 'O', 'F', 'A' }, { 'S', 'O', 'F', 'T' }, { 'S', 'O', 'I', 'L' }, { 'S', 'O', 'L', 'D' }, { 'S', 'O', 'M', 'E' }, { 'S', 'O', 'N', 'G' }, { 'S', 'O', 'O', 'N' }, { 'S', 'O', 'O', 'T' }, { 'S', 'O', 'R', 'E' }, { 'S', 'O', 'R', 'T' }, { 'S', 'O', 'U', 'L' }, { 'S', 'O', 'U', 'R' }, { 'S', 'O', 'W', 'N' }, { 'S', 'T', 'A', 'B' }, { 'S', 'T', 'A', 'G' }, { 'S', 'T', 'A', 'N' }, { 'S', 'T', 'A', 'R' }, { 'S', 'T', 'A', 'Y' }, { 'S', 'T', 'E', 'M' }, { 'S', 'T', 'E', 'W' }, { 'S', 'T', 'I', 'R' }, { 'S', 'T', 'O', 'W' }, { 'S', 'T', 'U', 'B' }, { 'S', 'T', 'U', 'N' }, { 'S', 'U', 'C', 'H' }, { 'S', 'U', 'D', 'S' }, { 'S', 'U', 'I', 'T' }, { 'S', 'U', 'L', 'K' }, { 'S', 'U', 'M', 'S' }, { 'S', 'U', 'N', 'G' }, { 'S', 'U', 'N', 'K' }, { 'S', 'U', 'R', 'E' }, { 'S', 'U', 'R', 'F' }, { 'S', 'W', 'A', 'B' }, { 'S', 'W', 'A', 'G' }, { 'S', 'W', 'A', 'M' }, { 'S', 'W', 'A', 'N' }, { 'S', 'W', 'A', 'T' }, { 'S', 'W', 'A', 'Y' }, { 'S', 'W', 'I', 'M' }, { 'S', 'W', 'U', 'M' }, { 'T', 'A', 'C', 'K' }, { 'T', 'A', 'C', 'T' }, { 'T', 'A', 'I', 'L' }, { 'T', 'A', 'K', 'E' }, { 'T', 'A', 'L', 'E' }, { 'T', 'A', 'L', 'K' }, { 'T', 'A', 'L', 'L' }, { 'T', 'A', 'N', 'K' }, { 'T', 'A', 'S', 'K' }, { 'T', 'A', 'T', 'E' }, { 'T', 'A', 'U', 'T' }, { 'T', 'E', 'A', 'L' }, { 'T', 'E', 'A', 'M' }, { 'T', 'E', 'A', 'R' }, { 'T', 'E', 'C', 'H' }, { 'T', 'E', 'E', 'M' }, { 'T', 'E', 'E', 'N' }, { 'T', 'E', 'E', 'T' }, { 'T', 'E', 'L', 'L' }, { 'T', 'E', 'N', 'D' }, { 'T', 'E', 'N', 'T' }, { 'T', 'E', 'R', 'M' }, { 'T', 'E', 'R', 'N' }, { 'T', 'E', 'S', 'S' }, { 'T', 'E', 'S', 'T' }, { 'T', 'H', 'A', 'N' }, { 'T', 'H', 'A', 'T' }, { 'T', 'H', 'E', 'E' }, { 'T', 'H', 'E', 'M' }, { 'T', 'H', 'E', 'N' }, { 'T', 'H', 'E', 'Y' }, { 'T', 'H', 'I', 'N' }, { 'T', 'H', 'I', 'S' }, { 'T', 'H', 'U', 'D' }, { 'T', 'H', 'U', 'G' }, { 'T', 'I', 'C', 'K' }, { 'T', 'I', 'D', 'E' }, { 'T', 'I', 'D', 'Y' }, { 'T', 'I', 'E', 'D' }, { 'T', 'I', 'E', 'R' }, { 'T', 'I', 'L', 'E' }, { 'T', 'I', 'L', 'L' }, { 'T', 'I', 'L', 'T' }, { 'T', 'I', 'M', 'E' }, { 'T', 'I', 'N', 'A' }, { 'T', 'I', 'N', 'E' }, { 'T', 'I', 'N', 'T' }, { 'T', 'I', 'N', 'Y' }, { 'T', 'I', 'R', 'E' }, { 'T', 'O', 'A', 'D' }, { 'T', 'O', 'G', 'O' }, { 'T', 'O', 'I', 'L' }, { 'T', 'O', 'L', 'D' }, { 'T', 'O', 'L', 'L' }, { 'T', 'O', 'N', 'E' }, { 'T', 'O', 'N', 'G' }, { 'T', 'O', 'N', 'Y' }, { 'T', 'O', 'O', 'K' }, { 'T', 'O', 'O', 'L' }, { 'T', 'O', 'O', 'T' }, { 'T', 'O', 'R', 'E' }, { 'T', 'O', 'R', 'N' }, { 'T', 'O', 'T', 'E' }, { 'T', 'O', 'U', 'R' }, { 'T', 'O', 'U', 'T' }, { 'T', 'O', 'W', 'N' }, { 'T', 'R', 'A', 'G' }, { 'T', 'R', 'A', 'M' }, { 'T', 'R', 'A', 'Y' }, { 'T', 'R', 'E', 'E' }, { 'T', 'R', 'E', 'K' }, { 'T', 'R', 'I', 'G' }, { 'T', 'R', 'I', 'M' }, { 'T', 'R', 'I', 'O' }, { 'T', 'R', 'O', 'D' }, { 'T', 'R', 'O', 'T' }, { 'T', 'R', 'O', 'Y' }, { 'T', 'R', 'U', 'E' }, { 'T', 'U', 'B', 'A' }, { 'T', 'U', 'B', 'E' }, { 'T', 'U', 'C', 'K' }, { 'T', 'U', 'F', 'T' }, { 'T', 'U', 'N', 'A' }, { 'T', 'U', 'N', 'E' }, { 'T', 'U', 'N', 'G' }, { 'T', 'U', 'R', 'F' }, { 'T', 'U', 'R', 'N' }, { 'T', 'U', 'S', 'K' }, { 'T', 'W', 'I', 'G' }, { 'T', 'W', 'I', 'N' }, { 'T', 'W', 'I', 'T' }, { 'U', 'L', 'A', 'N' }, { 'U', 'N', 'I', 'T' }, { 'U', 'R', 'G', 'E' }, { 'U', 'S', 'E', 'D' }, { 'U', 'S', 'E', 'R' }, { 'U', 'S', 'E', 'S' }, { 'U', 'T', 'A', 'H' }, { 'V', 'A', 'I', 'L' }, { 'V', 'A', 'I', 'N' }, { 'V', 'A', 'L', 'E' }, { 'V', 'A', 'R', 'Y' }, { 'V', 'A', 'S', 'E' }, { 'V', 'A', 'S', 'T' }, { 'V', 'E', 'A', 'L' }, { 'V', 'E', 'D', 'A' }, { 'V', 'E', 'I', 'L' }, { 'V', 'E', 'I', 'N' }, { 'V', 'E', 'N', 'D' }, { 'V', 'E', 'N', 'T' }, { 'V', 'E', 'R', 'B' }, { 'V', 'E', 'R', 'Y' }, { 'V', 'E', 'T', 'O' }, { 'V', 'I', 'C', 'E' }, { 'V', 'I', 'E', 'W' }, { 'V', 'I', 'N', 'E' }, { 'V', 'I', 'S', 'E' }, { 'V', 'O', 'I', 'D' }, { 'V', 'O', 'L', 'T' }, { 'V', 'O', 'T', 'E' }, { 'W', 'A', 'C', 'K' }, { 'W', 'A', 'D', 'E' }, { 'W', 'A', 'G', 'E' }, { 'W', 'A', 'I', 'L' }, { 'W', 'A', 'I', 'T' }, { 'W', 'A', 'K', 'E' }, { 'W', 'A', 'L', 'E' }, { 'W', 'A', 'L', 'K' }, { 'W', 'A', 'L', 'L' }, { 'W', 'A', 'L', 'T' }, { 'W', 'A', 'N', 'D' }, { 'W', 'A', 'N', 'E' }, { 'W', 'A', 'N', 'G' }, { 'W', 'A', 'N', 'T' }, { 'W', 'A', 'R', 'D' }, { 'W', 'A', 'R', 'M' }, { 'W', 'A', 'R', 'N' }, { 'W', 'A', 'R', 'T' }, { 'W', 'A', 'S', 'H' }, { 'W', 'A', 'S', 'T' }, { 'W', 'A', 'T', 'S' }, { 'W', 'A', 'T', 'T' }, { 'W', 'A', 'V', 'E' }, { 'W', 'A', 'V', 'Y' }, { 'W', 'A', 'Y', 'S' }, { 'W', 'E', 'A', 'K' }, { 'W', 'E', 'A', 'L' }, { 'W', 'E', 'A', 'N' }, { 'W', 'E', 'A', 'R' }, { 'W', 'E', 'E', 'D' }, { 'W', 'E', 'E', 'K' }, { 'W', 'E', 'I', 'R' }, { 'W', 'E', 'L', 'D' }, { 'W', 'E', 'L', 'L' }, { 'W', 'E', 'L', 'T' }, { 'W', 'E', 'N', 'T' }, { 'W', 'E', 'R', 'E' }, { 'W', 'E', 'R', 'T' }, { 'W', 'E', 'S', 'T' }, { 'W', 'H', 'A', 'M' }, { 'W', 'H', 'A', 'T' }, { 'W', 'H', 'E', 'E' }, { 'W', 'H', 'E', 'N' }, { 'W', 'H', 'E', 'T' }, { 'W', 'H', 'O', 'A' }, { 'W', 'H', 'O', 'M' }, { 'W', 'I', 'C', 'K' }, { 'W', 'I', 'F', 'E' }, { 'W', 'I', 'L', 'D' }, { 'W', 'I', 'L', 'L' }, { 'W', 'I', 'N', 'D' }, { 'W', 'I', 'N', 'E' }, { 'W', 'I', 'N', 'G' }, { 'W', 'I', 'N', 'K' }, { 'W', 'I', 'N', 'O' }, { 'W', 'I', 'R', 'E' }, { 'W', 'I', 'S', 'E' }, { 'W', 'I', 'S', 'H' }, { 'W', 'I', 'T', 'H' }, { 'W', 'O', 'L', 'F' }, { 'W', 'O', 'N', 'T' }, { 'W', 'O', 'O', 'D' }, { 'W', 'O', 'O', 'L' }, { 'W', 'O', 'R', 'D' }, { 'W', 'O', 'R', 'E' }, { 'W', 'O', 'R', 'K' }, { 'W', 'O', 'R', 'M' }, { 'W', 'O', 'R', 'N' }, { 'W', 'O', 'V', 'E' }, { 'W', 'R', 'I', 'T' }, { 'W', 'Y', 'N', 'N' }, { 'Y', 'A', 'L', 'E' }, { 'Y', 'A', 'N', 'G' }, { 'Y', 'A', 'N', 'K' }, { 'Y', 'A', 'R', 'D' }, { 'Y', 'A', 'R', 'N' }, { 'Y', 'A', 'W', 'L' }, { 'Y', 'A', 'W', 'N' }, { 'Y', 'E', 'A', 'H' }, { 'Y', 'E', 'A', 'R' }, { 'Y', 'E', 'L', 'L' }, { 'Y', 'O', 'G', 'A' }, { 'Y', 'O', 'K', 'E' } }; /* Extract LENGTH bits from the char array S starting with bit number START. It always reads three consecutive octects, which means it can read past end of data when START is at the edge of the region. */ static uint32_t extract (const unsigned char *s, int start, int length) { unsigned char cl = s[start / 8]; unsigned char cc = s[start / 8 + 1]; unsigned char cr = s[start / 8 + 2]; uint32_t x; x = (uint32_t)(cl << 8 | cc) << 8 | cr; x >>= 24 - (length + (start % 8)); x &= (0xffff >> (16 - length)); return x; } /* Length of a string known to be at least 1 and at most 4 chars long. */ #define STRLEN_1_4(s) (!(s)[1] ? 1 : !(s)[2] ? 2 : !(s)[3] ? 3 : 4) /* Encode 8 bytes in C as a string of English words and store them to STORE. Returns STORE. */ static char * btoe (char *store, const unsigned char *c) { unsigned char cp[10]; /* add in room for the parity 2 bits + extract() slop. */ int p, i; char *store_beg = store; *store = '\0'; /* Workaround for extract() reads beyond end of data */ xzero (cp); memcpy (cp, c, 8); /* Compute parity and append it to CP. */ for (p = 0, i = 0; i < 64; i += 2) p += extract (cp, i, 2); cp[8] = (char)p << 6; /* The 64 bits of input and the two parity bits comprise 66 bits of data that are now in CP. We convert that information, 11 bits at a time, to English words indexed from Wp. Since there are 2048 (2^11) words in Wp, every 11-bit combination corresponds to a distinct word. */ memcpy (store, &Wp[extract (cp, 0, 11)][0], 4); store += STRLEN_1_4 (store); *store++ = ' '; memcpy (store, &Wp[extract (cp, 11, 11)][0], 4); store += STRLEN_1_4 (store); *store++ = ' '; memcpy (store, &Wp[extract (cp, 22, 11)][0], 4); store += STRLEN_1_4 (store); *store++ = ' '; memcpy (store, &Wp[extract (cp, 33, 11)][0], 4); store += STRLEN_1_4 (store); *store++ = ' '; memcpy (store, &Wp[extract (cp, 44, 11)][0], 4); store += STRLEN_1_4 (store); *store++ = ' '; memcpy (store, &Wp[extract (cp, 55, 11)][0], 4); store[4] = '\0'; /* make sure the string is terminated */ DEBUGP (("wrote %s to STORE\n", quote (store_beg))); return store_beg; } /* Calculate the SKEY response, based on the sequence, seed (challenge), and the secret password. The calculated response is used instead of the real password when logging in to SKEY-enabled servers. The result is calculated like this: + Concatenate SEED and PASS and calculate the 16-byte MD5 checksum. + Shorten the checksum to eight bytes by folding the second eight bytes onto the first eight using XOR. The resulting eight-byte sequence is the key. + MD5-process the key, fold the checksum to eight bytes and store it back to the key. Repeat this crunching SEQUENCE times. (Sequence is a number that gets decremented every time the user logs in to the server. Therefore an eavesdropper would have to invert the hash function in order to guess the next one-time password.) + Convert the resulting 64-bit key to 6 English words separated by spaces (see btoe for details) and return the resulting ASCII string. All this is described in section 6 of rfc2289 in more detail. */ const char * skey_response (int sequence, const char *seed, const char *pass) { unsigned char key[8]; /* Room to hold 6 four-letter words (heh), 5 space separators, and the terminating \0. 24+5+1 == 30 */ static char english[30]; struct md5_ctx ctx; uint32_t checksum[4]; md5_init_ctx (&ctx); md5_process_bytes ((const unsigned char *) seed, strlen (seed), &ctx); md5_process_bytes ((const unsigned char *) pass, strlen (pass), &ctx); md5_finish_ctx (&ctx, (unsigned char *) checksum); checksum[0] ^= checksum[2]; checksum[1] ^= checksum[3]; memcpy (key, checksum, 8); while (sequence-- > 0) { md5_init_ctx (&ctx); md5_process_bytes ((unsigned char *) key, 8, &ctx); md5_finish_ctx (&ctx, (unsigned char *) checksum); checksum[0] ^= checksum[2]; checksum[1] ^= checksum[3]; memcpy (key, checksum, 8); } return btoe (english, key); } wget-1.21.2/src/ptimer.h0000644000000000000000000000324514115732710011677 00000000000000/* Declarations for ptimer.c. Copyright (C) 2005-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #ifndef PTIMER_H #define PTIMER_H struct ptimer; /* forward declaration; all struct members are private */ struct ptimer *ptimer_new (void); void ptimer_destroy (struct ptimer *); void ptimer_reset (struct ptimer *); double ptimer_measure (struct ptimer *); double ptimer_read (const struct ptimer *); double ptimer_resolution (void); #endif /* PTIMER_H */ wget-1.21.2/src/css.l0000644000000000000000000001245514057006176011203 00000000000000%option case-insensitive %option noyywrap %option never-interactive %option nounput %top{ /* config.h must precede flex's inclusion of in order for its _GNU_SOURCE definition to take effect. */ #include } %{ /* Lex source for CSS tokenizing. Taken from http://www.w3.org/TR/CSS21/grammar.html#q2 Copyright (C) 2006, 2009-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #define YY_NO_INPUT #include "css-tokens.h" #if defined __clang__ || __GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 5) #pragma GCC diagnostic ignored "-Wunknown-pragmas" // clang mourns about the next one #pragma GCC diagnostic ignored "-Wunused-function" #pragma GCC diagnostic ignored "-Wunused-macros" #pragma GCC diagnostic ignored "-Wunused-parameter" #pragma GCC diagnostic ignored "-Wsign-compare" #pragma GCC diagnostic ignored "-Wswitch-default" #pragma GCC diagnostic ignored "-Wunreachable-code" // clang #pragma clang diagnostic ignored "-Wshorten-64-to-32" #ifndef __clang__ #pragma GCC diagnostic ignored "-Wsuggest-attribute=pure" #endif #endif %} h [0-9a-f] nonascii [\240-\377] unicode \\{h}{1,6}(\r\n|[ \t\r\n\f])? escape {unicode}|\\[^\r\n\f0-9a-f] nmstart [_a-z]|{nonascii}|{escape} nmchar [_a-z0-9-]|{nonascii}|{escape} string1 \"([^\n\r\f\\"]|\\{nl}|{escape})*\" string2 \'([^\n\r\f\\']|\\{nl}|{escape})*\' badstring1 \"([^\n\r\f\\"]|\\{nl}|{escape})*\\? badstring2 \'([^\n\r\f\\']|\\{nl}|{escape})*\\? badcomment1 \/\*[^*]*\*+([^/*][^*]*\*+)* badcomment2 \/\*[^*]*(\*+[^/*][^*]*)* baduri1 url\({w}([!#$%&*-\[\]-~]|{nonascii}|{escape})*{w} baduri2 url\({w}{string}{w} baduri3 url\({w}{badstring} comment \/\*[^*]*\*+([^/*][^*]*\*+)*\/ ident -?{nmstart}{nmchar}* name {nmchar}+ num [0-9]+|[0-9]*"."[0-9]+ string {string1}|{string2} badstring {badstring1}|{badstring2} badcomment {badcomment1}|{badcomment2} baduri {baduri1}|{baduri2}|{baduri3} url ([!#$%&*-~]|{nonascii}|{escape})* s [ \t\r\n\f]+ w {s}? nl \n|\r\n|\r|\f A a|\\0{0,4}(41|61)(\r\n|[ \t\r\n\f])? C c|\\0{0,4}(43|63)(\r\n|[ \t\r\n\f])? D d|\\0{0,4}(44|64)(\r\n|[ \t\r\n\f])? E e|\\0{0,4}(45|65)(\r\n|[ \t\r\n\f])? G g|\\0{0,4}(47|67)(\r\n|[ \t\r\n\f])?|\\g H h|\\0{0,4}(48|68)(\r\n|[ \t\r\n\f])?|\\h I i|\\0{0,4}(49|69)(\r\n|[ \t\r\n\f])?|\\i K k|\\0{0,4}(4b|6b)(\r\n|[ \t\r\n\f])?|\\k L l|\\0{0,4}(4c|6c)(\r\n|[ \t\r\n\f])?|\\l M m|\\0{0,4}(4d|6d)(\r\n|[ \t\r\n\f])?|\\m N n|\\0{0,4}(4e|6e)(\r\n|[ \t\r\n\f])?|\\n O o|\\0{0,4}(4f|6f)(\r\n|[ \t\r\n\f])?|\\o P p|\\0{0,4}(50|70)(\r\n|[ \t\r\n\f])?|\\p R r|\\0{0,4}(52|72)(\r\n|[ \t\r\n\f])?|\\r S s|\\0{0,4}(53|73)(\r\n|[ \t\r\n\f])?|\\s T t|\\0{0,4}(54|74)(\r\n|[ \t\r\n\f])?|\\t U u|\\0{0,4}(55|75)(\r\n|[ \t\r\n\f])?|\\u X x|\\0{0,4}(58|78)(\r\n|[ \t\r\n\f])?|\\x Z z|\\0{0,4}(5a|7a)(\r\n|[ \t\r\n\f])?|\\z %% {s} {return S;} {comment} {return COMMENT;} #\/\*[^*]*\*+([^/*][^*]*\*+)*\/ /* ignore comments */ {badcomment} /* unclosed comment at EOF */ "" {return CDC;} "~=" {return INCLUDES;} "|=" {return DASHMATCH;} {string} {return STRING;} {badstring} {return BAD_STRING;} {ident} {return IDENT;} "#"{name} {return HASH;} @{I}{M}{P}{O}{R}{T} {return IMPORT_SYM;} @{P}{A}{G}{E} {return PAGE_SYM;} @{M}{E}{D}{I}{A} {return MEDIA_SYM;} "@charset " {return CHARSET_SYM;} "!"({w}|{comment})*{I}{M}{P}{O}{R}{T}{A}{N}{T} {return IMPORTANT_SYM;} {num}{E}{M} {return EMS;} {num}{E}{X} {return EXS;} {num}{P}{X} {return LENGTH;} {num}{C}{M} {return LENGTH;} {num}{M}{M} {return LENGTH;} {num}{I}{N} {return LENGTH;} {num}{P}{T} {return LENGTH;} {num}{P}{C} {return LENGTH;} {num}{D}{E}{G} {return ANGLE;} {num}{R}{A}{D} {return ANGLE;} {num}{G}{R}{A}{D} {return ANGLE;} {num}{M}{S} {return TIME;} {num}{S} {return TIME;} {num}{H}{Z} {return FREQ;} {num}{K}{H}{Z} {return FREQ;} {num}{ident} {return DIMENSION;} {num}% {return PERCENTAGE;} {num} {return NUMBER;} "url("{w}{string}{w}")" {return URI;} "url("{w}{url}{w}")" {return URI;} {baduri} {return BAD_URI;} {ident}"(" {return FUNCTION;} . {return *yytext;} %% wget-1.21.2/src/html-parse.h0000644000000000000000000000522714115732710012455 00000000000000/* Declarations for html-parse.c. Copyright (C) 1998-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ #ifndef HTML_PARSE_H #define HTML_PARSE_H struct attr_pair { char *name; /* attribute name */ char *value; /* attribute value */ /* Needed for URL conversion; the places where the value begins and ends, including the quotes and everything. */ const char *value_raw_beginning; int value_raw_size; /* Used internally by map_html_tags. */ int name_pool_index, value_pool_index; }; struct taginfo { char *name; /* tag name */ int end_tag_p; /* whether this is an end-tag */ int nattrs; /* number of attributes */ struct attr_pair *attrs; /* attributes */ const char *start_position; /* start position of tag */ const char *end_position; /* end position of tag */ const char *contents_begin; /* delimiters of tag contents */ const char *contents_end; /* only valid if end_tag_p */ }; struct hash_table; /* forward declaration */ /* Flags for map_html_tags: */ #define MHT_STRICT_COMMENTS 1 /* use strict comment interpretation */ #define MHT_TRIM_VALUES 2 /* trim attribute values, e.g. interpret as "foo" */ void map_html_tags (const char *, int, void (*) (struct taginfo *, void *), void *, int, const struct hash_table *, const struct hash_table *); #endif /* HTML_PARSE_H */ wget-1.21.2/src/html-parse.c0000644000000000000000000011416314115732710012450 00000000000000/* HTML parser for Wget. Copyright (C) 1998-2011, 2015, 2018-2021 Free Software Foundation, Inc. This file is part of GNU Wget. GNU Wget is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. GNU Wget is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Wget. If not, see . Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work. */ /* The only entry point to this module is map_html_tags(), which see. */ /* TODO: - Allow hooks for callers to process contents outside tags. This is needed to implement handling