pax_global_header00006660000000000000000000000064137067555130014526gustar00rootroot0000000000000052 comment=bc62254de93d63d9133baeb46b388141bbf4d3e7 webpy-0.61/000077500000000000000000000000001370675551300126025ustar00rootroot00000000000000webpy-0.61/.coveragerc000066400000000000000000000002271370675551300147240ustar00rootroot00000000000000[run] branch = 1 # NOTE: cannot use package easily, without chdir (https://github.com/nedbat/coveragepy/issues/268). source = web/,tests/ parallel = 1 webpy-0.61/.github/000077500000000000000000000000001370675551300141425ustar00rootroot00000000000000webpy-0.61/.github/workflows/000077500000000000000000000000001370675551300161775ustar00rootroot00000000000000webpy-0.61/.github/workflows/lint_python.yml000066400000000000000000000053611370675551300212760ustar00rootroot00000000000000name: lint_python on: pull_request: push: branches: [master] jobs: lint_python: runs-on: ubuntu-latest services: mariadb: image: mysql:5.7 ports: - 3306:3306 env: MYSQL_ROOT_PASSWORD: root options: --health-cmd "mysqladmin ping" --health-interval 10s --health-timeout 5s --health-retries 5 postgres: image: postgres:latest env: POSTGRES_USER: postgres POSTGRES_PASSWORD: postgres POSTGRES_DB: postgres ports: - 5432:5432 options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5 strategy: matrix: # TODO: (cclauss) add pypy3 https://github.com/webpy/webpy/issues/598 python-version: [3.5, 3.6, 3.7, 3.8] # , pypy3] steps: - uses: actions/checkout@v2 - uses: actions/setup-python@master with: python-version: ${{ matrix.python-version }} - run: pip install codespell flake8 isort pytest - if: matrix.python-version >= 3.6 run: | pip install black black --check . - run: codespell . --ignore-words-list=eith,gae --skip=./.* --quiet-level=2 - run: flake8 --count --ignore=E203,E265,E722,E731,W503 --max-line-length=477 --show-source --statistics - run: isort . - name: "Install dependent Python modules for testing." run: pip install -r requirements.txt -r test_requirements.txt # Use the default MySQL server offered by Github Actions. - name: "Generate /tmp/my.cnf" run: echo -e "[client]\nhost=127.0.0.1\nport=3306\nuser=root\npassword='root'" > /tmp/my.cnf - name: "Create MySQL user and database used for testing." run: mysql --defaults-file=/tmp/my.cnf -e "create user 'scott'@'%' identified by 'tiger'; create database webpy; grant all privileges on webpy.* to 'scott'@'%' with grant option;" - name: "Create PostgreSQL user and database." run: | createdb -h localhost -U postgres webpy createuser -h localhost -U postgres -d scott psql -h localhost -U postgres -d postgres -c "ALTER USER scott WITH ENCRYPTED PASSWORD 'tiger'" psql -h localhost -U postgres -d postgres -c "ALTER DATABASE webpy OWNER TO scott" env: PGPASSWORD: postgres # Run pytest and get detailed output for easy debugging if test failed. # Env variables `WEBPY_DB_` are required for sql db connections. - run: pytest --capture=no --exitfirst --verbose . env: WEBPY_DB_HOST: 127.0.0.1 WEBPY_DB_MYSQL_PORT: 3306 WEBPY_DB_PG_PORT: 5432 WEBPY_DB_NAME: webpy WEBPY_DB_USER: scott WEBPY_DB_PASSWORD: tiger webpy-0.61/.gitignore000066400000000000000000000000771370675551300145760ustar00rootroot00000000000000*.pyc .DS_Store build/ dist/ docs/_build/ *.egg-info .coverage webpy-0.61/.pre-commit-config.yaml000066400000000000000000000005671370675551300170730ustar00rootroot00000000000000default_language_version: python: python3.7 repos: - repo: https://github.com/python/black rev: stable hooks: - id: black - repo: https://gitlab.com/pycqa/flake8 rev: '3.7.9' # pick a git hash / tag to point to hooks: - id: flake8 args: [--count, "--ignore=E203,E265,E722,E731,W503", --max-line-length=477, --show-source, --statistics] webpy-0.61/.travis.yml000066400000000000000000000022451370675551300147160ustar00rootroot00000000000000language: python python: - 3.5 - 3.6 - 3.7 - 3.8 services: - mysql - postgresql install: - pip install codespell flake8 pytest-cov -r requirements.txt -r test_requirements.txt before_script: - if [ "$TRAVIS_PYTHON_VERSION" = "3.8" ]; then pip install black ; black --check . ; fi - codespell . --ignore-words-list=eith,gae --skip=./.* --quiet-level=2 - flake8 --count --ignore=E203,E265,E722,E731,W503 --max-line-length=477 --show-source --statistics # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide - flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics - psql -c 'create database webpy;' -U postgres - mysql --user=root -e "create user 'scott'@'localhost' identified by 'tiger'; create database webpy; grant all privileges on webpy.* to 'scott'@'localhost' with grant option;" script: - pytest --cov --cov-report=term-missing --cov-report=xml after_success: - | flags=py${TRAVIS_PYTHON_VERSION//./} bash <(curl -s https://codecov.io/bash) -Z -X gcov -X search -X xcode -X fix -X coveragepy -f coverage.xml -F "$flags" notifications: irc: irc.freenode.org#webpy webpy-0.61/ChangeLog.txt000066400000000000000000000542431370675551300152020ustar00rootroot00000000000000# web.py changelog ## 2020-06-23 0.61 * setup.py: Add python_requires='>=3.5' #662 ## 2020-06-23 0.60 * Python-2 support has been completely dropped. Welcome to Python 3. * Fixed: session store `DiskStore` doesn't return correctly if session directory doesn't exist. #652 * Fixed: incorrect str/bytes type of session data. #644 #645 * Fixed: `db.query("insert... returning")` fails to commit. #648 #649 ## 2020-03-20 0.50 * New session store `MemoryStore`, used to save a session in memory. Should be useful where there are limited fs writes to the disk, like flash memories. #174 * Fixed: not support `samesite=none`. #592 * Fixed Python-3 compatibility issues: #574, #576. * Support tuple and set in `sqlquote()`. * Drop support for SQL driver `pgdb`. It was dead, you cannot even find its website or download link. * Drop support for SQL driver `psycopg`. The latest version was released in 2006 (14 years ago), please use `psycopg2` instead. * Removed function `web.safemarkdown`. if it's used in your application, you can install the `Markdown` module from pypi (https://pypi.org/project/Markdown/), then replace `web.safemarkdown()` by `markdown.markdown()`. ## 2019-09-25 0.40 Note: `0.40` is the last release which supports Python 2. Future releases will drop support for Python 2. Broken backward compatibilities: - `web.utils.utf8` and `web.utf8` (it's an alias of `web.utils.utf8`) were removed. Please replace them by `web.safestr` instead. - `db.select()` doesn't support specifying offset in `limit` like this: `db.select(..., limit="2, 10", ...)` (equals to raw SQL statement `SELECT ... LIMIT 2, 10`). Please replace them by moving the offset to `offset` keyword like this: `db.select(..., offset=2, limit=10)`. Major changes since 0.39: * Fixed lots of Python-3 compatibility issues. * Drop support for Python < 2.7. * Allow to get form data from http PATCH request (fixes #259, tx @kufd) * Only store new session data if the data is non-default (fixes #161, tx @shish) * Supports `SameSite` cookie attribute (fixes #61 #99 #337) * Cookie expire time is now set to same as session `timeout` (fixes #409 #410) * Supports url for SQLite database like `sqlite:///mydb.sqlite`, `sqlite:////absolute/path/mydb.sqlite` (fixes #209, tx @iamFIREcracker) * Allow HTML5 form input elements in `web.form.Input()` (fixes #440, tx @jimgregory) * Add more form classes for different types: `Email`, `Url`, `Number`, `Range`, `Color`, `Search`, `Telephone` and `Datalist` (fixes #98 #497, tx @faruken @gjdv) * Return body for `NoMethod` error handler (fixes #240, tx @waldhol) * Directory `experimental/` has been removed, it's not used and out of date. * Module `web/webopenid.py` has been removed, it uses old `python-openid` module which was released 9 years ago. If you need openid support, consider `python-openid2` or other packages available on https://pypi.org/. * Fixed unicode in request url (fixes #461, tx @schneidersoft) * Fixed inline comment in Templator which leads to unexpected behavior (fixes #432, tx @lucylqe) * Fixed missing exception (ValueError) for socket.inet_pton to be compatible with twisted patched `socket.inet_pton` (fixes #464, tx @tclh123) * Fixed incorrect order of arguments for sending email with boto (fixes #204, tx @asldevi) * Fixed notfound message is not utf-8 charset (fixes #500, tx @by-z) * Fixed error in creating pooled PostgresDB with pgdb driver (fixes #255, tx @PriceChild) * Fixed IP address which contains space should not pass validation (fixes #140, tx @chuangbo) * Fixed incorrect returned row ids with `multiple_insert()` (fixes #263 #447) * Fixed not correctly render the `id` attribute after changed (fixes #339, tx @jimgregory) * Fixed DiskStore concurrency issue (fixes Fixes #83 #182 #191 #289 #470, tx @skawouter) * Fixed app module isn't picked up by `Reloader` for first code change (fixes #438, tx @jzellman) ## 2018-02-28 0.39 * Fixed a security issue with the form module (tx Orange Tsai) * Fixed a security issue with the db module (tx Adrián Brav and Orange Tsai) ## 2016-07-08 0.38 * Fixed failing tests in test/session.py when postgres is not installed. (tx Michael Diamond) * Fixed an error with Python 2.3 (tx Michael Diamond) * web.database now accepts a URL, $DATABASE_URL (fixes #171) (tx Aaron Swartz, we miss you) * support port use 'port' as keyword for postgres database with used eith pgdb (tx Sandesh Singh) * Fixes to FirebirdDB database (tx Ben Hanna) * Added a gaerun method to start application for google app engine (tx Matt Habel) * Better error message from `db.multiple_insert` when not all rows have the same keys (tx Ben Hoyt) * Allow custom messages for most errors (tx Shaun Sharples) * IPv6 support (tx Matthew of Boswell and zamabe) * Fixed sending email using Amazon SES (tx asldevi) * Fixed handling of long numbers in sqlify. closes #213. (tx cjrolo) * Escape HTML characters when emitting API docs. (tx Jeff Zellman) * Fixed an inconsistency in form.Dropdown when numbers are used for args and value. (tx Noprianto) * Fixed a potential remote execution risk in `reparam` (tx Adrián Brav) * The where clause in db queries can be a dict now * Added `first` method to iterbetter * Fix to unexpected session when used with MySQL (tx suhashpatil) * Change dburl2dict to use urlparse and to support the simple case of just a database name. (tx Jeff Zellman) * Support '204 No Content' status code (tx Matteo Landi) * Support `451 Unavailable For Legal Reasons` status code(tx Yannik Robin Kettenbach) * Updates to documentation (tx goodrone, asldevi) ## 2012-06-26 0.37 * Fixed datestr issue on Windows -- #155 * Fixed Python 2.4 compatibility issues (tx fredludlow) * Fixed error in utils.safewrite (tx shuge) -- #95 * Allow use of web.data() with app.request() -- #105 * Fixed an issue with session initializaton (tx beardedprojamz) -- #109 * Allow custom message on 400 Bad Request (tx patryk) -- #121 * Made djangoerror work on GAE. -- #80 * Handle malformatted data in the urls. -- #117 * Made it easier to stop the dev server -- #100, #122 * Added support for customizing cookie_path in session (tx larsga) -- #89 * Added exception for "415 Unsupported Media" (tx JirkaChadima) -- #145 * Added GroupedDropdown to support `` tag (tx jzellman) -- #152 * Fixed failure in embedded interpreter - #87 * Optimized web.cookies (tx benhoyt) - #148 ## 2011-07-04 0.36 * Upgraded to CherryPy WSGIServer 3.2.0. -- #66 * Various Jython compatibility fixes (tx Ben Noordhuis) * allow strips to accept lists -- #69 * Improvements to setcookie (tx lovelylain) -- #65 * Added __contains__ method to Session. (tx lovelylain) #65 * Added secure option to session. -- #38 * Fixed db.delete error with `using` clause (tx berndtj) -- #28 * Fixed the case of no where-clauses in db.where * Fixed threadlocal error in python2.3 -- #77 * Fixed TemplateResult inconsistent behavior -- #78 * Fixed query execution issues with MSSQL -- #71 ## 2011-05-15 0.35 * Better ThreaedDict implementation using threadlocal (tx Ben Hoyt) * Make Form a new-style class -- #53 * Optimized SQLQuery.join and generation of multiple_insert query -- #58 * New: support for Amazon's Simple Email Service * Added httponly keyword to setcookie (tx Justin Davis) * Added httponly only option to sessions and enabled it by default (tx Justin Davis) * made htmlquote and htmlunquote work with unicode * Added doseq support for web.url * New flag web.config.debug_sql to control printing of db queries (tx Nimrod S. Kerrett) * Fixed inserting default values into MySQL -- #49 * Fixed rendering of Dropdown with multiple values (tx krowbar) -- #43 * Fixed multiple set-cookie header issue with session -- #45 * Fixed error in safeunicode when used with appengine datastore objects * Fixed unicode error in generating debugerror -- #26 * Fixed GAE compilation issue -- #24 * Fixed unicode encoding issue in templates -- #17 * Fixed a bug in form.RadioButton when called with tuple options (tx fhsm) -- #13 * Fixed error in creating PostgresDB with pgdb driver (tx cninucci) -- #23 * Support auto conversion of timestamp/date datatypes in sqlite to datetime.data objects -- #22 * Fixed escaping issue on GAE -- #10 * fixed form.validates for checkbox (tx Justin Davis). * fixed duplicate content-type in web.sendmail -- #20 * Fix: create session dirs if required (tx Martin Marcher) * Fixed safestr to make use of encoding argument (tx s7v7nislands) * Don't allow /static/../foo urls in dev webserver (tx Arnar Lundesgaard) * Disabled debug mode in flup server (tx irrelative) -- #35 * And a lot of unicode fixes ## 2010-03-20 0.34 * fix: boolean test works even for sqlite results (tx Emyr Thomas for the idea) * fix issue with loop.xx variables in templetor (Bug#476708) * hide unwanted tracebacks in debugerror * display correct template line numbers in debugerror * new utilities: counter, safeiter, safewrite, requeue, restack (by Aaron Swartz) * various form.py fixes and improvements * automatically escape % characters in the db query (Bug#516516) * fix non-deterministic template order (Bug#490209) * attachment support for web.sendmail (tx gregglind) * template.py optimizations and extension support ## 2009-10-28 0.33 * form.Button takes optional argument `html` * remove obsolete write function in http.py (tx Justin) (Bug#315337) * refactor httpserver.runsimple code * improve form.py for customizability * new: add background updating to memoize * fix: use sendmail from web.config.sendmail_path (tx Daniel Schwartz) * fix: make web.profiler work on Windows (tx asmo) (Bug#325139) * fix changequery to make it work correctly even when the input has multi-valued fields (Bug#118229) * fix: make sure sequence exists before queying for currval(seqname) when executing postgres insert query (Bug#268705) * fix: raise web.notfound() instead of return in autodelegate (tx SeC) * fix: raise NotSupportedError when len or bool is used on sqlite result (Bug#179644) * fix: make db parameter optional for creating postgres DB to allow taking it from environ. (Bug#153491) * fix unicode errors in db module * fix: convert unicode strings to UTF8 before printing SQL queries * fix unicode error in debugerror * fix: don't convert file upload data to unicode even when file={} is not passed to web.input * fix checkbox value/checked confusion (Bug#128233) * fix: consider empty lines as part of the indented block in templetor * fix: fix a bug in web.group ## 2009-06-04 0.32 * optional from_address to web.emailerrors * upgrade wsgiserver to CherryPy/3.1.2 * support for extensions in Jinja2 templates (tx Zhang Huangbin) * support web.datestr for datetime.date objects also * support for lists in db queries * new: uniq and iterview * fix: set debug=False when application is run with mod_wsgi (tx Patrick Swieskowski) [Bug#370904](https://bugs.launchpad.net/webpy/+bug/370904) * fix: make web.commify work with decimals [Bug#317204](https://bugs.launchpad.net/webpy/+bug/317204) * fix: unicode issues with sqlite database [Bug#373219](https://bugs.launchpad.net/webpy/+bug/373219) * fix: urlquote url when the server is lighttpd [Bug#339858](https://bugs.launchpad.net/webpy/+bug/339858) * fix: issue with using date.format in templates * fix: use TOP instead of LIMIT in mssql database [Bug#324049](https://bugs.launchpad.net/webpy/+bug/324049) * fix: make sessions work well with expirations * fix: accept both list and tuple as arg values in form.Dropdown [Bug#314970](https://bugs.launchpad.net/webpy/+bug/314970) * fix: match parenthesis when parsing `for` statement in templates * fix: fix python 2.3 compatibility * fix: ignore dot folders when compiling templates (tx Stuart Langridge) * fix: don't consume KeyboardInterrupt and SystemExit errors * fix: make application work well with iterators ## 2008-12-10: 0.31 * new: browser module * new: test utilities * new: ShelfStore * fix: web.cookies error when default is None * fix: paramstyle for OracleDB (tx kromakey) * fix: performance issue in SQLQuery.join * fix: use wsgi.url_scheme to find ctx.protocol ## 2008-12-06: 0.3 * new: replace print with return (backward-incompatible) * new: application framework (backward-incompatible) * new: modular database system (backward-incompatible) * new: templetor reimplementation * new: better unicode support * new: debug mode (web.config.debug) * new: better db pooling * new: sessions * new: support for GAE * new: etag support * new: web.openid module * new: web.nthstr * fix: various form.py fixes * fix: python 2.6 compatibility * fix: file uploads are not loaded into memory * fix: SQLLiteral issue (Bug#180027) * change: web.background is moved to experimental (backward-incompatible) * improved API doc generation (tx Colin Rothwell) ## 2008-01-19: 0.23 * fix: for web.background gotcha ([133079](http://bugs.launchpad.net/webpy/+bug/133079)) * fix: for postgres unicode bug ([177265](http://bugs.launchpad.net/webpy/+bug/177265)) * fix: web.profile behavior in python 2.5 ([133080](http://bugs.launchpad.net/webpy/+bug/133080)) * fix: only uppercase HTTP methods are allowed. ([176415](http://bugs.launchpad.net/webpy/+bug/176415)) * fix: transaction error in with statement ([125118](http://bugs.launchpad.net/webpy/+bug/125118)) * fix: fix in web.reparam ([162085](http://bugs.launchpad.net/webpy/+bug/162085)) * fix: various unicode issues ([137042](http://bugs.launchpad.net/webpy/+bug/137042), [180510](http://bugs.launchpad.net/webpy/+bug/180510), [180549](http://bugs.launchpad.net/webpy/+bug/180549), [180653](http://bugs.launchpad.net/webpy/+bug/180653)) * new: support for https * new: support for secure cookies * new: sendmail * new: htmlunquote ## 2007-08-23: 0.22 * compatibility with new DBUtils API ([122112](https://bugs.launchpad.net/webpy/+bug/122112)) * fix reloading ([118683](https://bugs.launchpad.net/webpy/+bug/118683)) * fix compatibility between `changequery` and `redirect` ([118234](https://bugs.launchpad.net/webpy/+bug/118234)) * fix relative URI in `web.redirect` ([118236](https://bugs.launchpad.net/webpy/+bug/118236)) * fix `ctx._write` support in built-in HTTP server ([121908](https://bugs.launchpad.net/webpy/+bug/121908)) * fix `numify` strips things after '.'s ([118644](https://bugs.launchpad.net/webpy/+bug/118644)) * fix various unicode issues ([114703](https://bugs.launchpad.net/webpy/+bug/114703), [120644](https://bugs.launchpad.net/webpy/+bug/120644), [124280](https://bugs.launchpad.net/webpy/+bug/124280)) ## 2007-05-28: 0.21 * security fix: prevent bad characters in headers * support for cheetah template reloading * support for form validation * new `form.File` * new `web.url` * fix rendering issues with hidden and button inputs * fix 2.3 incompatibility with `numify` * fix multiple headers with same name * fix web.redirect issues when homepath is not / * new CherryPy wsgi server * new nested transactions * new sqlliteral ## 2006-05-09: 0.138 * New function: `intget` * New function: `datestr` * New function: `validaddr` * New function: `sqlwhere` * New function: `background`, `backgrounder` * New function: `changequery` * New function: `flush` * New function: `load`, `unload` * New variable: `loadhooks`, `unloadhooks` * Better docs; generating [docs](documentation) from web.py now * global variable `REAL_SCRIPT_NAME` can now be used to work around lighttpd madness * fastcgi/scgi servers now can listen on sockets * `output` now encodes Unicode * `input` now takes optional `_method` argument * Potentially-incompatible change: `input` now returns `badrequest` automatically when `requireds` aren't found * `storify` now takes lists and dictionaries as requests (see docs) * `redirect` now blanks any existing output * Quote SQL better when `db_printing` is on * Fix delay in `nomethod` * Fix `urlquote` to encode better. * Fix 2.3 incompatibility with `iters` (tx ??) * Fix duplicate headers * Improve `storify` docs * Fix `IterBetter` to raise IndexError, not KeyError ## 2006-03-27: 0.137 * Add function `dictfindall` (tx Steve Huffman) * Add support to `autodelegate` for arguments * Add functions `httpdate` and `parsehttpdate` * Add function `modified` * Add support for FastCGI server mode * Clarify `dictadd` documentation (tx Steve Huffman) * Changed license to public domain * Clean up to use `ctx` and `env` instead of `context` and `environ` * Improved support for PUT, DELETE, etc. (tx list) * Fix `ctx.fullpath` (tx Jesir Vargas) * Fix sqlite support (tx Dubhead) * Fix documentation bug in `lstrips` (tx Gregory Petrosyan) * Fix support for IPs and ports (1/2 tx Jesir Vargas) * Fix `ctx.fullpath` (tx Jesir Vargas) * Fix sqlite support (tx Dubhead) * Fix documentation bug in `lstrips` (tx Gregory Petrosyan) * Fix `iters` bug with sets * Fix some breakage introduced by Vargas's patch * Fix `sqlors` bug * Fix various small style things (tx Jesir Vargas) * Fix bug with `input` ignoring GET input ## 2006-02-22: 0.136 (svn) * Major code cleanup (tx to Jesir Vargas for the patch). * 2006-02-15: 0.135 * Really fix that mysql regression (tx Sean Leach). * 2006-02-15: 0.134 * The `StopIteration` exception is now caught. This can be used by functions that do things like check to see if a user is logged in. If the user isn't, they can output a message with a login box and raise StopIteration, preventing the caller from executing. * Fix some documentation bugs. * Fix mysql regression (tx mrstone). ## 2006-02-12: 0.133 * Docstrings! (tx numerous, esp. Jonathan Mark (for the patch) and Guido van Rossum (for the prod)) * Add `set` to web.iters. * Make the `len` returned by `query` an int (tx ??). * Backwards-incompatible change: `base` now called `prefixurl`. * Backwards-incompatible change: `autoassign` now takes `self` and `locals()` as arguments. ## 2006-02-07: 0.132 * New variable `iters` is now a listing of possible list-like types (currently list, tuple, and, if it exists, Set). * New function `dictreverse` turns `{1:2}` into `{2:1}`. * `Storage` now a dictionary subclass. * `tryall` now takes an optional prefix of functions to run. * `sqlors` has various improvements. * Fix a bunch of DB API bugs. * Fix bug with `storify` when it received multiple inputs (tx Ben Woosley). * Fix bug with returning a generator (tx Zbynek Winkler). * Fix bug where len returned a long on query results (tx F.S). ## 2006-01-31: 0.131 (not officially released) * New function `_interpolate` used internally for interpolating strings. * Redone database API. `select`, `insert`, `update`, and `delete` all made consistent. Database queries can now do more complicated expressions like `$foo.bar` and `${a+b}`. You now have to explicitly pass the dictionary to look up variables in. Pass `vars=locals()` to get the old functionality of looking up variables . * New functions `sqllist` and `sqlors` generate certain kinds of SQL. ## 2006-01-30: 0.13 * New functions `found`, `seeother`, and `tempredirect` now let you do other kinds of redirects. `redirect` now also takes an optional status parameter. (tx many) * New functions `expires` and `lastmodified` make it easy to send those headers. * New function `gone` returns a 410 Gone (tx David Terrell). * New function `urlquote` applies url encoding to a string. * New function `iterbetter` wraps an iterator and allows you to do __getitem__s on it. * Have `query` return an `iterbetter` instead of an iterator. * Have `debugerror` show tracebacks with the innermost frame first. * Add `__hash__` function to `threadeddict` (and thus, `ctx`). * Add `context.host` value for the requested host name. * Add option `db_printing` that prints database queries and the time they take. * Add support for database pooling (tx Steve Huffman). * Add support for passing values to functions called by `handle`. If you do `('foo', 'value')` it will add `'value'` as an argument when it calls `foo`. * Add support for scgi (tx David Terrell for the patch). * Add support for web.py functions that are iterators (tx Brendan O'Connor for the patch). * Use new database cursors on each call instead of reusing one. * `setcookie` now takes an optional `domain` argument. * Fix bug in autoassign. * Fix bug where `debugerror` would break on objects it couldn't display. * Fix bug where you couldn't do `#include`s inline. * Fix bug with `reloader` and database calls. * Fix bug with `reloader` and base templates. * Fix bug with CGI mode on certain operating systems. * Fix bug where `debug` would crash if called outside a request. * Fix bug with `context.ip` giving weird values with proxies. ## 2006-01-29: 0.129 * Add Python 2.2 support. ## 2006-01-28: 0.128 * Fix typo in `web.profile`. ## 2006-01-28: 0.127 * Fix bug in error message if invalid dbn is sent (tx Panos Laganakos). ## 2006-01-27: 0.126 * Fix typos in Content-Type headers (tx Beat Bolli for the prod). ## 2006-01-22: 0.125 * Support Cheetah 2.0. ## 2006-01-22: 0.124 * Fix spacing bug (tx Tommi Raivio for the prod). ## 2006-01-16: 0.123 * Fix bug with CGI usage (tx Eddie Sowden for the prod). ## 2006-01-14: 0.122 * Allow DELETEs from `web.query` (tx Joost Molenaar for the prod). ## 2006-01-08: 0.121 * Allow import of submodules like `pkg.mod.cn` (tx Sridhar Ratna). * Fix a bug in `update` (tx Sergey Khenkin). ## 2006-01-05: 0.12 * Backwards-incompatible change: `db_parameters` is now a dictionary. * Backwards-incompatible change: `sumdicts` is now `dictadd`. * Add support for PyGreSQL, MySQL (tx Hallgrimur H. Gunnarsson). * Use HTML for non-Cheetah error message. * New function `htmlquote()`. * New function `tryall()`. * `ctx.output` can now be set to a generator. (tx Brendan O'Connor) ## 2006-01-04: 0.117 * Add support for psycopg 1.x. (tx Gregory Price) ## 2006-01-04: 0.116 * Add support for Python 2.3. (tx Evan Jones) ## 2006-01-04: 0.115 * Fix some bugs where database queries weren't reparameterized. Oops! * Fix a bug where `run()` wasn't getting the right functions. * Remove a debug statement accidentally left in. * Allow `storify` to be used on dictionaries. (tx Joseph Trent) ## 2006-01-04: 0.114 * Make `reloader` work on Windows. (tx manatlan) * Fix some small typos that affected colorization. (tx Gregory Price) ## 2006-01-03: 0.113 * Reorganize `run()` internals so mod_python can be used. (tx Nicholas Matsakis) ## 2006-01-03: 0.112 * Make `reloader` work when `code.py` is called with a full path. (tx David Terrell) ## 2006-01-03: 0.111 * Fixed bug in `strips()`. (tx Michael Josephson) ## 2006-01-03: 0.11 * First public version. webpy-0.61/LICENSE.txt000066400000000000000000000001451370675551300144250ustar00rootroot00000000000000web.py is in the public domain; it can be used for whatever purpose with absolutely no restrictions. webpy-0.61/MANIFEST.in000066400000000000000000000002041370675551300143340ustar00rootroot00000000000000include requirements.txt include test_requirements.txt recursive-include docs * recursive-include tests * recursive-include tools * webpy-0.61/README.md000066400000000000000000000017371370675551300140710ustar00rootroot00000000000000web.py is a web framework for Python that is as simple as it is powerful. Visit http://webpy.org/ for more information. [![build status](https://secure.travis-ci.org/webpy/webpy.png?branch=master)](https://travis-ci.org/webpy/webpy) [![Codecov Test Coverage](https://codecov.io/gh/webpy/webpy/branch/master/graphs/badge.svg?style=flat)](https://codecov.io/gh/webpy/webpy) The latest stable release `0.61` only supports Python >= 3.5. To install it, please run: ``` # For Python 3 python3 -m pip install web.py==0.61 ``` If you are still using Python 2.7, then please use web.py version 0.51 which is intended to be our last release that supports Python 2. ``` # For Python 2.7 python2 -m pip install web.py==0.51 ``` You can also download it from [GitHub Releases](https://github.com/webpy/webpy/releases) page, then install it manually: ``` unzip webpy-0.61.zip cd webpy-0.61/ python3 setup.py install ``` Note: `0.5x` (e.g. 0.50, 0.51) are our last releases which support Python 2. webpy-0.61/docs/000077500000000000000000000000001370675551300135325ustar00rootroot00000000000000webpy-0.61/docs/Makefile000066400000000000000000000151461370675551300152010ustar00rootroot00000000000000# Makefile for Sphinx documentation # # You can set these variables from the command line. SPHINXOPTS = SPHINXBUILD = sphinx-build PAPER = BUILDDIR = _build # User-friendly check for sphinx-build ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1) $(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/) endif # Internal variables. PAPEROPT_a4 = -D latex_paper_size=a4 PAPEROPT_letter = -D latex_paper_size=letter ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . # the i18n builder cannot share the environment and doctrees with the others I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext help: @echo "Please use \`make ' where is one of" @echo " html to make standalone HTML files" @echo " dirhtml to make HTML files named index.html in directories" @echo " singlehtml to make a single large HTML file" @echo " pickle to make pickle files" @echo " json to make JSON files" @echo " htmlhelp to make HTML files and a HTML help project" @echo " qthelp to make HTML files and a qthelp project" @echo " devhelp to make HTML files and a Devhelp project" @echo " epub to make an epub" @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" @echo " latexpdf to make LaTeX files and run them through pdflatex" @echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx" @echo " text to make text files" @echo " man to make manual pages" @echo " texinfo to make Texinfo files" @echo " info to make Texinfo files and run them through makeinfo" @echo " gettext to make PO message catalogs" @echo " changes to make an overview of all changed/added/deprecated items" @echo " xml to make Docutils-native XML files" @echo " pseudoxml to make pseudoxml-XML files for display purposes" @echo " linkcheck to check all external links for integrity" @echo " doctest to run all doctests embedded in the documentation (if enabled)" clean: rm -rf $(BUILDDIR)/* html: $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." dirhtml: $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." singlehtml: $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml @echo @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." pickle: $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle @echo @echo "Build finished; now you can process the pickle files." json: $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json @echo @echo "Build finished; now you can process the JSON files." htmlhelp: $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp @echo @echo "Build finished; now you can run HTML Help Workshop with the" \ ".hhp project file in $(BUILDDIR)/htmlhelp." qthelp: $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp @echo @echo "Build finished; now you can run "qcollectiongenerator" with the" \ ".qhcp project file in $(BUILDDIR)/qthelp, like this:" @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/webpy.qhcp" @echo "To view the help file:" @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/webpy.qhc" devhelp: $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp @echo @echo "Build finished." @echo "To view the help file:" @echo "# mkdir -p $$HOME/.local/share/devhelp/webpy" @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/webpy" @echo "# devhelp" epub: $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub @echo @echo "Build finished. The epub file is in $(BUILDDIR)/epub." latex: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." @echo "Run \`make' in that directory to run these through (pdf)latex" \ "(use \`make latexpdf' here to do that automatically)." latexpdf: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo "Running LaTeX files through pdflatex..." $(MAKE) -C $(BUILDDIR)/latex all-pdf @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." latexpdfja: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo "Running LaTeX files through platex and dvipdfmx..." $(MAKE) -C $(BUILDDIR)/latex all-pdf-ja @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." text: $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text @echo @echo "Build finished. The text files are in $(BUILDDIR)/text." man: $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man @echo @echo "Build finished. The manual pages are in $(BUILDDIR)/man." texinfo: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." @echo "Run \`make' in that directory to run these through makeinfo" \ "(use \`make info' here to do that automatically)." info: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo "Running Texinfo files through makeinfo..." make -C $(BUILDDIR)/texinfo info @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." gettext: $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale @echo @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." changes: $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes @echo @echo "The overview file is in $(BUILDDIR)/changes." linkcheck: $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck @echo @echo "Link check complete; look for any errors in the above output " \ "or in $(BUILDDIR)/linkcheck/output.txt." doctest: $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest @echo "Testing of doctests in the sources finished, look at the " \ "results in $(BUILDDIR)/doctest/output.txt." xml: $(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml @echo @echo "Build finished. The XML files are in $(BUILDDIR)/xml." pseudoxml: $(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml @echo @echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml." webpy-0.61/docs/api.rst000066400000000000000000000011261370675551300150350ustar00rootroot00000000000000web.py API ========== web.application --------------- .. automodule:: web.application :members: web.db ------ .. automodule:: web.db :members: web.net ------- .. automodule:: web.net :members: web.form -------- .. automodule:: web.form :members: web.http -------- .. automodule:: web.http :members: web.session ----------- .. automodule:: web.session :members: web.template ------------ .. automodule:: web.template :members: web.utils --------- .. automodule:: web.utils :members: web.webapi ---------- .. automodule:: web.webapi :members: webpy-0.61/docs/conf.py000066400000000000000000000200431370675551300150300ustar00rootroot00000000000000# -*- coding: utf-8 -*- # # web.py documentation build configuration file, created by # sphinx-quickstart on Sun Oct 27 15:35:05 2013. # # This file is execfile()d with the current directory set to its # containing dir. # # Note that not all possible configuration values are present in this # autogenerated file. # # All configuration values have a default; values that are commented out # serve to show the default. # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. # sys.path.insert(0, os.path.abspath('.')) # -- General configuration ------------------------------------------------ # If your documentation needs a minimal Sphinx version, state it here. # needs_sphinx = '1.0' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. extensions = ["sphinx.ext.autodoc"] # Add any paths that contain templates here, relative to this directory. templates_path = ["_templates"] # The suffix of source filenames. source_suffix = ".rst" # The encoding of source files. # source_encoding = 'utf-8-sig' # The master toctree document. master_doc = "index" # General information about the project. project = u"web.py" copyright = u"" # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # # The short X.Y version. version = "0.39" # The full version, including alpha/beta/rc tags. release = "0.39" # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. # language = None # There are two options for replacing |today|: either, you set today to some # non-false value, then it is used: # today = '' # Else, today_fmt is used as the format for a strftime call. # today_fmt = '%B %d, %Y' # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. exclude_patterns = ["_build"] # The reST default role (used for this markup: `text`) to use for all # documents. # default_role = None # If true, '()' will be appended to :func: etc. cross-reference text. # add_function_parentheses = True # If true, the current module name will be prepended to all description # unit titles (such as .. function::). # add_module_names = True # If true, sectionauthor and moduleauthor directives will be shown in the # output. They are ignored by default. # show_authors = False # The name of the Pygments (syntax highlighting) style to use. pygments_style = "sphinx" # A list of ignored prefixes for module index sorting. # modindex_common_prefix = [] # If true, keep warnings as "system message" paragraphs in the built documents. # keep_warnings = False # -- Options for HTML output ---------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. html_theme = "default" # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. # html_theme_options = {} # Add any paths that contain custom themes here, relative to this directory. # html_theme_path = [] # The name for this set of Sphinx documents. If None, it defaults to # " v documentation". # html_title = None # A shorter title for the navigation bar. Default is the same as html_title. # html_short_title = None # The name of an image file (relative to this directory) to place at the top # of the sidebar. # html_logo = None # The name of an image file (within the static path) to use as favicon of the # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 # pixels large. # html_favicon = None # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". html_static_path = ["_static"] # Add any extra paths that contain custom files (such as robots.txt or # .htaccess) here, relative to this directory. These files are copied # directly to the root of the documentation. # html_extra_path = [] # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, # using the given strftime format. # html_last_updated_fmt = '%b %d, %Y' # If true, SmartyPants will be used to convert quotes and dashes to # typographically correct entities. # html_use_smartypants = True # Custom sidebar templates, maps document names to template names. # html_sidebars = {} # Additional templates that should be rendered to pages, maps page names to # template names. # html_additional_pages = {} # If false, no module index is generated. # html_domain_indices = True # If false, no index is generated. # html_use_index = True # If true, the index is split into individual pages for each letter. # html_split_index = False # If true, links to the reST sources are added to the pages. # html_show_sourcelink = True # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. # html_show_sphinx = True # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. # html_show_copyright = True # If true, an OpenSearch description file will be output, and all pages will # contain a tag referring to it. The value of this option must be the # base URL from which the finished HTML is served. # html_use_opensearch = '' # This is the file name suffix for HTML files (e.g. ".xhtml"). # html_file_suffix = None # Output file base name for HTML help builder. htmlhelp_basename = "webpydoc" # -- Options for LaTeX output --------------------------------------------- latex_elements = { # The paper size ('letterpaper' or 'a4paper'). #'papersize': 'letterpaper', # The font size ('10pt', '11pt' or '12pt'). #'pointsize': '10pt', # Additional stuff for the LaTeX preamble. #'preamble': '', } # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, # author, documentclass [howto, manual, or own class]). latex_documents = [ ("index", "webpy.tex", u"web.py Documentation", u"Anand Chitipothu", "manual") ] # The name of an image file (relative to this directory) to place at the top of # the title page. # latex_logo = None # For "manual" documents, if this is true, then toplevel headings are parts, # not chapters. # latex_use_parts = False # If true, show page references after internal links. # latex_show_pagerefs = False # If true, show URL addresses after external links. # latex_show_urls = False # Documents to append as an appendix to all manuals. # latex_appendices = [] # If false, no module index is generated. # latex_domain_indices = True # -- Options for manual page output --------------------------------------- # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [("index", "webpy", u"web.py Documentation", [u"Anand Chitipothu"], 1)] # If true, show URL addresses after external links. # man_show_urls = False # -- Options for Texinfo output ------------------------------------------- # Grouping the document tree into Texinfo files. List of tuples # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ ( "index", "webpy", u"web.py Documentation", u"Anand Chitipothu", "webpy", "One line description of project.", "Miscellaneous", ) ] # Documents to append as an appendix to all manuals. # texinfo_appendices = [] # If false, no module index is generated. # texinfo_domain_indices = True # How to display URL addresses: 'footnote', 'no', or 'inline'. # texinfo_show_urls = 'footnote' # If true, do not generate a @detailmenu in the "Top" node's menu. # texinfo_no_detailmenu = False webpy-0.61/docs/db.rst000066400000000000000000000121101370675551300146440ustar00rootroot00000000000000Accessing the database ====================== Web.py provides a simple and uniform interface to the database that you want to work with, whether it is PostgreSQL, MySQL, SQLite or any other. It doesn't try to build layers between you and your database. Rather, it tries to make it easy to perform common tasks, and get out of your way when you need to do more advanced things. Create database object ------------------------ The first thing to work with databases from web.py is to create a create a database object with `web.database()`. It returns database object, which has convenient methods for you to use. Make sure that you have appropriate database library installed (`psycopg2` for PostgreSQL, `MySQLdb` for MySQL, `sqlite3` for SQLite). :: db = web.database(dbn='postgres', db='dbname', user='username', pw='password') `dbn` for MySQL is `mysql` and `sqlite` for SQLite. SQLite doesn't take `user` `pw` parameters. To close database connection: :: db.ctx.db.close() Multiple databases `````````````````` Working with more databases is not at all difficult with web.py. Here's what you do. :: db1 = web.database(dbn='postgres', db='dbname1', user='username1', pw='password2') db2 = web.database(dbn='postgres', db='dbname2', user='username2', pw='password2') And use `db1`, `db2` to access those databases respectively. Operations ---------- `web.database()` returns an object which provide you all the functionality to insert, select, update and delete data from your database. For each of the methods on `db` below, you can pass `_test=True` to see the SQL statement rather than executing it. Inserting ````````` :: # Insert an entry into table 'user' userid = db.insert('user', firstname="Bob", lastname="Smith", joindate=web.SQLLiteral("NOW()")) The first argument is the table name and the rest of them are set of named arguments which represent the fields in the table. If values are not given, the database may create default values or issue a warning. For bulk insertion rather than inserting record by record, use `Multiple Inserts` rather. Selecting ````````` The `select` method is used for selecting rows from the database. It returns a `web.iterbetter` object, which can be looped through. To select `all` the rows from the `user` table, you would simply do :: users = db.select('user') For the real world use cases, `select` method takes `vars`, `what`, `where`, `order`, `group`, `limit`, `offset`, and `_test` optional parameters. :: users = db.select('users', where="id>100") To prevent SQL injection attacks, you can use `$key` in where clause and pass the `vars` which has { 'key': value }. :: vars = dict(name="Bob") results = db.select('users', where="name = $name", vars=vars, _test=True) >>> results Updating ```````` The `update` method accepts same kind of arguments as Select. It returns the number of rows updated. :: num_updated = db.update('users', where="id = 10", firstname = "Foo") Deleting ```````` The `delete` method returns the number of rows deleted. It also accepts "using" and "vars" parameters. See ``Selecting`` for more details on `vars`. :: num_deleted = db.delete('users', where="id=10") Multiple Inserts ```````````````` The `multiple_insert` method on the `db` object allows you to do that. All that's needed is to prepare a list of dictionaries, one for each row to be inserted, each with the same set of keys and pass it to `multiple_insert` along with the table name. It returns the list of ids of the inserted rows. The value of `db.supports_multiple_insert` tells you if your database supports multiple inserts. :: values = [{"name": "foo", "email": "foo@example.com"}, {"name": "bar", "email": "bar@example.com"}] db.multiple_insert('person', values=values) Advanced querying ````````````````` Many a times, there is more to do with the database, rather than the simple operations which can be done by `insert`, `select`, `delete` and `update` - Things like your favorite (or scary) joins, counts etc. All these are possible with `query` method, which also takes `vars`. :: results = db.query("SELECT COUNT(*) AS total_users FROM users") print results[0].total_users # prints number of entries in 'users' table Joining tables :: results = db.query("SELECT * FROM entries JOIN users WHERE entries.author_id = users.id") Transactions ```````````` The database object has a method `transaction` which starts a new transaction and returns the transaction object. The transaction object can be used to commit or rollback that transaction. It is also possible to have nested transactions. From Python 2.5 onwards, which support `with` statements, you would do :: with db.transaction(): userid = db.insert('users', name='foo') authorid = db.insert('authors', userid=userid) For earlier versions of Python, you can do :: t = db.transaction() try: userid = db.insert('users', name='foo') authorid = db.insert('authors', userid=userid) except: t.rollback() raise else: t.commit() webpy-0.61/docs/deploying.rst000066400000000000000000000060251370675551300162610ustar00rootroot00000000000000Deploying web.py applications ============================= FastCGI ------- web.py uses `flup`_ library for supporting fastcgi. Make sure it is installed. .. _flup: http://trac.saddi.com/flup You just need to make sure you application file is executable. Make it so by adding the following line to tell the system to execute it using python:: #! /usr/bin/env python3 and setting the exeutable flag on the file:: chmod +x /path/to/yourapp.py Configuring lighttpd ^^^^^^^^^^^^^^^^^^^^ Here is a sample lighttpd configuration file to expose a web.py app using fastcgi. :: # Enable mod_fastcgi and mod_rewrite modules server.modules += ( "mod_fastcgi" ) server.modules += ( "mod_rewrite" ) # configure the application fastcgi.server = ( "/yourapp.py" => (( # path to the socket file "socket" => "/tmp/yourapp-fastcgi.socket", # path to the application "bin-path" => "/path/to/yourapp.py", # number of fastcgi processes to start "max-procs" => 1, "bin-environment" => ( "REAL_SCRIPT_NAME" => "" ), "check-local" => "disable" )) ) url.rewrite-once = ( # favicon is usually placed in static/ "^/favicon.ico$" => "/static/favicon.ico", # Let lighttpd serve resources from /static/. # The web.py dev server automatically servers /static/, but this is # required when deploying in production. "^/static/(.*)$" => "/static/$1", # everything else should go to the application, which is already configured above. "^/(.*)$" => "/yourapp.py/$1", ) With this configuration lighttpd takes care of starting the application. The webserver talks to your application using fastcgi via a unix domain socket. This means both the webserver and the application will run on the same machine. nginx + Gunicorn ---------------- Gunicorn 'Green Unicorn' is a Python WSGI HTTP Server for UNIX. It's a pre-fork worker model ported from Ruby's Unicorn project. To make a web.py application work with gunicorn, you'll need to get the wsgi app from web.py application object. :: import web ... app = web.application(urls, globals()) # get the wsgi app from web.py application object wsgiapp = app.wsgifunc() Once that change is made, gunicorn server be started using:: gunicorn -w 4 -b 127.0.0.1:4000 yourapp:wsgiapp This starts gunicorn with 4 workers and listens at port 4000 on localhost. It is best to use Gunicorn behind HTTP proxy server. The gunicorn team strongly advises to use nginx. Here is a sample nginx configuration which proxies to application running on `127.0.0.1:4000`. :: server { listen 80; server_name example.org; access_log /var/log/nginx/example.log; location / { proxy_pass http://127.0.0.1:4000; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } } webpy-0.61/docs/index.rst000066400000000000000000000020301370675551300153660ustar00rootroot00000000000000.. web.py documentation master file, created by sphinx-quickstart on Sun Oct 27 15:35:05 2013. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. Welcome to web.py's documentation! ================================== Contents: .. toctree:: :maxdepth: 3 urlmapping input db templating deploying api Getting Started =============== Building webapps with web.py is easy. To get started, save the following code as say, `hello.py` and run it with `python hello.py`. Now point your browser to `http://localhost:8080/` which responds you with 'Hello, world!'. Hey, you're done with your first program with with web.py - with just 8 lines of code! :: import web urls = ("/.*", "hello") app = web.application(urls, globals()) class hello: def GET(self): return 'Hello, world!' if __name__ == "__main__": app.run() Indices and tables ================== * :ref:`genindex` * :ref:`modindex` * :ref:`search` webpy-0.61/docs/input.rst000066400000000000000000000057151370675551300154330ustar00rootroot00000000000000Accessing User Input ==================== While building web applications, one basic and important thing is to respond to the user input that is sent to the server. Web.py makes it easy to access that whether it is parameters in the url (`GET` request) or the form data (`POST` or `PUT` request). The `web.input()` method returns a dictionary-like object (more specifically a `web.storage` object) that contains the user input, whatever the request method is. To access the URL parameters (?key=value) from the `web.input` object, just use `web.input().key`. GET --- For a URL which looks like `/page?id=1&action=edit`, you do :: class Page(object): def GET(self): data = web.input() id = int(data.id) # all the inputs are now strings. Cast it to int, to get integer. action = data.action ... `KeyError` exception is thrown if `key` is not there in the URL parameters. Web.py makes it easier to handle that with default values to web.input(). :: class Page(object): def GET(self): data = web.input(id=1, action='read') id, action = int(data.id), data.action ... POST ---- It works exactly the same way with POST method. If you have a form with `name` and `password` elements, you would do :: class Login(object): def POST(self): data = web.input() name, password = data.name, data.password ... Multiple inputs with same name ------------------------------ What if you have a URL which looks like `/page?id=1&id=2&id=3` or you have a form with multiple selects? What would `web.input().id` give us? It simply swallows all but one value. But to let web.input() know that we're expecting more values with the same name is simple. Just pass `[]` as the default argument for that name. :: class Page(object): def GET(self): data = web.input(id=[]) ids = data.id # now, `ids` is a list with all the `id`s. ... File uploads ------------ Uploading files is easy with web.py. `web.input()` takes care of that too. Just make sure that the upload form has an attribute enctype="multipart/form-data". The `input()` gives you `filename` and `value`, which are the uploaded file name and the contents of it, respectively. To make things simpler, it also gives you `file`, a file-like object if you pass `myfile={}` where `myfile` is the name of the input element in your form. :: class Upload(object): def GET(self): return render.upload() def POST(self): data = web.input(myfile={}) fp = data.myfile save(fp) # fp.filename, fp.read() gives name and contents of the file ... or :: class Upload(object): ... def POST(self): data = web.input() # notice that `myfile={}` is missing here. fp = data.myfile save(fp.filename, fp.value) ... webpy-0.61/docs/templating.rst000066400000000000000000000164751370675551300164450ustar00rootroot00000000000000Templating ========== There are almost as many Python templating systems as there are web frameworks (and, indeed, it seems like many templating systems are adopting web framework-like features). The following are the goals of `templetor`, which is the (codename of) templating system of web.py. 1. The templating system has to *look* decent. No ``<%#foo#%>`` crud. 2. Reuse Python terms and semantics as much as possible. 3. Expressive enough to do real computation. 4. Usable for any text language, not just HTML and XML. And requirements for the implementation as well: 5. Sandboxable so that you can let untrusted users write templates. 6. Simple and fast implementation. So here it is. Variable substitution --------------------- :: Look, a $string. Hark, an ${arbitrary + expression}. Gawk, a $dictionary[key].function('argument'). Cool, a $(limit)ing. Stop, \$money isn't evaluated. We use basically the same semantics as (rejected) `PEP 215 `__. Variables can go anywhere in a document. Newline suppression ------------------- :: If you put a backslash \ at the end of a line \ (like these) \ then there will be no newline. renders as all one line. Expressions ----------- :: Here are some expressions: $for var in iterator: I like $var! $if times > max: Stop! In the name of love. $else: Keep on, you can do it. That's all, folks. All your old Python friends are here: ``if``, ``while``, ``for``, ``else``, ``break``, ``continue``, and ``pass`` also act as you'd expect. (Obviously, you can't have variables named any of these.) The Python code starts at the ``$`` and ends at the ``:``. The ``$`` has to be at the beginning of the line, but that's not such a burden because of newline suppression (above). Also, we're very careful about spacing -- all the lines will render with no spaces at the beginning. (Open question: what if you want spaces at the beginning?) Also, a trailing space might break your code. There are a couple changes from Python: ``for`` and ``while`` now take an ``else`` clause that gets called if the loop is never evaluated. (Possible feature to add: Django-style for loop variables.) Comments -------- :: $# Here's where we hoodwink the folks at home: Please enter in your deets: CC: [ ] $#this is the important one SSN: $#Social Security Number#$ [ ] Comments start with ``$#`` and go to ``#$`` or the end of the line, whichever is first. Code ---- **NOTE: This feature has not been implemented in the current web.py implementation of templetor.** :: Sometimes you just need to break out the Python. $ mapping = { $ 'cool': ['nice', 'sweet', 'hot'], $ 'suck': ['bad', 'evil', 'awful'] $ } Isn't that $mapping[thought]? That's$ del mapping $ fine with me. $ complicatedfunc() $ for x in bugs: $ if bug.level == 'severe': Ooh, this one is bad. $ continue And there's $x... **Body of loops have to be indented with exactly 4 spaces.** Code begins with a ``$`` and a space and goes until the next ``$`` or the end of the line, whichever comes first. Nothing ever gets output if the first character after the ``$`` is a space (so ``complicatedfunc`` above doesn't write anything to the screen like it might without the space). Python integration ------------------ A template begins with a line like this: :: $def with (name, title, company='BigCo') which declares that the template takes those arguments. (The ``with`` keyword is special, like ``def`` or ``if``.) **Don't forget to put spaces in the definition** The following *will not work*: :: $def with (name,title,company='BigCo') Inside Python, the template looks like a function that takes these arguments. It returns a storage object with the special property that evaluating it as a string returns the value of the body of the template. The elements in the storage object are the results of the ``def``\ s and the ``set``\ s. Perhaps an example will make this clearer. Here's a template, "entry": :: $def with (post) $var title: $post.title

$markdown(post.body)

by $post.author

Here's another; "base": :: $def with (self) $self.title

$self.title

$:self Now let's say we compile both from within Python, the first as ``entry``, the second as ``base``. Here's how we might use them: :: print base( entry( post ) ) ``entry`` takes the argument post and returns an object whose string value is a bit of HTML showing the post with its title in the property ``title``. ``base`` takes this object and places the title in the appropriate place and displays the page itself in the body of the page. The Python code prints out the result. *Where did ``markdown`` come from? It wasn't passed as an argument.* You can pass a list of functions and variables to the template compiler to be made globally available to templates. *Why $:self?* See below Here's an example: :: import template render = template.render('templates/') template.Template.globals['len'] = len print render.base(render.message('Hello, world!')) The first line imports templetor. The second says that our templates are in the directory ``templates/``. The third give all our templates access to the ``len`` function. The fourth grabs the template ``message.html``, passes it the argument ``'Hello, world!'``, passes the result of rendering it to `mcitp `__ the template ``base.html`` and prints the result. (If your templates don't end in ``.html`` or ``.xml``, templetor will still find them, but it won't do its automatic HTML-encoding.) Turning Off Filter ------------------ By default ``template.render`` will use ``web.websafe`` filter to do HTML-encoding. To turn it off, put a : after the $ as in: :: $:form.render() Output from form.render() will be displayed as is. :: $:fooBar $# fooBar = lorem ipsum Output from variable in template will be displayed as is. Including / nesting templates ----------------------------- If you want to nest one template within another, you nest the ``render()`` calls, and then include the variable (unfiltered) in the page. In your handler: :: print render.foo(render.bar()) or (to make things a little more clear): :: barhtml = render.bar() print render.foo(barhtml) Then in the template ``foo.html``: :: $def with (bar) html goes here $:bar more html This replaces the ``$:bar`` with the output of the ``render.bar()`` call (which is why it must be ``$:``/unfiltered, so that you get un-encoded HTML (unless you want something else of course)). You can pass variables in, in the same way: :: print render.foo(render.bar(baz), qux) In the template bar (``bar.html``): :: $def with (baz) bar stuff goes here + baz In template foo (``foo.html``): :: $def with (bar, qux) html goes here $:bar Value of qux is $qux Escaping -------- web.py automatically escapes any variables used in templates, so that if for some reason name is set to a value containing some HTML, it will get properly escaped and appear as plain text. If you want to turn this off, write $:name instead of $name. webpy-0.61/pyproject.toml000066400000000000000000000004231370675551300155150ustar00rootroot00000000000000[tool.black] exclude = ''' ( /( \.eggs # exclude a few common directories in the | \.git # root of the project | \.hg | \.mypy_cache | \.tox | \.venv | _build | buck-out | build | dist )/ | docs/conf.py ) ''' webpy-0.61/requirements.txt000066400000000000000000000000171370675551300160640ustar00rootroot00000000000000cheroot>=6.0.0 webpy-0.61/runtests.sh000077500000000000000000000000371370675551300150300ustar00rootroot00000000000000#! /bin/bash py.test tests $* webpy-0.61/setup.py000066400000000000000000000021731370675551300143170ustar00rootroot00000000000000#!/usr/bin/env python3 import os from setuptools import setup from web import __version__ rootdir = os.path.abspath(os.path.dirname(__file__)) # Get the long description from the README file with open(os.path.join(rootdir, "README.md")) as in_file: long_description = in_file.read() setup( name="web.py", version=__version__, description="web.py: makes web apps", author="Aaron Swartz", author_email="me@aaronsw.com", maintainer="Anand Chitipothu", maintainer_email="anandology@gmail.com", url="http://webpy.org/", packages=["web", "web.contrib"], install_requires=["cheroot"], long_description=long_description, long_description_content_type="text/markdown", license="Public domain", platforms=["any"], python_requires=">=3.5", classifiers=[ "License :: Public Domain", "Programming Language :: Python", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.5", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Programming Language :: Python :: 3.8", ], ) webpy-0.61/test_requirements.txt000066400000000000000000000002261370675551300171250ustar00rootroot00000000000000cheroot>=6.0.0 DBUtils pytest>=5.4.1 # DB drivers. # mysql mysqlclient>=1.4.6 PyMySQL>=0.9.3 mysql-connector-python>=8.0.19 # pgsql psycopg2>=2.8.4 webpy-0.61/tests/000077500000000000000000000000001370675551300137445ustar00rootroot00000000000000webpy-0.61/tests/__init__.py000066400000000000000000000000001370675551300160430ustar00rootroot00000000000000webpy-0.61/tests/test_application.py000066400000000000000000000272531370675551300176710ustar00rootroot00000000000000import os import shutil import sys import threading import time import unittest import web try: from urllib.parse import urlencode except ImportError: from urllib import urlencode data = """ import web urls = ("/", "%(classname)s") app = web.application(urls, globals(), autoreload=True) class %(classname)s: def GET(self): return "%(output)s" """ urls = ("/iter", "do_iter") app = web.application(urls, globals()) class do_iter: def GET(self): yield "hello, " yield web.input(name="world").name POST = GET def write(filename, data): f = open(filename, "w") f.write(data) f.close() class ApplicationTest(unittest.TestCase): def test_reloader(self): write("foo.py", data % dict(classname="a", output="a")) import foo app = foo.app self.assertEqual(app.request("/").data, b"a") # test class change time.sleep(1) write("foo.py", data % dict(classname="a", output="b")) self.assertEqual(app.request("/").data, b"b") # test urls change time.sleep(1) write("foo.py", data % dict(classname="c", output="c")) self.assertEqual(app.request("/").data, b"c") def test_reloader_nested(self): try: shutil.rmtree("testpackage") except OSError: pass os.mkdir("testpackage") write("testpackage/__init__.py", "") write("testpackage/bar.py", data % dict(classname="a", output="a")) import testpackage.bar app = testpackage.bar.app self.assertEqual(app.request("/").data, b"a") # test class change time.sleep(1) write("testpackage/bar.py", data % dict(classname="a", output="b")) self.assertEqual(app.request("/").data, b"b") # test urls change time.sleep(1) write("testpackage/bar.py", data % dict(classname="c", output="c")) self.assertEqual(app.request("/").data, b"c") def testUppercaseMethods(self): urls = ("/", "hello") app = web.application(urls, locals()) class hello: def GET(self): return "hello" def internal(self): return "secret" response = app.request("/", method="internal") self.assertEqual(response.status, "405 Method Not Allowed") def testRedirect(self): # fmt: off urls = ( "/a", "redirect /hello/", "/b/(.*)", r"redirect /hello/\1", "/hello/(.*)", "hello" ) # fmt: on app = web.application(urls, locals()) class hello: def GET(self, name): name = name or "world" return "hello " + name response = app.request("/a") self.assertEqual(response.status, "301 Moved Permanently") self.assertEqual(response.headers["Location"], "http://0.0.0.0:8080/hello/") response = app.request("/a?x=2") self.assertEqual(response.status, "301 Moved Permanently") self.assertEqual(response.headers["Location"], "http://0.0.0.0:8080/hello/?x=2") response = app.request("/b/foo?x=2") self.assertEqual(response.status, "301 Moved Permanently") self.assertEqual( response.headers["Location"], "http://0.0.0.0:8080/hello/foo?x=2" ) def test_routing(self): urls = ("/foo", "foo") class foo: def GET(self): return "foo" app = web.application(urls, {"foo": foo}) self.assertEqual(app.request("/foo\n").data, b"not found") self.assertEqual(app.request("/foo").data, b"foo") def test_subdirs(self): urls = ("/(.*)", "blog") class blog: def GET(self, path): return "blog " + path app_blog = web.application(urls, locals()) # fmt: off urls = ( "/blog", app_blog, "/(.*)", "index" ) # fmt: on class index: def GET(self, path): return "hello " + path app = web.application(urls, locals()) self.assertEqual(app.request("/blog/foo").data, b"blog foo") self.assertEqual(app.request("/foo").data, b"hello foo") def processor(handler): return web.ctx.path + ":" + handler() app.add_processor(processor) self.assertEqual(app.request("/blog/foo").data, b"/blog/foo:blog foo") def test_subdomains(self): def create_app(name): urls = ("/", "index") class index: def GET(self): return name return web.application(urls, locals()) # fmt: off urls = ( "a.example.com", create_app('a'), "b.example.com", create_app('b'), ".*.example.com", create_app('*') ) # fmt: on app = web.subdomain_application(urls, locals()) def test(host, expected_result): result = app.request("/", host=host) self.assertEqual(result.data, expected_result) test("a.example.com", b"a") test("b.example.com", b"b") test("c.example.com", b"*") test("d.example.com", b"*") def test_redirect(self): urls = ("/(.*)", "blog") class blog: def GET(self, path): if path == "foo": raise web.seeother("/login", absolute=True) else: raise web.seeother("/bar") app_blog = web.application(urls, locals()) # fmt: off urls = ( "/blog", app_blog, "/(.*)", "index" ) # fmt: on class index: def GET(self, path): return "hello " + path app = web.application(urls, locals()) response = app.request("/blog/foo") self.assertEqual(response.headers["Location"], "http://0.0.0.0:8080/login") response = app.request("/blog/foo", env={"SCRIPT_NAME": "/x"}) self.assertEqual(response.headers["Location"], "http://0.0.0.0:8080/x/login") response = app.request("/blog/foo2") self.assertEqual(response.headers["Location"], "http://0.0.0.0:8080/blog/bar") response = app.request("/blog/foo2", env={"SCRIPT_NAME": "/x"}) self.assertEqual(response.headers["Location"], "http://0.0.0.0:8080/x/blog/bar") def test_processors(self): urls = ("/(.*)", "blog") class blog: def GET(self, path): return "blog " + path state = web.storage(x=0, y=0) def f(): state.x += 1 app_blog = web.application(urls, locals()) app_blog.add_processor(web.loadhook(f)) # fmt: off urls = ( "/blog", app_blog, "/(.*)", "index" ) # fmt: on class index: def GET(self, path): return "hello " + path app = web.application(urls, locals()) def g(): state.y += 1 app.add_processor(web.loadhook(g)) app.request("/blog/foo") assert state.x == 1 and state.y == 1, repr(state) app.request("/foo") assert state.x == 1 and state.y == 2, repr(state) def testUnicodeInput(self): urls = ("(/.*)", "foo") class foo: def GET(self, path): i = web.input(name="") return repr(i.name) def POST(self, path): if path == "/multipart": i = web.input(file={}) return i.file.value else: i = web.input() return repr(dict(i)).replace("u", "") app = web.application(urls, locals()) def f(name): path = "/?" + urlencode({"name": name.encode("utf-8")}) self.assertEqual(app.request(path).data.decode("utf-8"), repr(name)) f(u"\u1234") f(u"foo") response = app.request("/", method="POST", data=dict(name="foo")) self.assertEqual(response.data, b"{'name': 'foo'}") data = '--boundary\r\nContent-Disposition: form-data; name="x"\r\n\r\nfoo\r\n--boundary\r\nContent-Disposition: form-data; name="file"; filename="a.txt"\r\nContent-Type: text/plain\r\n\r\na\r\n--boundary--\r\n' headers = {"Content-Type": "multipart/form-data; boundary=boundary"} response = app.request("/multipart", method="POST", data=data, headers=headers) self.assertEqual(response.data, b"a") def testCustomNotFound(self): urls_a = ("/", "a") urls_b = ("/", "b") app_a = web.application(urls_a, locals()) app_b = web.application(urls_b, locals()) app_a.notfound = lambda: web.HTTPError("404 Not Found", {}, "not found 1") # fmt: off urls = ( "/a", app_a, "/b", app_b ) # fmt: on app = web.application(urls, locals()) def assert_notfound(path, message): response = app.request(path) self.assertEqual(response.status.split()[0], "404") self.assertEqual(response.data, message) assert_notfound("/a/foo", b"not found 1") assert_notfound("/b/foo", b"not found") app.notfound = lambda: web.HTTPError("404 Not Found", {}, "not found 2") assert_notfound("/a/foo", b"not found 1") assert_notfound("/b/foo", b"not found 2") def testIter(self): self.assertEqual(app.request("/iter").data, b"hello, world") self.assertEqual(app.request("/iter?name=web").data, b"hello, web") self.assertEqual(app.request("/iter", method="POST").data, b"hello, world") self.assertEqual( app.request("/iter", method="POST", data="name=web").data, b"hello, web" ) def testUnload(self): x = web.storage(a=0) # fmt: off urls = ( "/foo", "foo", "/bar", "bar" ) # fmt: on class foo: def GET(self): return "foo" class bar: def GET(self): raise web.notfound() app = web.application(urls, locals()) def unload(): x.a += 1 app.add_processor(web.unloadhook(unload)) app.request("/foo") self.assertEqual(x.a, 1) app.request("/bar") self.assertEqual(x.a, 2) def test_changequery(self): urls = ("/", "index") class index: def GET(self): return web.changequery(x=1) app = web.application(urls, locals()) def f(path): return app.request(path).data self.assertEqual(f("/?x=2"), b"/?x=1") p = f("/?y=1&y=2&x=2") self.assertTrue(p == b"/?y=1&y=2&x=1" or p == b"/?x=1&y=1&y=2") def test_setcookie(self): urls = ("/", "index") class index: def GET(self): web.setcookie("foo", "bar") return "hello" app = web.application(urls, locals()) def f(script_name=""): response = app.request("/", env={"SCRIPT_NAME": script_name}) return response.headers["Set-Cookie"] self.assertEqual(f(""), "foo=bar; Path=/") self.assertEqual(f("/admin"), "foo=bar; Path=/admin/") def test_stopsimpleserver(self): urls = ("/", "index") class index: def GET(self): pass # reset command-line arguments sys.argv = ["code.py"] app = web.application(urls, locals()) thread = threading.Thread(target=app.run) thread.start() time.sleep(1) self.assertTrue(thread.is_alive()) app.stop() thread.join(timeout=1) self.assertFalse(thread.is_alive()) webpy-0.61/tests/test_browser.py000066400000000000000000000031241370675551300170400ustar00rootroot00000000000000import unittest import web # fmt: off urls = ( "/", "index", "/hello/(.*)", "hello", "/cookie", "cookie", "/setcookie", "setcookie", "/redirect", "redirect", ) # fmt: on app = web.application(urls, globals()) class index: def GET(self): return "welcome" class hello: def GET(self, name): name = name or "world" return "hello, " + name + "!" class cookie: def GET(self): return ",".join(sorted(web.cookies().keys())) class setcookie: def GET(self): i = web.input() for k, v in i.items(): web.setcookie(k, v) return "done" class redirect: def GET(self): i = web.input(url="/") raise web.seeother(i.url) class BrowserTest(unittest.TestCase): def testCookies(self): b = app.browser() b.open("http://0.0.0.0/setcookie?x=1&y=2") b.open("http://0.0.0.0/cookie") self.assertEqual(b.text, "x,y") def testNotfound(self): b = app.browser() b.open("http://0.0.0.0/notfound") self.assertEqual(b.status, 404) def testRedirect(self): b = app.browser() b.open("http://0.0.0.0:8080/redirect") self.assertEqual(b.url, "http://0.0.0.0:8080/") b.open("http://0.0.0.0:8080/redirect?url=/hello/foo") self.assertEqual(b.url, "http://0.0.0.0:8080/hello/foo") b.open("https://0.0.0.0:8080/redirect") self.assertEqual(b.url, "https://0.0.0.0:8080/") b.open("https://0.0.0.0:8080/redirect?url=/hello/foo") self.assertEqual(b.url, "https://0.0.0.0:8080/hello/foo") webpy-0.61/tests/test_db.py000066400000000000000000000233031370675551300157430ustar00rootroot00000000000000"""DB test""" import importlib import os import unittest import web def try_import(name): try: return importlib.import_module(name) except ImportError: return None def requires_module(name): module = try_import(name) # this doesn't seem to be working. The simple decorator below seems to be working. # return pytest.mark.skipif(module is None, reason="requires {} module".format(name)) def decorator(cls): if module: return cls else: class Foo: pass print( "skipping all tests from {} as {} module is not found".format( cls.__name__, name ) ) return Foo return decorator def setup_database(dbname, driver=None, pooling=False): if dbname == "sqlite": db = web.database(dbn=dbname, db="webpy.db", pooling=pooling, driver=driver) elif dbname == "postgres": db = web.database( dbn=dbname, host=os.getenv("WEBPY_DB_HOST", "localhost"), port=int(os.getenv("WEBPY_DB_PG_PORT", 5432)), db=os.getenv("WEBPY_DB_NAME", "webpy"), user=os.getenv("WEBPY_DB_USER", os.getenv("USER")), pw=os.getenv("WEBPY_DB_PASSWORD", ""), pooling=pooling, driver=driver, ) else: db = web.database( dbn=dbname, host=os.getenv("WEBPY_DB_HOST", "localhost"), port=int(os.getenv("WEBPY_DB_MYSQL_PORT", 3306)), db=os.getenv("WEBPY_DB_NAME", "webpy"), user=os.getenv("WEBPY_DB_USER", os.getenv("USER")), pw=os.getenv("WEBPY_DB_PASSWORD", ""), pooling=pooling, driver=driver, ) db.printing = True return db class DBTest(unittest.TestCase): dbname = "postgres" driver = None def setUp(self): self.db = setup_database(self.dbname, driver=self.driver) self.db.query("DROP TABLE IF EXISTS person") self.db.query("CREATE TABLE person (name text, email text, active boolean)") def tearDown(self): self.db.query("DROP TABLE IF EXISTS person") self.db.query("DROP TABLE IF EXISTS mi") self.db.ctx.db.close() def _testable(self): try: setup_database(self.dbname, driver=self.driver) print("Running tests for %s" % self.__class__.__name__, file=web.debug) return True except ImportError as e: print(str(e), "(ignoring %s)" % self.__class__.__name__, file=web.debug) return False def testUnicode(self): # Bug#177265: unicode queries throw errors self.db.select("person", where="name=$name", vars={"name": u"\xf4"}) def assertRows(self, n): result = self.db.select("person") self.assertEqual(len(list(result)), n) def testCommit(self): t = self.db.transaction() self.db.insert("person", False, name="user1") t.commit() t = self.db.transaction() self.db.insert("person", False, name="user2") self.db.insert("person", False, name="user3") t.commit() self.assertRows(3) def testRollback(self): t = self.db.transaction() self.db.insert("person", False, name="user1") self.db.insert("person", False, name="user2") self.db.insert("person", False, name="user3") t.rollback() self.assertRows(0) def testWrongQuery(self): # It should be possible to run a correct query after getting an error from a wrong query. try: self.db.select("notthere") except: pass self.db.select("person") def testNestedTransactions(self): t1 = self.db.transaction() self.db.insert("person", False, name="user1") self.assertRows(1) t2 = self.db.transaction() self.db.insert("person", False, name="user2") self.assertRows(2) t2.rollback() self.assertRows(1) t3 = self.db.transaction() self.db.insert("person", False, name="user3") self.assertRows(2) t3.commit() t1.commit() self.assertRows(2) def testPooling(self): # can't test pooling if DBUtils is not installed try: import DBUtils # noqa except ImportError: return db = setup_database(self.dbname, pooling=True) try: self.assertEqual(db.ctx.db.__class__.__module__, "DBUtils.PooledDB") db.select("person", limit=1) finally: db.ctx.db.close() def test_multiple_insert(self): db = self.db db.multiple_insert("person", [dict(name="a"), dict(name="b")], seqname=False) assert db.select("person", where="name='a'").list() assert db.select("person", where="name='b'").list() # Create table `mi` if self.driver in web.db.pg_drivers: db.query("CREATE TABLE mi (id SERIAL PRIMARY KEY, v VARCHAR(5))") elif self.driver in web.db.mysql_drivers: self.db.query( "CREATE TABLE mi (id INT(10) UNSIGNED AUTO_INCREMENT, v VARCHAR(5), PRIMARY KEY (`id`))" ) elif self.driver in web.db.sqlite_drivers: self.db.query( "CREATE TABLE mi (id INTEGER PRIMARY KEY NOT NULL, v VARCHAR(5))" ) # Insert rows and verify returned row id. if ( self.driver in web.db.pg_drivers + web.db.mysql_drivers + web.db.sqlite_drivers ): values = [{"v": "a"}, {"v": "b"}, {"v": "c"}] ids = db.multiple_insert("mi", values) # `psycopg2` returns `range(1, 4)` instead of `[1, 2, 3]` on Python-3. assert list(ids) == [1, 2, 3] ids = db.multiple_insert("mi", values) assert list(ids) == [4, 5, 6] def test_result_is_true(self): self.db.insert("person", False, name="user") self.assertEqual(bool(self.db.select("person")), True) def testBoolean(self): def t(active): name = "name-%s" % active self.db.insert("person", False, name=name, active=active) a = self.db.select("person", where="name=$name", vars=locals())[0].active self.assertEqual(a, active) t(False) t(True) def test_insert_default_values(self): self.db.insert("person") def test_where(self): self.db.insert("person", False, name="Foo") d = self.db.where("person", name="Foo").list() assert len(d) == 1 d = self.db.where("person").list() assert len(d) == 1 @requires_module("psycopg2") class PostgresTest2(DBTest): dbname = "postgres" driver = "psycopg2" def setUp(self): super().setUp() self.db.query("DROP TABLE IF EXISTS post") self.db.query( "create table post (id serial primary key, title text, body text)" ) def tearDown(self): self.db.query("DROP TABLE IF EXISTS post") super().tearDown() def test_limit_with_unsafe_value(self): db = self.db db.insert("person", False, name="Foo") assert len(db.select("person").list()) == 1 try: db.select("person", limit="1; DELETE FROM person;") except db.db_module.Error: # It is alright if the db engine rejects this query pass assert len(db.select("person").list()) == 1 def test_offset_with_unsafe_value(self): db = self.db db.insert("person", False, name="Foo") assert len(db.select("person").list()) == 1 try: db.select("person", offset="1; DELETE FROM person;") except db.db_module.Error: # It is alright if the db engine rejects this query pass assert len(db.select("person").list()) == 1 def test_insert_returning(self): db = self.db row = db.query( "insert into post (title, body) values ('foo', 'bar') returning *" ).first() assert row.title == "foo" assert row.id is not None # create a new connection to the db db = setup_database(self.dbname, driver=self.driver) row = db.select("post").first() assert row is not None assert row.title == "foo" @requires_module("sqlite3") class SqliteTest(DBTest): dbname = "sqlite" driver = "sqlite3" def testNestedTransactions(self): # nested transactions does not work with sqlite pass def testPooling(self): # pooling is not support for sqlite pass @requires_module("pysqlite2.dbapi2") class SqliteTest_pysqlite2(SqliteTest): driver = "pysqlite2.dbapi2" @requires_module("MySQLdb") class MySQLTest_MySQLdb(DBTest): dbname = "mysql" driver = "MySQLdb" def setUp(self): self.db = setup_database(self.dbname, driver=self.driver) # In mysql, transactions are supported only with INNODB engine. self.db.query("CREATE TABLE person (name text, email text) ENGINE=INNODB") def testBoolean(self): # boolean datatype is not supported in MySQL (at least until v5.0) pass @requires_module("pymysql") class MySQLTest_PYMYSQL(DBTest): dbname = "mysql" driver = "pymysql" def setUp(self): self.db = setup_database(self.dbname, driver=self.driver) # In mysql, transactions are supported only with INNODB engine. self.db.query("CREATE TABLE person (name text, email text) ENGINE=INNODB") def testBoolean(self): # boolean datatype is not supported in MySQL (at least until v5.0) pass @requires_module("mysql.connector") class MySQLTest_MySQLConnector(MySQLTest_PYMYSQL): driver = "mysql.connector" del DBTest webpy-0.61/tests/test_form.py000066400000000000000000000003011370675551300163120ustar00rootroot00000000000000from web.form import Form, Textbox def test_id_escape_issue(): f = Form(Textbox("x", id="x ")) assert "" not in f.render() assert "" not in f.render_css() webpy-0.61/tests/test_session.py000066400000000000000000000100471370675551300170420ustar00rootroot00000000000000import os import time import tempfile import threading import unittest import shutil import web class SessionTest(unittest.TestCase): def setUp(self): app = web.auto_application() session = self.make_session(app) class count(app.page): def GET(self): session.count += 1 return str(session.count) class reset(app.page): def GET(self): session.kill() return "" class redirect(app.page): def GET(self): session.request_token = "123" raise web.redirect("/count") class get_session(app.page): path = "/session/(.*)" def GET(self, name): return session[name] self.app = app self.session = session def make_session(self, app): dir = tempfile.mkdtemp() store = web.session.DiskStore(dir) return web.session.Session(app, store, {"count": 0}) def testSession(self): b = self.app.browser() self.assertEqual(b.open("/count").read(), b"1") self.assertEqual(b.open("/count").read(), b"2") self.assertEqual(b.open("/count").read(), b"3") b.open("/reset") self.assertEqual(b.open("/count").read(), b"1") def testParallelSessions(self): b1 = self.app.browser() b2 = self.app.browser() b1.open("/count") for i in range(1, 10): self.assertEqual(b1.open("/count").read(), str(i + 1).encode("utf8")) self.assertEqual(b2.open("/count").read(), str(i).encode("utf8")) def testBadSessionId(self): b = self.app.browser() self.assertEqual(b.open("/count").read(), b"1") self.assertEqual(b.open("/count").read(), b"2") cookie = b.cookiejar._cookies["0.0.0.0"]["/"]["webpy_session_id"] cookie.value = "/etc/password" self.assertEqual(b.open("/count").read(), b"1") def testSlowCookies(self): b = self.app.browser() self.assertEqual(b.open("/count").read(), b"1") self.assertEqual(b.open("/count").read(), b"2") cookie = b.cookiejar._cookies["0.0.0.0"]["/"]["webpy_session_id"] cookie.value = '"/etc/password"' self.assertEqual(b.open("/count").read(), b"1") def testRedirect(self): b = self.app.browser() b.open("/redirect") b.open("/session/request_token") self.assertEqual(b.data, b"123") class DiskStoreTest(unittest.TestCase): def testRemovedSessionDir(self): # make sure `cleanup()` correctly returns when session directory was # removed. dir = tempfile.mkdtemp() s = web.session.DiskStore(dir) s["count"] = 20 self.assertEqual(s["count"], 20) shutil.rmtree(dir) time.sleep(2) s.cleanup(1) def testStoreConcurrent(self): dir = tempfile.mkdtemp() store = web.session.DiskStore(dir) def set_val(): store["fail"] = "value" for c in range(10): m = threading.Thread(target=set_val) m.start() try: value = store["fail"] except KeyError: pass self.assertEqual(value, "value") class DBSessionTest(SessionTest): """Session test with db store.""" def make_session(self, app): if os.path.exists("webpy.db"): os.remove("webpy.db") db = web.database(dbn="sqlite", db="webpy.db") # db.printing = True db.query( "" + "CREATE TABLE session (" + " session_id char(128) unique not null," + " atime timestamp default (datetime('now','utc'))," + " data text)" ) store = web.session.DBStore(db, "session") return web.session.Session(app, store, {"count": 0}) class MemorySessionTest(SessionTest): """Session test with db store.""" def make_session(self, app): store = web.session.MemoryStore() return web.session.Session(app, store, {"count": 0}) webpy-0.61/tests/test_template.py000066400000000000000000000022731370675551300171740ustar00rootroot00000000000000import unittest import web from web.template import SecurityError, Template class _TestResult: def __init__(self, t): self.t = t def __getattr__(self, name): return getattr(self.t, name) def __repr__(self): return repr(str(self.t)) def t(code, **keywords): tmpl = Template(code, **keywords) return lambda *a, **kw: _TestResult(tmpl(*a, **kw)) class TemplateTest(unittest.TestCase): """Tests for the template security feature.""" def testPrint(self): tpl = "$code:\n print('blah')" self.assertRaises(NameError, t(tpl)) def testAttr(self): tpl = "$code:\n (lambda x: x+1).func_code" self.assertRaises(SecurityError, t, tpl) tpl = "$def with (a)\n$code:\n a.b = 3" self.assertRaises(SecurityError, t, tpl) # these two should execute themselves flawlessly t("$code:\n foo = {'a': 1}.items()")() t("$code:\n bar = {k:0 for k in [1,2,3]}")() class TestRender: def test_template_without_ext(self, tmpdir): tmpdir.join("foobar").write("hello") render = web.template.render(str(tmpdir)) assert str(render.foobar()).strip() == "hello" webpy-0.61/tests/test_utils.py000066400000000000000000000010011370675551300165050ustar00rootroot00000000000000from web import utils def test_group(): assert list(utils.group([], 2)) == [] assert list(utils.group([1, 2, 3, 4, 5, 6, 7, 8, 9], 3)) == [ [1, 2, 3], [4, 5, 6], [7, 8, 9], ] assert list(utils.group([1, 2, 3, 4, 5, 6, 7, 8, 9], 4)) == [ [1, 2, 3, 4], [5, 6, 7, 8], [9], ] class TestIterBetter: def test_iter(self): assert list(utils.IterBetter(iter([]))) == [] assert list(utils.IterBetter(iter([1, 2, 3]))) == [1, 2, 3] webpy-0.61/tests/test_wsgi.py000066400000000000000000000031711370675551300163300ustar00rootroot00000000000000import threading import time import unittest import web try: # PY 3 from urllib.parse import unquote_to_bytes as unquote except ImportError: # PY 2 from urllib import unquote class WSGITest(unittest.TestCase): def test_layers_unicode(self): urls = ("/", "uni") class uni: def GET(self): return u"\u0C05\u0C06" app = web.application(urls, locals()) thread = threading.Thread(target=app.run) thread.start() time.sleep(0.5) b = web.browser.AppBrowser(app) r = b.open("/").read() s = r.decode("utf8") self.assertEqual(s, u"\u0C05\u0C06") app.stop() thread.join() def test_layers_bytes(self): urls = ("/", "bytes") class bytes: def GET(self): return b"abcdef" app = web.application(urls, locals()) thread = threading.Thread(target=app.run) thread.start() time.sleep(0.5) b = web.browser.AppBrowser(app) r = b.open("/") self.assertEqual(r.read(), b"abcdef") app.stop() thread.join() def test_unicode_url(self): urls = ("/([^/]+)", "url_passthrough") class url_passthrough: def GET(self, arg): return arg app = web.application(urls, locals()) thread = threading.Thread(target=app.run) thread.start() time.sleep(0.5) b = web.browser.AppBrowser(app) r = b.open("/%E2%84%A6") s = unquote(r.read()) self.assertEqual(s, b"\xE2\x84\xA6") app.stop() thread.join() webpy-0.61/tools/000077500000000000000000000000001370675551300137425ustar00rootroot00000000000000webpy-0.61/tools/_makedoc.py000066400000000000000000000050131370675551300160550ustar00rootroot00000000000000import os import web class Parser: def __init__(self): self.mode = "normal" self.text = "" def go(self, pyfile): for line in open(pyfile): if self.mode == "in def": self.text += " " + line.strip() if line.strip().endswith(":"): if self.definition(self.text): self.text = "" self.mode = "in func" else: self.text = "" self.mode = "normal" elif self.mode == "in func": if '"""' in line: self.text += line.strip().strip('"') self.mode = "in doc" if line.count('"""') == 2: self.mode = "normal" self.docstring(self.text) self.text = "" else: self.mode = "normal" elif self.mode == "in doc": self.text += " " + line if '"""' in line: self.mode = "normal" self.docstring(self.text.strip().strip('"')) self.text = "" elif line.startswith("## "): self.header(line.strip().strip("#")) elif line.startswith("def ") or line.startswith("class "): self.text += line.strip().strip(":") if line.strip().endswith(":"): if self.definition(self.text): self.text = "" self.mode = "in func" else: self.text = "" self.mode = "normal" else: self.mode = "in def" def clean(self, text): text = text.strip() text = text.replace("*", r"\*") return text def definition(self, text): text = web.lstrips(text, "def ") if text.startswith("_") or text.startswith("class _"): return False print("`" + text.strip() + "`") return True def docstring(self, text): print(" :", text.strip()) print() def header(self, text): print("##", text.strip()) print() for pyfile in os.listdir("web"): if pyfile[-2:] == "py": print() print("## " + pyfile) print() Parser().go("web/" + pyfile) print("`ctx`\n :", end=" ") print("\n".join(" " + x for x in web.ctx.__doc__.strip().split("\n"))) webpy-0.61/tools/makedoc.py000066400000000000000000000110741370675551300157220ustar00rootroot00000000000000""" Outputs web.py docs as html version 2.0: documents all code, and indents nicely. By Colin Rothwell (TheBoff) """ import inspect import sys import markdown from web.net import websafe sys.path.insert(0, "..") ALL_MODULES = [ "web.application", "web.contrib.template", "web.db", "web.debugerror", "web.form", "web.http", "web.httpserver", "web.net", "web.session", "web.template", "web.utils", "web.webapi", "web.wsgi", ] item_start = '' item_end = "" indent_amount = 30 doc_these = ( # These are the types of object that should be docced "module", "classobj", "instancemethod", "function", "type", "property", ) not_these_names = ( # Any particular object names that shouldn't be doced "fget", "fset", "fdel", "storage", # These stop the lower case versions getting docced "memoize", "iterbetter", "capturesstdout", "profile", "threadeddict", "d", # Don't know what this is, but only only conclude it shouldn't be doc'd ) css = """ """ indent_start = '
' indent_end = "
" header = """ """ def type_string(ob): return str(type(ob)).split("'")[1] def ts_css(text): """applies nice css to the type string""" return '%s' % text def arg_string(func): """Returns a nice argstring for a function or method""" return inspect.formatargspec(*inspect.getargspec(func)) def recurse_over(ob, name, indent_level=0): ts = type_string(ob) if ts not in doc_these: return # stos what shouldn't be docced getting docced if indent_level > 0 and ts == "module": return # Stops it getting into the stdlib if name in not_these_names: return # Stops things we don't want getting docced indent = indent_level * indent_amount # Indents nicely ds_indent = indent + (indent_amount / 2) if indent_level > 0: print(indent_start % indent) argstr = "" if ts.endswith(("function", "method")): argstr = arg_string(ob) elif ts == "classobj" or ts == "type": if ts == "classobj": ts = "class" if hasattr(ob, "__init__"): if type_string(ob.__init__) == "instancemethod": argstr = arg_string(ob.__init__) else: argstr = "(self)" if ts == "instancemethod": ts = "method" # looks much nicer ds = inspect.getdoc(ob) if ds is None: ds = "" ds = markdown.Markdown(ds) mlink = '' % name if ts == "module" else "" mend = "" if ts == "module" else "" print( "".join( ( "

", ts_css(ts), item_start % ts, " ", mlink, name, websafe(argstr), mend, item_end, "
", ) ) ) print("".join((indent_start % ds_indent, ds, indent_end, "

"))) # Although ''.join looks weird, it's a lot faster is string addition members = "" if hasattr(ob, "__all__"): members = ob.__all__ else: members = [item for item in dir(ob) if not item.startswith("_")] if "im_class" not in members: for name in members: recurse_over(getattr(ob, name), name, indent_level + 1) if indent_level > 0: print(indent_end) def main(modules=None): modules = modules or ALL_MODULES print("
") # Stops markdown vandalising my html. print(css) print(header) print("
    ") for name in modules: print('
  • %(name)s
    • ' % dict(name=name)) print("
") for name in modules: try: mod = __import__(name, {}, {}, "x") recurse_over(mod, name) except ImportError as e: print("Unable to import module %s (Error: %s)" % (name, e), file=sys.stderr) pass print("
") if __name__ == "__main__": main(sys.argv[1:]) webpy-0.61/web/000077500000000000000000000000001370675551300133575ustar00rootroot00000000000000webpy-0.61/web/__init__.py000066400000000000000000000014461370675551300154750ustar00rootroot00000000000000#!/usr/bin/env python3 """web.py: makes web apps (http://webpy.org)""" from . import ( # noqa: F401 db, debugerror, form, http, httpserver, net, session, template, utils, webapi, wsgi, ) from .application import * # noqa: F401,F403 from .db import * # noqa: F401,F403 from .debugerror import * # noqa: F401,F403 from .http import * # noqa: F401,F403 from .httpserver import * # noqa: F401,F403 from .net import * # noqa: F401,F403 from .utils import * # noqa: F401,F403 from .webapi import * # noqa: F401,F403 from .wsgi import * # noqa: F401,F403 __version__ = "0.61" __author__ = [ "Aaron Swartz ", "Anand Chitipothu ", ] __license__ = "public domain" __contributors__ = "see http://webpy.org/changes" webpy-0.61/web/application.py000066400000000000000000000622651370675551300162470ustar00rootroot00000000000000""" Web application (from web.py) """ import itertools import os import sys import traceback import wsgiref.handlers from inspect import isclass from io import BytesIO from . import browser, httpserver, utils from . import webapi as web from . import wsgi from .debugerror import debugerror from .py3helpers import iteritems from .utils import lstrips from urllib.parse import urlparse, urlencode, unquote try: reload # Python 2 except NameError: from importlib import reload # Python 3 __all__ = [ "application", "auto_application", "subdir_application", "subdomain_application", "loadhook", "unloadhook", "autodelegate", ] class application: """ Application to delegate requests based on path. >>> urls = ("/hello", "hello") >>> app = application(urls, globals()) >>> class hello: ... def GET(self): return "hello" >>> >>> app.request("/hello").data 'hello' """ # PY3DOCTEST: b'hello' def __init__(self, mapping=(), fvars={}, autoreload=None): if autoreload is None: autoreload = web.config.get("debug", False) self.init_mapping(mapping) self.fvars = fvars self.processors = [] self.add_processor(loadhook(self._load)) self.add_processor(unloadhook(self._unload)) if autoreload: def main_module_name(): mod = sys.modules["__main__"] file = getattr( mod, "__file__", None ) # make sure this works even from python interpreter return file and os.path.splitext(os.path.basename(file))[0] def modname(fvars): """find name of the module name from fvars.""" file, name = fvars.get("__file__"), fvars.get("__name__") if file is None or name is None: return None if name == "__main__": # Since the __main__ module can't be reloaded, the module has # to be imported using its file name. name = main_module_name() return name mapping_name = utils.dictfind(fvars, mapping) module_name = modname(fvars) def reload_mapping(): """loadhook to reload mapping and fvars.""" mod = __import__(module_name, None, None, [""]) mapping = getattr(mod, mapping_name, None) if mapping: self.fvars = mod.__dict__ self.init_mapping(mapping) self.add_processor(loadhook(Reloader())) if mapping_name and module_name: # when app is ran as part of a package, this puts the app into # `sys.modules` correctly, otherwise the first change to the # app module will not be picked up by Reloader reload_mapping() self.add_processor(loadhook(reload_mapping)) # load __main__ module usings its filename, so that it can be reloaded. if main_module_name() and "__main__" in sys.argv: try: __import__(main_module_name()) except ImportError: pass def _load(self): web.ctx.app_stack.append(self) def _unload(self): web.ctx.app_stack = web.ctx.app_stack[:-1] if web.ctx.app_stack: # this is a sub-application, revert ctx to earlier state. oldctx = web.ctx.get("_oldctx") if oldctx: web.ctx.home = oldctx.home web.ctx.homepath = oldctx.homepath web.ctx.path = oldctx.path web.ctx.fullpath = oldctx.fullpath def _cleanup(self): # Threads can be recycled by WSGI servers. # Clearing up all thread-local state to avoid interefereing with subsequent requests. utils.ThreadedDict.clear_all() def init_mapping(self, mapping): self.mapping = list(utils.group(mapping, 2)) def add_mapping(self, pattern, classname): self.mapping.append((pattern, classname)) def add_processor(self, processor): """ Adds a processor to the application. >>> urls = ("/(.*)", "echo") >>> app = application(urls, globals()) >>> class echo: ... def GET(self, name): return name ... >>> >>> def hello(handler): return "hello, " + handler() ... >>> app.add_processor(hello) >>> app.request("/web.py").data 'hello, web.py' """ # PY3DOCTEST: b'hello, web.py' self.processors.append(processor) def request( self, localpart="/", method="GET", data=None, host="0.0.0.0:8080", headers=None, https=False, **kw ): """Makes request to this application for the specified path and method. Response will be a storage object with data, status and headers. >>> urls = ("/hello", "hello") >>> app = application(urls, globals()) >>> class hello: ... def GET(self): ... web.header('Content-Type', 'text/plain') ... return "hello" ... >>> response = app.request("/hello") >>> response.data 'hello' >>> response.status '200 OK' >>> response.headers['Content-Type'] 'text/plain' To use https, use https=True. >>> urls = ("/redirect", "redirect") >>> app = application(urls, globals()) >>> class redirect: ... def GET(self): raise web.seeother("/foo") ... >>> response = app.request("/redirect") >>> response.headers['Location'] 'http://0.0.0.0:8080/foo' >>> response = app.request("/redirect", https=True) >>> response.headers['Location'] 'https://0.0.0.0:8080/foo' The headers argument specifies HTTP headers as a mapping object such as a dict. >>> urls = ('/ua', 'uaprinter') >>> class uaprinter: ... def GET(self): ... return 'your user-agent is ' + web.ctx.env['HTTP_USER_AGENT'] ... >>> app = application(urls, globals()) >>> app.request('/ua', headers = { ... 'User-Agent': 'a small jumping bean/1.0 (compatible)' ... }).data 'your user-agent is a small jumping bean/1.0 (compatible)' """ # PY3DOCTEST: b'hello' # PY3DOCTEST: b'your user-agent is a small jumping bean/1.0 (compatible)' _p = urlparse(localpart) path = _p.path maybe_query = _p.query query = maybe_query or "" if "env" in kw: env = kw["env"] else: env = {} env = dict( env, HTTP_HOST=host, REQUEST_METHOD=method, PATH_INFO=path, QUERY_STRING=query, HTTPS=str(https), ) headers = headers or {} for k, v in headers.items(): env["HTTP_" + k.upper().replace("-", "_")] = v if "HTTP_CONTENT_LENGTH" in env: env["CONTENT_LENGTH"] = env.pop("HTTP_CONTENT_LENGTH") if "HTTP_CONTENT_TYPE" in env: env["CONTENT_TYPE"] = env.pop("HTTP_CONTENT_TYPE") if method not in ["HEAD", "GET"]: data = data or "" if isinstance(data, dict): q = urlencode(data) else: q = data env["wsgi.input"] = BytesIO(q.encode("utf-8")) # if not env.get('CONTENT_TYPE', '').lower().startswith('multipart/') and 'CONTENT_LENGTH' not in env: if "CONTENT_LENGTH" not in env: env["CONTENT_LENGTH"] = len(q) response = web.storage() def start_response(status, headers): response.status = status response.headers = dict(headers) response.header_items = headers data = self.wsgifunc()(env, start_response) response.data = b"".join(data) return response def browser(self): return browser.AppBrowser(self) def handle(self): fn, args = self._match(self.mapping, web.ctx.path) return self._delegate(fn, self.fvars, args) def handle_with_processors(self): def process(processors): try: if processors: p, processors = processors[0], processors[1:] return p(lambda: process(processors)) else: return self.handle() except web.HTTPError: raise except (KeyboardInterrupt, SystemExit): raise except: print(traceback.format_exc(), file=web.debug) raise self.internalerror() # processors must be applied in the resvere order. (??) return process(self.processors) def wsgifunc(self, *middleware): """Returns a WSGI-compatible function for this application.""" def peep(iterator): """Peeps into an iterator by doing an iteration and returns an equivalent iterator. """ # wsgi requires the headers first # so we need to do an iteration # and save the result for later try: firstchunk = next(iterator) except StopIteration: firstchunk = "" return itertools.chain([firstchunk], iterator) def wsgi(env, start_resp): # clear threadlocal to avoid inteference of previous requests self._cleanup() self.load(env) try: # allow uppercase methods only if web.ctx.method.upper() != web.ctx.method: raise web.nomethod() result = self.handle_with_processors() if result and hasattr(result, "__next__"): result = peep(result) else: result = [result] except web.HTTPError as e: result = [e.data] def build_result(result): for r in result: if isinstance(r, bytes): yield r else: yield str(r).encode("utf-8") result = build_result(result) status, headers = web.ctx.status, web.ctx.headers start_resp(status, headers) def cleanup(): self._cleanup() yield b"" # force this function to be a generator return itertools.chain(result, cleanup()) for m in middleware: wsgi = m(wsgi) return wsgi def run(self, *middleware): """ Starts handling requests. If called in a CGI or FastCGI context, it will follow that protocol. If called from the command line, it will start an HTTP server on the port named in the first command line argument, or, if there is no argument, on port 8080. `middleware` is a list of WSGI middleware which is applied to the resulting WSGI function. """ return wsgi.runwsgi(self.wsgifunc(*middleware)) def stop(self): """Stops the http server started by run. """ if httpserver.server: httpserver.server.stop() httpserver.server = None def cgirun(self, *middleware): """ Return a CGI handler. This is mostly useful with Google App Engine. There you can just do: main = app.cgirun() """ wsgiapp = self.wsgifunc(*middleware) try: from google.appengine.ext.webapp.util import run_wsgi_app return run_wsgi_app(wsgiapp) except ImportError: # we're not running from within Google App Engine return wsgiref.handlers.CGIHandler().run(wsgiapp) def gaerun(self, *middleware): """ Starts the program in a way that will work with Google app engine, no matter which version you are using (2.5 / 2.7) If it is 2.5, just normally start it with app.gaerun() If it is 2.7, make sure to change the app.yaml handler to point to the global variable that contains the result of app.gaerun() For example: in app.yaml (where code.py is where the main code is located) handlers: - url: /.* script: code.app Make sure that the app variable is globally accessible """ wsgiapp = self.wsgifunc(*middleware) try: # check what version of python is running version = sys.version_info[:2] major = version[0] minor = version[1] if major != 2: raise EnvironmentError( "Google App Engine only supports python 2.5 and 2.7" ) # if 2.7, return a function that can be run by gae if minor == 7: return wsgiapp # if 2.5, use run_wsgi_app elif minor == 5: from google.appengine.ext.webapp.util import run_wsgi_app return run_wsgi_app(wsgiapp) else: raise EnvironmentError( "Not a supported platform, use python 2.5 or 2.7" ) except ImportError: return wsgiref.handlers.CGIHandler().run(wsgiapp) def load(self, env): """Initializes ctx using env.""" ctx = web.ctx ctx.clear() ctx.status = "200 OK" ctx.headers = [] ctx.output = "" ctx.environ = ctx.env = env ctx.host = env.get("HTTP_HOST") if env.get("wsgi.url_scheme") in ["http", "https"]: ctx.protocol = env["wsgi.url_scheme"] elif env.get("HTTPS", "").lower() in ["on", "true", "1"]: ctx.protocol = "https" else: ctx.protocol = "http" ctx.homedomain = ctx.protocol + "://" + env.get("HTTP_HOST", "[unknown]") ctx.homepath = os.environ.get("REAL_SCRIPT_NAME", env.get("SCRIPT_NAME", "")) ctx.home = ctx.homedomain + ctx.homepath # @@ home is changed when the request is handled to a sub-application. # @@ but the real home is required for doing absolute redirects. ctx.realhome = ctx.home ctx.ip = env.get("REMOTE_ADDR") ctx.method = env.get("REQUEST_METHOD") ctx.path = env.get("PATH_INFO").encode("latin1").decode("utf8") # http://trac.lighttpd.net/trac/ticket/406 requires: if env.get("SERVER_SOFTWARE", "").startswith("lighttpd/"): ctx.path = lstrips(env.get("REQUEST_URI").split("?")[0], ctx.homepath) # Apache and CherryPy webservers unquote the url but lighttpd doesn't. # unquote explicitly for lighttpd to make ctx.path uniform across all servers. ctx.path = unquote(ctx.path) if env.get("QUERY_STRING"): ctx.query = "?" + env.get("QUERY_STRING", "") else: ctx.query = "" ctx.fullpath = ctx.path + ctx.query for k, v in iteritems(ctx): # convert all string values to unicode values and replace # malformed data with a suitable replacement marker. if isinstance(v, bytes): ctx[k] = v.decode("utf-8", "replace") # status must always be str ctx.status = "200 OK" ctx.app_stack = [] def _delegate(self, f, fvars, args=[]): def handle_class(cls): meth = web.ctx.method if meth == "HEAD" and not hasattr(cls, meth): meth = "GET" if not hasattr(cls, meth): raise web.nomethod(cls) tocall = getattr(cls(), meth) return tocall(*args) if f is None: raise web.notfound() elif isinstance(f, application): return f.handle_with_processors() elif isclass(f): return handle_class(f) elif isinstance(f, str): if f.startswith("redirect "): url = f.split(" ", 1)[1] if web.ctx.method == "GET": x = web.ctx.env.get("QUERY_STRING", "") if x: url += "?" + x raise web.redirect(url) elif "." in f: mod, cls = f.rsplit(".", 1) mod = __import__(mod, None, None, [""]) cls = getattr(mod, cls) else: cls = fvars[f] return handle_class(cls) elif hasattr(f, "__call__"): return f() else: return web.notfound() def _match(self, mapping, value): for pat, what in mapping: if isinstance(what, application): if value.startswith(pat): f = lambda: self._delegate_sub_application(pat, what) return f, None else: continue elif isinstance(what, str): what, result = utils.re_subm(r"^%s\Z" % (pat,), what, value) else: result = utils.re_compile(r"^%s\Z" % (pat,)).match(value) if result: # it's a match return what, [x for x in result.groups()] return None, None def _delegate_sub_application(self, dir, app): """Deletes request to sub application `app` rooted at the directory `dir`. The home, homepath, path and fullpath values in web.ctx are updated to mimic request to the subapp and are restored after it is handled. @@Any issues with when used with yield? """ web.ctx._oldctx = web.storage(web.ctx) web.ctx.home += dir web.ctx.homepath += dir web.ctx.path = web.ctx.path[len(dir) :] web.ctx.fullpath = web.ctx.fullpath[len(dir) :] return app.handle_with_processors() def get_parent_app(self): if self in web.ctx.app_stack: index = web.ctx.app_stack.index(self) if index > 0: return web.ctx.app_stack[index - 1] def notfound(self): """Returns HTTPError with '404 not found' message""" parent = self.get_parent_app() if parent: return parent.notfound() else: return web._NotFound() def internalerror(self): """Returns HTTPError with '500 internal error' message""" parent = self.get_parent_app() if parent: return parent.internalerror() elif web.config.get("debug"): return debugerror() else: return web._InternalError() def with_metaclass(mcls): def decorator(cls): body = vars(cls).copy() # clean out class body body.pop("__dict__", None) body.pop("__weakref__", None) return mcls(cls.__name__, cls.__bases__, body) return decorator class auto_application(application): """Application similar to `application` but urls are constructed automatically using metaclass. >>> app = auto_application() >>> class hello(app.page): ... def GET(self): return "hello, world" ... >>> class foo(app.page): ... path = '/foo/.*' ... def GET(self): return "foo" >>> app.request("/hello").data 'hello, world' >>> app.request('/foo/bar').data 'foo' """ # PY3DOCTEST: b'hello, world' # PY3DOCTEST: b'foo' def __init__(self): application.__init__(self) class metapage(type): def __init__(klass, name, bases, attrs): type.__init__(klass, name, bases, attrs) path = attrs.get("path", "/" + name) # path can be specified as None to ignore that class # typically required to create a abstract base class. if path is not None: self.add_mapping(path, klass) @with_metaclass(metapage) # little hack needed for Py2 and Py3 compatibility class page: path = None self.page = page # The application class already has the required functionality of subdir_application subdir_application = application class subdomain_application(application): r""" Application to delegate requests based on the host. >>> urls = ("/hello", "hello") >>> app = application(urls, globals()) >>> class hello: ... def GET(self): return "hello" >>> >>> mapping = (r"hello\.example\.com", app) >>> app2 = subdomain_application(mapping) >>> app2.request("/hello", host="hello.example.com").data 'hello' >>> response = app2.request("/hello", host="something.example.com") >>> response.status '404 Not Found' >>> response.data 'not found' """ # PY3DOCTEST: b'hello' # PY3DOCTEST: b'not found' def handle(self): host = web.ctx.host.split(":")[0] # strip port fn, args = self._match(self.mapping, host) return self._delegate(fn, self.fvars, args) def _match(self, mapping, value): for pat, what in mapping: if isinstance(what, str): what, result = utils.re_subm("^" + pat + "$", what, value) else: result = utils.re_compile("^" + pat + "$").match(value) if result: # it's a match return what, [x for x in result.groups()] return None, None def loadhook(h): """ Converts a load hook into an application processor. >>> app = auto_application() >>> def f(): "something done before handling request" ... >>> app.add_processor(loadhook(f)) """ def processor(handler): h() return handler() return processor def unloadhook(h): """ Converts an unload hook into an application processor. >>> app = auto_application() >>> def f(): "something done after handling request" ... >>> app.add_processor(unloadhook(f)) """ def processor(handler): try: result = handler() except: # run the hook even when handler raises some exception h() raise if result and hasattr(result, "__next__"): return wrap(result) else: h() return result def wrap(result): def next_hook(): try: return next(result) except: # call the hook at the and of iterator h() raise result = iter(result) while True: try: yield next_hook() except StopIteration: return return processor def autodelegate(prefix=""): """ Returns a method that takes one argument and calls the method named prefix+arg, calling `notfound()` if there isn't one. Example: urls = ('/prefs/(.*)', 'prefs') class prefs: GET = autodelegate('GET_') def GET_password(self): pass def GET_privacy(self): pass `GET_password` would get called for `/prefs/password` while `GET_privacy` for `GET_privacy` gets called for `/prefs/privacy`. If a user visits `/prefs/password/change` then `GET_password(self, '/change')` is called. """ def internal(self, arg): if "/" in arg: first, rest = arg.split("/", 1) func = prefix + first args = ["/" + rest] else: func = prefix + arg args = [] if hasattr(self, func): try: return getattr(self, func)(*args) except TypeError: raise web.notfound() else: raise web.notfound() return internal class Reloader: """Checks to see if any loaded modules have changed on disk and, if so, reloads them. """ """File suffix of compiled modules.""" if sys.platform.startswith("java"): SUFFIX = "$py.class" else: SUFFIX = ".pyc" def __init__(self): self.mtimes = {} def __call__(self): sys_modules = list(sys.modules.values()) for mod in sys_modules: self.check(mod) def check(self, mod): # jython registers java packages as modules but they either # don't have a __file__ attribute or its value is None if not (mod and hasattr(mod, "__file__") and mod.__file__): return try: mtime = os.stat(mod.__file__).st_mtime except (OSError, IOError): return if mod.__file__.endswith(self.__class__.SUFFIX) and os.path.exists( mod.__file__[:-1] ): mtime = max(os.stat(mod.__file__[:-1]).st_mtime, mtime) if mod not in self.mtimes: self.mtimes[mod] = mtime elif self.mtimes[mod] < mtime: try: reload(mod) self.mtimes[mod] = mtime except ImportError: pass if __name__ == "__main__": import doctest doctest.testmod() webpy-0.61/web/browser.py000066400000000000000000000177671370675551300154360ustar00rootroot00000000000000"""Browser to test web applications. (from web.py) """ import os import webbrowser from io import BytesIO from .net import htmlunquote from .utils import re_compile from urllib.request import HTTPHandler, HTTPCookieProcessor, Request, HTTPError from urllib.request import build_opener as urllib_build_opener from urllib.parse import urljoin from http.cookiejar import CookieJar from urllib.response import addinfourl DEBUG = False __all__ = ["BrowserError", "Browser", "AppBrowser", "AppHandler"] class BrowserError(Exception): pass class Browser(object): def __init__(self): self.cookiejar = CookieJar() self._cookie_processor = HTTPCookieProcessor(self.cookiejar) self.form = None self.url = "http://0.0.0.0:8080/" self.path = "/" self.status = None self.data = None self._response = None self._forms = None @property def text(self): return self.data.decode("utf-8") def reset(self): """Clears all cookies and history.""" self.cookiejar.clear() def build_opener(self): """Builds the opener using (urllib2/urllib.request).build_opener. Subclasses can override this function to prodive custom openers. """ return urllib_build_opener() def do_request(self, req): if DEBUG: print("requesting", req.get_method(), req.get_full_url()) opener = self.build_opener() opener.add_handler(self._cookie_processor) try: self._response = opener.open(req) except HTTPError as e: self._response = e self.url = self._response.geturl() self.path = Request(self.url).selector self.data = self._response.read() self.status = self._response.code self._forms = None self.form = None return self.get_response() def open(self, url, data=None, headers={}): """Opens the specified url.""" url = urljoin(self.url, url) req = Request(url, data, headers) return self.do_request(req) def show(self): """Opens the current page in real web browser.""" f = open("page.html", "w") f.write(self.data) f.close() url = "file://" + os.path.abspath("page.html") webbrowser.open(url) def get_response(self): """Returns a copy of the current response.""" return addinfourl( BytesIO(self.data), self._response.info(), self._response.geturl() ) def get_soup(self): """Returns beautiful soup of the current document.""" import BeautifulSoup return BeautifulSoup.BeautifulSoup(self.data) def get_text(self, e=None): """Returns content of e or the current document as plain text.""" e = e or self.get_soup() return "".join( [htmlunquote(c) for c in e.recursiveChildGenerator() if isinstance(c, str)] ) def _get_links(self): soup = self.get_soup() return [a for a in soup.findAll(name="a")] def get_links( self, text=None, text_regex=None, url=None, url_regex=None, predicate=None ): """Returns all links in the document.""" return self._filter_links( self._get_links(), text=text, text_regex=text_regex, url=url, url_regex=url_regex, predicate=predicate, ) def follow_link( self, link=None, text=None, text_regex=None, url=None, url_regex=None, predicate=None, ): if link is None: links = self._filter_links( self.get_links(), text=text, text_regex=text_regex, url=url, url_regex=url_regex, predicate=predicate, ) link = links and links[0] if link: return self.open(link["href"]) else: raise BrowserError("No link found") def find_link( self, text=None, text_regex=None, url=None, url_regex=None, predicate=None ): links = self._filter_links( self.get_links(), text=text, text_regex=text_regex, url=url, url_regex=url_regex, predicate=predicate, ) return links and links[0] or None def _filter_links( self, links, text=None, text_regex=None, url=None, url_regex=None, predicate=None, ): predicates = [] if text is not None: predicates.append(lambda link: link.string == text) if text_regex is not None: predicates.append( lambda link: re_compile(text_regex).search(link.string or "") ) if url is not None: predicates.append(lambda link: link.get("href") == url) if url_regex is not None: predicates.append( lambda link: re_compile(url_regex).search(link.get("href", "")) ) if predicate: predicate.append(predicate) def f(link): for p in predicates: if not p(link): return False return True return [link for link in links if f(link)] def get_forms(self): """Returns all forms in the current document. The returned form objects implement the ClientForm.HTMLForm interface. """ if self._forms is None: import ClientForm self._forms = ClientForm.ParseResponse( self.get_response(), backwards_compat=False ) return self._forms def select_form(self, name=None, predicate=None, index=0): """Selects the specified form.""" forms = self.get_forms() if name is not None: forms = [f for f in forms if f.name == name] if predicate: forms = [f for f in forms if predicate(f)] if forms: self.form = forms[index] return self.form else: raise BrowserError("No form selected.") def submit(self, **kw): """submits the currently selected form.""" if self.form is None: raise BrowserError("No form selected.") req = self.form.click(**kw) return self.do_request(req) def __getitem__(self, key): return self.form[key] def __setitem__(self, key, value): self.form[key] = value class AppBrowser(Browser): """Browser interface to test web.py apps. b = AppBrowser(app) b.open('/') b.follow_link(text='Login') b.select_form(name='login') b['username'] = 'joe' b['password'] = 'secret' b.submit() assert b.path == '/' assert 'Welcome joe' in b.get_text() """ def __init__(self, app): Browser.__init__(self) self.app = app def build_opener(self): return urllib_build_opener(AppHandler(self.app)) class AppHandler(HTTPHandler): """urllib2 handler to handle requests using web.py application.""" handler_order = 100 https_request = HTTPHandler.do_request_ def __init__(self, app): self.app = app def http_open(self, req): result = self.app.request( localpart=req.selector, method=req.get_method(), host=req.host, data=req.data, headers=dict(req.header_items()), https=(req.type == "https"), ) return self._make_response(result, req.get_full_url()) def https_open(self, req): return self.http_open(req) def _make_response(self, result, url): data = "\r\n".join(["%s: %s" % (k, v) for k, v in result.header_items]) import email headers = email.message_from_string(data) response = addinfourl(BytesIO(result.data), headers, url) code, msg = result.status.split(None, 1) response.code, response.msg = int(code), msg return response webpy-0.61/web/contrib/000077500000000000000000000000001370675551300150175ustar00rootroot00000000000000webpy-0.61/web/contrib/__init__.py000066400000000000000000000000001370675551300171160ustar00rootroot00000000000000webpy-0.61/web/contrib/template.py000066400000000000000000000066041370675551300172120ustar00rootroot00000000000000""" Interface to various templating engines. """ import os.path __all__ = ["render_cheetah", "render_genshi", "render_mako", "cache"] class render_cheetah: """Rendering interface to Cheetah Templates. Example: render = render_cheetah('templates') render.hello(name="cheetah") """ def __init__(self, path): # give error if Chetah is not installed from Cheetah.Template import Template # noqa: F401 self.path = path def __getattr__(self, name): from Cheetah.Template import Template path = os.path.join(self.path, name + ".html") def template(**kw): t = Template(file=path, searchList=[kw]) return t.respond() return template class render_genshi: """Rendering interface genshi templates. Example: for xml/html templates. render = render_genshi(['templates/']) render.hello(name='genshi') For text templates: render = render_genshi(['templates/'], type='text') render.hello(name='genshi') """ def __init__(self, *a, **kwargs): from genshi.template import TemplateLoader self._type = kwargs.pop("type", None) self._loader = TemplateLoader(*a, **kwargs) def __getattr__(self, name): # Assuming all templates are html path = name + ".html" if self._type == "text": from genshi.template import TextTemplate cls = TextTemplate type = "text" else: cls = None type = self._type t = self._loader.load(path, cls=cls) def template(**kw): stream = t.generate(**kw) if type: return stream.render(type) else: return stream.render() return template class render_jinja: """Rendering interface to Jinja2 Templates Example: render= render_jinja('templates') render.hello(name='jinja2') """ def __init__(self, *a, **kwargs): extensions = kwargs.pop("extensions", []) globals = kwargs.pop("globals", {}) from jinja2 import Environment, FileSystemLoader self._lookup = Environment( loader=FileSystemLoader(*a, **kwargs), extensions=extensions ) self._lookup.globals.update(globals) def __getattr__(self, name): # Assuming all templates end with .html path = name + ".html" t = self._lookup.get_template(path) return t.render class render_mako: """Rendering interface to Mako Templates. Example: render = render_mako(directories=['templates']) render.hello(name="mako") """ def __init__(self, *a, **kwargs): from mako.lookup import TemplateLookup self._lookup = TemplateLookup(*a, **kwargs) def __getattr__(self, name): # Assuming all templates are html path = name + ".html" t = self._lookup.get_template(path) return t.render class cache: """Cache for any rendering interface. Example: render = cache(render_cheetah("templates/")) render.hello(name='cache') """ def __init__(self, render): self._render = render self._cache = {} def __getattr__(self, name): if name not in self._cache: self._cache[name] = getattr(self._render, name) return self._cache[name] webpy-0.61/web/db.py000066400000000000000000001513171370675551300143260ustar00rootroot00000000000000""" Database API (part of web.py) """ import datetime import os import re import time from .py3helpers import iteritems from .utils import iters, safestr, safeunicode, storage, threadeddict try: from urllib import parse as urlparse from urllib.parse import unquote except ImportError: import urlparse from urllib import unquote try: import ast except ImportError: ast = None try: # db module can work independent of web.py from .webapi import debug, config except ImportError: import sys debug = sys.stderr config = storage() __all__ = [ "UnknownParamstyle", "UnknownDB", "TransactionError", "sqllist", "sqlors", "reparam", "sqlquote", "SQLQuery", "SQLParam", "sqlparam", "SQLLiteral", "sqlliteral", "database", "DB", ] TOKEN = "[ \\f\\t]*(\\\\\\r?\\n[ \\f\\t]*)*(#[^\\r\\n]*)?(((\\d+[jJ]|((\\d+\\.\\d*|\\.\\d+)([eE][-+]?\\d+)?|\\d+[eE][-+]?\\d+)[jJ])|((\\d+\\.\\d*|\\.\\d+)([eE][-+]?\\d+)?|\\d+[eE][-+]?\\d+)|(0[xX][\\da-fA-F]+[lL]?|0[bB][01]+[lL]?|(0[oO][0-7]+)|(0[0-7]*)[lL]?|[1-9]\\d*[lL]?))|((\\*\\*=?|>>=?|<<=?|<>|!=|//=?|[+\\-*/%&|^=<>]=?|~)|[][(){}]|(\\r?\\n|[:;.,`@]))|([uUbB]?[rR]?'[^\\n'\\\\]*(?:\\\\.[^\\n'\\\\]*)*'|[uUbB]?[rR]?\"[^\\n\"\\\\]*(?:\\\\.[^\\n\"\\\\]*)*\")|[a-zA-Z_]\\w*)" tokenprog = re.compile(TOKEN) # Supported db drivers. pg_drivers = ("psycopg2",) mysql_drivers = ("pymysql", "MySQLdb", "mysql.connector") sqlite_drivers = ("sqlite3", "pysqlite2.dbapi2", "sqlite") class UnknownDB(Exception): """raised for unsupported dbms""" pass class _ItplError(ValueError): def __init__(self, text, pos): ValueError.__init__(self) self.text = text self.pos = pos def __str__(self): return "unfinished expression in %s at char %d" % (repr(self.text), self.pos) class TransactionError(Exception): pass class UnknownParamstyle(Exception): """ raised for unsupported db paramstyles (currently supported: qmark, numeric, format, pyformat) """ pass class SQLParam(object): """ Parameter in SQLQuery. >>> q = SQLQuery(["SELECT * FROM test WHERE name=", SQLParam("joe")]) >>> q >>> q.query() 'SELECT * FROM test WHERE name=%s' >>> q.values() ['joe'] """ __slots__ = ["value"] def __init__(self, value): self.value = value def get_marker(self, paramstyle="pyformat"): if paramstyle == "qmark": return "?" elif paramstyle == "numeric": return ":1" elif paramstyle is None or paramstyle in ["format", "pyformat"]: return "%s" raise UnknownParamstyle(paramstyle) def sqlquery(self): return SQLQuery([self]) def __add__(self, other): return self.sqlquery() + other def __radd__(self, other): return other + self.sqlquery() def __str__(self): return str(self.value) def __eq__(self, other): return isinstance(other, SQLParam) and other.value == self.value def __repr__(self): return "" % repr(self.value) sqlparam = SQLParam class SQLQuery(object): """ You can pass this sort of thing as a clause in any db function. Otherwise, you can pass a dictionary to the keyword argument `vars` and the function will call reparam for you. Internally, consists of `items`, which is a list of strings and SQLParams, which get concatenated to produce the actual query. """ __slots__ = ["items"] # tested in sqlquote's docstring def __init__(self, items=None): r"""Creates a new SQLQuery. >>> SQLQuery("x") >>> q = SQLQuery(['SELECT * FROM ', 'test', ' WHERE x=', SQLParam(1)]) >>> q >>> q.query(), q.values() ('SELECT * FROM test WHERE x=%s', [1]) >>> SQLQuery(SQLParam(1)) """ if items is None: self.items = [] elif isinstance(items, list): self.items = items elif isinstance(items, SQLParam): self.items = [items] elif isinstance(items, SQLQuery): self.items = list(items.items) else: self.items = [items] # Take care of SQLLiterals for i, item in enumerate(self.items): if isinstance(item, SQLParam) and isinstance(item.value, SQLLiteral): self.items[i] = item.value.v def append(self, value): self.items.append(value) def __add__(self, other): if isinstance(other, str): items = [other] elif isinstance(other, SQLQuery): items = other.items else: return NotImplemented return SQLQuery(self.items + items) def __radd__(self, other): if isinstance(other, str): items = [other] elif isinstance(other, SQLQuery): items = other.items else: return NotImplemented return SQLQuery(items + self.items) def __iadd__(self, other): if isinstance(other, (str, SQLParam)): self.items.append(other) elif isinstance(other, SQLQuery): self.items.extend(other.items) else: return NotImplemented return self def __len__(self): return len(self.query()) def __eq__(self, other): return isinstance(other, SQLQuery) and other.items == self.items def query(self, paramstyle=None): """ Returns the query part of the sql query. >>> q = SQLQuery(["SELECT * FROM test WHERE name=", SQLParam('joe')]) >>> q.query() 'SELECT * FROM test WHERE name=%s' >>> q.query(paramstyle='qmark') 'SELECT * FROM test WHERE name=?' """ s = [] for x in self.items: if isinstance(x, SQLParam): x = x.get_marker(paramstyle) s.append(safestr(x)) else: x = safestr(x) # automatically escape % characters in the query # For backward compatibility, ignore escaping when the query # looks already escaped if paramstyle in ["format", "pyformat"]: if "%" in x and "%%" not in x: x = x.replace("%", "%%") s.append(x) return "".join(s) def values(self): """ Returns the values of the parameters used in the sql query. >>> q = SQLQuery(["SELECT * FROM test WHERE name=", SQLParam('joe')]) >>> q.values() ['joe'] """ return [i.value for i in self.items if isinstance(i, SQLParam)] def join(items, sep=" ", prefix=None, suffix=None, target=None): """ Joins multiple queries. >>> SQLQuery.join(['a', 'b'], ', ') Optionally, prefix and suffix arguments can be provided. >>> SQLQuery.join(['a', 'b'], ', ', prefix='(', suffix=')') If target argument is provided, the items are appended to target instead of creating a new SQLQuery. """ if target is None: target = SQLQuery() target_items = target.items if prefix: target_items.append(prefix) for i, item in enumerate(items): if i != 0 and sep != "": target_items.append(sep) if isinstance(item, SQLQuery): target_items.extend(item.items) elif item == "": # joins with empty strings continue else: target_items.append(item) if suffix: target_items.append(suffix) return target join = staticmethod(join) def _str(self): try: return self.query() % tuple([sqlify(x) for x in self.values()]) except (ValueError, TypeError): return self.query() def __str__(self): return safestr(self._str()) def __unicode__(self): return safeunicode(self._str()) def __repr__(self): return "" % repr(str(self)) class SQLLiteral: """ Protects a string from `sqlquote`. >>> sqlquote('NOW()') >>> sqlquote(SQLLiteral('NOW()')) """ def __init__(self, v): self.v = v def __repr__(self): return "" % self.v sqlliteral = SQLLiteral def _sqllist(values): """ >>> _sqllist([1, 2, 3]) >>> _sqllist(set([5, 1, 3, 2])) >>> _sqllist((5, 1, 3, 2, 2, 5)) """ items = [] items.append("(") if isinstance(values, set): values = list(values) elif isinstance(values, tuple): values = list(set(values)) for i, v in enumerate(values): if i != 0: items.append(", ") items.append(sqlparam(v)) items.append(")") return SQLQuery(items) def reparam(string_, dictionary): """ Takes a string and a dictionary and interpolates the string using values from the dictionary. Returns an `SQLQuery` for the result. >>> reparam("s = $s", dict(s=True)) >>> reparam("s IN $s", dict(s=[1, 2])) """ return SafeEval().safeeval(string_, dictionary) dictionary = dictionary.copy() # eval mucks with it # disable builtins to avoid risk for remote code execution. dictionary["__builtins__"] = object() result = [] for live, chunk in _interpolate(string_): if live: v = eval(chunk, dictionary) result.append(sqlquote(v)) else: result.append(chunk) return SQLQuery.join(result, "") def sqlify(obj): """ converts `obj` to its proper SQL version >>> sqlify(None) 'NULL' >>> sqlify(True) "'t'" >>> sqlify(3) '3' """ # because `1 == True and hash(1) == hash(True)` # we have to do this the hard way... if obj is None: return "NULL" elif obj is True: return "'t'" elif obj is False: return "'f'" elif isinstance(obj, int): return str(obj) elif isinstance(obj, datetime.datetime): return repr(obj.isoformat()) else: return repr(obj) def sqllist(lst): """ Converts the arguments for use in something like a WHERE clause. >>> sqllist(['a', 'b']) 'a, b' >>> sqllist('a') 'a' """ if isinstance(lst, str): return lst else: return ", ".join(lst) def sqlors(left, lst): """ `left is a SQL clause like `tablename.arg = ` and `lst` is a list of values. Returns a reparam-style pair featuring the SQL that ORs together the clause for each item in the lst. >>> sqlors('foo = ', []) >>> sqlors('foo = ', [1]) >>> sqlors('foo = ', 1) >>> sqlors('foo = ', [1,2,3]) """ if isinstance(lst, iters): lst = list(lst) ln = len(lst) if ln == 0: return SQLQuery("1=2") if ln == 1: lst = lst[0] if isinstance(lst, iters): return SQLQuery( ["("] + sum([[left, sqlparam(x), " OR "] for x in lst], []) + ["1=2)"] ) else: return left + sqlparam(lst) def sqlwhere(data, grouping=" AND "): """ Converts a two-tuple (key, value) iterable `data` to an SQL WHERE clause `SQLQuery`. >>> sqlwhere((('cust_id', 2), ('order_id',3))) >>> sqlwhere((('order_id', 3), ('cust_id', 2)), grouping=', ') >>> sqlwhere((('a', 'a'), ('b', 'b'))).query() 'a = %s AND b = %s' """ return SQLQuery.join([k + " = " + sqlparam(v) for k, v in data], grouping) def sqlquote(a): """ Ensures `a` is quoted properly for use in a SQL query. >>> 'WHERE x = ' + sqlquote(True) + ' AND y = ' + sqlquote(3) >>> 'WHERE x = ' + sqlquote(True) + ' AND y IN ' + sqlquote([2, 3]) >>> 'WHERE x = ' + sqlquote(True) + ' AND y IN ' + sqlquote(set([3, 2, 3, 4])) >>> 'WHERE x = ' + sqlquote(True) + ' AND y IN ' + sqlquote((3, 2, 3, 4)) """ if isinstance(a, (list, tuple, set)): return _sqllist(a) else: return sqlparam(a).sqlquery() class BaseResultSet: """Base implementation of Result Set, the result of a db query. """ def __init__(self, cursor): self.cursor = cursor self.names = [x[0] for x in cursor.description] self._index = 0 def list(self): rows = [self._prepare_row(d) for d in self.cursor.fetchall()] self._index += len(rows) return rows def _prepare_row(self, row): return storage(dict(zip(self.names, row))) def __iter__(self): return self def __next__(self): row = self.cursor.fetchone() if row is None: raise StopIteration() self._index += 1 return self._prepare_row(row) next = __next__ # for python 2.7 support def first(self, default=None): """Returns the first row of this ResultSet or None when there are no elements. If the optional argument default is specified, that is returned instead of None when there are no elements. """ try: return next(iter(self)) except StopIteration: return default def __getitem__(self, i): # todo: slices if i < self._index: raise IndexError("already passed " + str(i)) try: while i > self._index: next(self) self._index += 1 # now self._index == i self._index += 1 return next(self) except StopIteration: raise IndexError(str(i)) class ResultSet(BaseResultSet): """The result of a database query. """ def __len__(self): return int(self.cursor.rowcount) class SqliteResultSet(BaseResultSet): """Result Set for sqlite. Same functionally as ResultSet except len is not supported. """ def __init__(self, cursor): BaseResultSet.__init__(self, cursor) self._head = None def __next__(self): if self._head is not None: self._index += 1 return self._head else: return super().__next__() def __bool__(self): # The ResultSet class class doesn't need to support __bool__ explicitly # because it has __len__. Since SqliteResultSet doesn't support len, # we need to peep into the result to find if the result is empty of not. if self._head is None: try: self._head = next(self) self._index -= 1 # reset the index except StopIteration: return False return True class Transaction: """Database transaction.""" def __init__(self, ctx): self.ctx = ctx self.transaction_count = transaction_count = len(ctx.transactions) class transaction_engine: """Transaction Engine used in top level transactions.""" def do_transact(self): ctx.commit(unload=False) def do_commit(self): ctx.commit() def do_rollback(self): ctx.rollback() class subtransaction_engine: """Transaction Engine used in sub transactions.""" def query(self, q): db_cursor = ctx.db.cursor() ctx.db_execute(db_cursor, SQLQuery(q % transaction_count)) def do_transact(self): self.query("SAVEPOINT webpy_sp_%s") def do_commit(self): self.query("RELEASE SAVEPOINT webpy_sp_%s") def do_rollback(self): self.query("ROLLBACK TO SAVEPOINT webpy_sp_%s") class dummy_engine: """Transaction Engine used instead of subtransaction_engine when sub transactions are not supported.""" do_transact = do_commit = do_rollback = lambda self: None if self.transaction_count: # nested transactions are not supported in some databases if self.ctx.get("ignore_nested_transactions"): self.engine = dummy_engine() else: self.engine = subtransaction_engine() else: self.engine = transaction_engine() self.engine.do_transact() self.ctx.transactions.append(self) def __enter__(self): return self def __exit__(self, exctype, excvalue, traceback): if exctype is not None: self.rollback() else: self.commit() def commit(self): if len(self.ctx.transactions) > self.transaction_count: self.engine.do_commit() self.ctx.transactions = self.ctx.transactions[: self.transaction_count] def rollback(self): if len(self.ctx.transactions) > self.transaction_count: self.engine.do_rollback() self.ctx.transactions = self.ctx.transactions[: self.transaction_count] class DB: """Database""" def __init__(self, db_module, keywords): """Creates a database. """ # some DB implementations take optional parameter `driver` to use a # specific driver module but it should not be passed to `connect`. keywords.pop("driver", None) self.db_module = db_module self.keywords = keywords self._ctx = threadeddict() # flag to enable/disable printing queries self.printing = config.get("debug_sql", config.get("debug", False)) self.supports_multiple_insert = False try: import DBUtils # noqa, flake8 F401 # enable pooling if DBUtils module is available. self.has_pooling = True except ImportError: self.has_pooling = False # Pooling can be disabled by passing pooling=False in the keywords. self.has_pooling = self.keywords.pop("pooling", True) and self.has_pooling def _getctx(self): if not self._ctx.get("db"): self._load_context(self._ctx) return self._ctx ctx = property(_getctx) def _load_context(self, ctx): ctx.dbq_count = 0 ctx.transactions = [] # stack of transactions if self.has_pooling: ctx.db = self._connect_with_pooling(self.keywords) else: ctx.db = self._connect(self.keywords) ctx.db_execute = self._db_execute if not hasattr(ctx.db, "commit"): ctx.db.commit = lambda: None if not hasattr(ctx.db, "rollback"): ctx.db.rollback = lambda: None def commit(unload=True): # do db commit and release the connection if pooling is enabled. ctx.db.commit() if unload and self.has_pooling: self._unload_context(self._ctx) def rollback(): # do db rollback and release the connection if pooling is enabled. ctx.db.rollback() if self.has_pooling: self._unload_context(self._ctx) ctx.commit = commit ctx.rollback = rollback def _unload_context(self, ctx): del ctx.db def _connect(self, keywords): return self.db_module.connect(**keywords) def _connect_with_pooling(self, keywords): def get_pooled_db(): from DBUtils import PooledDB # In DBUtils 0.9.3, `dbapi` argument is renamed as `creator` # see Bug#122112 if PooledDB.__version__.split(".") < "0.9.3".split("."): return PooledDB.PooledDB(dbapi=self.db_module, **keywords) else: return PooledDB.PooledDB(creator=self.db_module, **keywords) if getattr(self, "_pooleddb", None) is None: self._pooleddb = get_pooled_db() return self._pooleddb.connection() def _db_cursor(self): return self.ctx.db.cursor() def _param_marker(self): """Returns parameter marker based on paramstyle attribute if this database.""" style = getattr(self, "paramstyle", "pyformat") if style == "qmark": return "?" elif style == "numeric": return ":1" elif style in ["format", "pyformat"]: return "%s" raise UnknownParamstyle(style) def _db_execute(self, cur, sql_query): """executes an sql query""" self.ctx.dbq_count += 1 try: a = time.time() query, params = self._process_query(sql_query) out = cur.execute(query, params) b = time.time() except: if self.printing: print("ERR:", str(sql_query), file=debug) if self.ctx.transactions: self.ctx.transactions[-1].rollback() else: self.ctx.rollback() raise if self.printing: print( "%s (%s): %s" % (round(b - a, 2), self.ctx.dbq_count, str(sql_query)), file=debug, ) return out def _process_query(self, sql_query): """Takes the SQLQuery object and returns query string and parameters. """ paramstyle = getattr(self, "paramstyle", "pyformat") query = sql_query.query(paramstyle) params = sql_query.values() return query, params def _where(self, where, vars): if isinstance(where, int): where = "id = " + sqlparam(where) # @@@ for backward-compatibility elif isinstance(where, (list, tuple)) and len(where) == 2: where = SQLQuery(where[0], where[1]) elif isinstance(where, dict): where = self._where_dict(where) elif isinstance(where, SQLQuery): pass else: where = reparam(where, vars) return where def _where_dict(self, where): where_clauses = [] for k, v in sorted(iteritems(where), key=lambda t: t[0]): where_clauses.append(k + " = " + sqlquote(v)) if where_clauses: return SQLQuery.join(where_clauses, " AND ") else: return None def query(self, sql_query, vars=None, processed=False, _test=False): """ Execute SQL query `sql_query` using dictionary `vars` to interpolate it. If `processed=True`, `vars` is a `reparam`-style list to use instead of interpolating. >>> db = DB(None, {}) >>> db.query("SELECT * FROM foo", _test=True) >>> db.query("SELECT * FROM foo WHERE x = $x", vars=dict(x='f'), _test=True) >>> db.query("SELECT * FROM foo WHERE x = " + sqlquote('f'), _test=True) """ if vars is None: vars = {} if not processed and not isinstance(sql_query, SQLQuery): sql_query = reparam(sql_query, vars) if _test: return sql_query db_cursor = self._db_cursor() self._db_execute(db_cursor, sql_query) if db_cursor.description: out = self.create_result_set(db_cursor) else: out = db_cursor.rowcount if not self.ctx.transactions: self.ctx.commit() return out def create_result_set(self, cursor): return ResultSet(cursor) def select( self, tables, vars=None, what="*", where=None, order=None, group=None, limit=None, offset=None, _test=False, ): """ Selects `what` from `tables` with clauses `where`, `order`, `group`, `limit`, and `offset`. Uses vars to interpolate. Otherwise, each clause can be a SQLQuery. >>> db = DB(None, {}) >>> db.select('foo', _test=True) >>> db.select(['foo', 'bar'], where="foo.bar_id = bar.id", limit=5, _test=True) >>> db.select('foo', where={'id': 5}, _test=True) """ if vars is None: vars = {} sql_clauses = self.sql_clauses(what, tables, where, group, order, limit, offset) clauses = [ self.gen_clause(sql, val, vars) for sql, val in sql_clauses if val is not None ] qout = SQLQuery.join(clauses) if _test: return qout return self.query(qout, processed=True) def where( self, table, what="*", order=None, group=None, limit=None, offset=None, _test=False, **kwargs ): """ Selects from `table` where keys are equal to values in `kwargs`. >>> db = DB(None, {}) >>> db.where('foo', bar_id=3, _test=True) >>> db.where('foo', source=2, crust='dewey', _test=True) >>> db.where('foo', _test=True) """ where = self._where_dict(kwargs) return self.select( table, what=what, order=order, group=group, limit=limit, offset=offset, _test=_test, where=where, ) def sql_clauses(self, what, tables, where, group, order, limit, offset): return ( ("SELECT", what), ("FROM", sqllist(tables)), ("WHERE", where), ("GROUP BY", group), ("ORDER BY", order), # The limit and offset could be the values provided by # the end-user and are potentially unsafe. # Using them as parameters to avoid any risk. ("LIMIT", limit and SQLParam(limit).sqlquery()), ("OFFSET", offset and SQLParam(offset).sqlquery()), ) def gen_clause(self, sql, val, vars): if isinstance(val, int): if sql == "WHERE": nout = "id = " + sqlquote(val) else: nout = SQLQuery(val) # @@@ elif isinstance(val, (list, tuple)) and len(val) == 2: nout = SQLQuery(val[0], val[1]) # backwards-compatibility elif sql == "WHERE" and isinstance(val, dict): nout = self._where_dict(val) elif isinstance(val, SQLQuery): nout = val else: nout = reparam(val, vars) def xjoin(a, b): if a and b: return a + " " + b else: return a or b return xjoin(sql, nout) def insert(self, tablename, seqname=None, _test=False, **values): """ Inserts `values` into `tablename`. Returns current sequence ID. Set `seqname` to the ID if it's not the default, or to `False` if there isn't one. >>> db = DB(None, {}) >>> q = db.insert('foo', name='bob', age=2, created=SQLLiteral('NOW()'), _test=True) >>> q >>> q.query() 'INSERT INTO foo (age, created, name) VALUES (%s, NOW(), %s)' >>> q.values() [2, 'bob'] """ def q(x): return "(" + x + ")" if values: # needed for Py3 compatibility with the above doctests sorted_values = sorted(values.items(), key=lambda t: t[0]) _keys = SQLQuery.join(map(lambda t: t[0], sorted_values), ", ") _values = SQLQuery.join( [sqlparam(v) for v in map(lambda t: t[1], sorted_values)], ", " ) sql_query = ( "INSERT INTO %s " % tablename + q(_keys) + " VALUES " + q(_values) ) else: sql_query = SQLQuery(self._get_insert_default_values_query(tablename)) if _test: return sql_query db_cursor = self._db_cursor() if seqname is not False: sql_query = self._process_insert_query(sql_query, tablename, seqname) if isinstance(sql_query, tuple): # for some databases, a separate query has to be made to find # the id of the inserted row. q1, q2 = sql_query self._db_execute(db_cursor, q1) self._db_execute(db_cursor, q2) else: self._db_execute(db_cursor, sql_query) try: out = db_cursor.fetchone()[0] except Exception: out = None if not self.ctx.transactions: self.ctx.commit() return out def _get_insert_default_values_query(self, table): return "INSERT INTO %s DEFAULT VALUES" % table def multiple_insert(self, tablename, values, seqname=None, _test=False): """ Inserts multiple rows into `tablename`. The `values` must be a list of dictionaries, one for each row to be inserted, each with the same set of keys. Returns the list of ids of the inserted rows. Set `seqname` to the ID if it's not the default, or to `False` if there isn't one. >>> db = DB(None, {}) >>> db.supports_multiple_insert = True >>> values = [{"name": "foo", "email": "foo@example.com"}, {"name": "bar", "email": "bar@example.com"}] >>> db.multiple_insert('person', values=values, _test=True) """ if not values: return [] if not self.supports_multiple_insert: out = [ self.insert(tablename, seqname=seqname, _test=_test, **v) for v in values ] if seqname is False: return None else: return out keys = values[0].keys() # @@ make sure all keys are valid for v in values: if v.keys() != keys: raise ValueError("Not all rows have the same keys") # enforce query order for the above doctest compatibility with Py3 keys = sorted(keys) sql_query = SQLQuery( "INSERT INTO %s (%s) VALUES " % (tablename, ", ".join(keys)) ) for i, row in enumerate(values): if i != 0: sql_query.append(", ") SQLQuery.join( [SQLParam(row[k]) for k in keys], sep=", ", target=sql_query, prefix="(", suffix=")", ) if _test: return sql_query db_cursor = self._db_cursor() if seqname is not False: sql_query = self._process_insert_query(sql_query, tablename, seqname) if isinstance(sql_query, tuple): # for some databases, a separate query has to be made to find # the id of the inserted row. q1, q2 = sql_query self._db_execute(db_cursor, q1) self._db_execute(db_cursor, q2) else: self._db_execute(db_cursor, sql_query) try: out = db_cursor.fetchone()[0] # MySQL gives the first id of multiple inserted rows. # PostgreSQL and SQLite give the last id. if self.db_module.__name__ in mysql_drivers: out = range(out, out + len(values)) else: out = range(out - len(values) + 1, out + 1) except Exception: out = None if not self.ctx.transactions: self.ctx.commit() return out def update(self, tables, where, vars=None, _test=False, **values): """ Update `tables` with clause `where` (interpolated using `vars`) and setting `values`. >>> db = DB(None, {}) >>> name = 'Joseph' >>> q = db.update('foo', where='name = $name', name='bob', age=2, ... created=SQLLiteral('NOW()'), vars=locals(), _test=True) >>> q >>> q.query() 'UPDATE foo SET age = %s, created = NOW(), name = %s WHERE name = %s' >>> q.values() [2, 'bob', 'Joseph'] """ if vars is None: vars = {} where = self._where(where, vars) values = sorted(values.items(), key=lambda t: t[0]) query = ( "UPDATE " + sqllist(tables) + " SET " + sqlwhere(values, ", ") + " WHERE " + where ) if _test: return query db_cursor = self._db_cursor() self._db_execute(db_cursor, query) if not self.ctx.transactions: self.ctx.commit() return db_cursor.rowcount def delete(self, table, where, using=None, vars=None, _test=False): """ Deletes from `table` with clauses `where` and `using`. >>> db = DB(None, {}) >>> name = 'Joe' >>> db.delete('foo', where='name = $name', vars=locals(), _test=True) """ if vars is None: vars = {} where = self._where(where, vars) q = "DELETE FROM " + table if using: q += " USING " + sqllist(using) if where: q += " WHERE " + where if _test: return q db_cursor = self._db_cursor() self._db_execute(db_cursor, q) if not self.ctx.transactions: self.ctx.commit() return db_cursor.rowcount def _process_insert_query(self, query, tablename, seqname): return query def transaction(self): """Start a transaction.""" return Transaction(self.ctx) class PostgresDB(DB): """Postgres driver.""" def __init__(self, **keywords): if "pw" in keywords: keywords["password"] = keywords.pop("pw") db_module = import_driver(pg_drivers, preferred=keywords.pop("driver", None)) if db_module.__name__ == "psycopg2": import psycopg2.extensions psycopg2.extensions.register_type(psycopg2.extensions.UNICODE) # if db is not provided `postgres` driver will take it from PGDATABASE # environment variable. if "db" in keywords: keywords["database"] = keywords.pop("db") self.dbname = "postgres" self.paramstyle = db_module.paramstyle DB.__init__(self, db_module, keywords) self.supports_multiple_insert = True self._sequences = None def _process_insert_query(self, query, tablename, seqname): if seqname is None: # when seqname is not provided guess the seqname and make sure it exists seqname = tablename + "_id_seq" if seqname not in self._get_all_sequences(): seqname = None if seqname: query += "; SELECT currval('%s')" % seqname return query def _get_all_sequences(self): """Query postgres to find names of all sequences used in this database.""" if self._sequences is None: q = "SELECT c.relname FROM pg_class c WHERE c.relkind = 'S'" self._sequences = set([c.relname for c in self.query(q)]) return self._sequences def _connect(self, keywords): conn = DB._connect(self, keywords) conn.set_client_encoding("UTF8") return conn def _connect_with_pooling(self, keywords): conn = DB._connect_with_pooling(self, keywords) conn._con._con.set_client_encoding("UTF8") return conn class MySQLDB(DB): def __init__(self, **keywords): db = import_driver(mysql_drivers, preferred=keywords.pop("driver", None)) if db.__name__ == "pymysql": if "pw" in keywords: keywords["password"] = keywords["pw"] del keywords["pw"] elif db.__name__ == "MySQLdb": if "pw" in keywords: keywords["passwd"] = keywords.pop("pw") elif db.__name__ == "mysql.connector": # Enabled buffered so that len can work as expected. keywords.setdefault("buffered", True) if "pw" in keywords: keywords["password"] = keywords["pw"] del keywords["pw"] if "charset" not in keywords: keywords["charset"] = "utf8" elif keywords["charset"] is None: del keywords["charset"] self.paramstyle = db.paramstyle = "pyformat" # it's both self.dbname = "mysql" DB.__init__(self, db, keywords) self.supports_multiple_insert = True def _process_insert_query(self, query, tablename, seqname): return query, SQLQuery("SELECT last_insert_id();") def _get_insert_default_values_query(self, table): return "INSERT INTO %s () VALUES()" % table def import_driver(drivers, preferred=None): """Import the first available driver or preferred driver. """ if preferred: drivers = (preferred,) for d in drivers: try: return __import__(d, None, None, ["x"]) except ImportError: pass raise ImportError("Unable to import " + " or ".join(drivers)) class SqliteDB(DB): def __init__(self, **keywords): db = import_driver(sqlite_drivers, preferred=keywords.pop("driver", None)) if db.__name__ in ["sqlite3", "pysqlite2.dbapi2"]: db.paramstyle = "qmark" # sqlite driver doesn't create datatime objects for timestamp columns # unless `detect_types` option is passed. # It seems to be supported in `sqlite3` and `pysqlite2` drivers, not # surte about `sqlite`. keywords.setdefault("detect_types", db.PARSE_DECLTYPES) self.dbname = "sqlite" self.paramstyle = db.paramstyle keywords["database"] = keywords.pop("db") # sqlite don't allows connections to be shared by threads keywords["pooling"] = False DB.__init__(self, db, keywords) def _process_insert_query(self, query, tablename, seqname): return query, SQLQuery("SELECT last_insert_rowid();") def create_result_set(self, cursor): return SqliteResultSet(cursor) class FirebirdDB(DB): """Firebird Database. """ def __init__(self, **keywords): try: import kinterbasdb as db except Exception: db = None pass if "pw" in keywords: keywords["password"] = keywords.pop("pw") keywords["database"] = keywords.pop("db") self.paramstyle = db.paramstyle DB.__init__(self, db, keywords) def delete(self, table, where=None, using=None, vars=None, _test=False): # firebird doesn't support using clause using = None return DB.delete(self, table, where, using, vars, _test) def sql_clauses(self, what, tables, where, group, order, limit, offset): return ( ("SELECT", ""), ("FIRST", limit), ("SKIP", offset), ("", what), ("FROM", sqllist(tables)), ("WHERE", where), ("GROUP BY", group), ("ORDER BY", order), ) class MSSQLDB(DB): def __init__(self, **keywords): import pymssql as db if "pw" in keywords: keywords["password"] = keywords.pop("pw") keywords["database"] = keywords.pop("db") self.dbname = "mssql" DB.__init__(self, db, keywords) def _process_query(self, sql_query): """Takes the SQLQuery object and returns query string and parameters. """ # MSSQLDB expects params to be a tuple. # Overwriting the default implementation to convert params to tuple. paramstyle = getattr(self, "paramstyle", "pyformat") query = sql_query.query(paramstyle) params = sql_query.values() return query, tuple(params) def sql_clauses(self, what, tables, where, group, order, limit, offset): return ( ("SELECT", what), ("TOP", limit), ("FROM", sqllist(tables)), ("WHERE", where), ("GROUP BY", group), ("ORDER BY", order), ("OFFSET", offset), ) def _test(self): """Test LIMIT. Fake presence of pymssql module for running tests. >>> import sys >>> sys.modules['pymssql'] = sys.modules['sys'] MSSQL has TOP clause instead of LIMIT clause. >>> db = MSSQLDB(db='test', user='joe', pw='secret') >>> db.select('foo', limit=4, _test=True) """ pass class OracleDB(DB): def __init__(self, **keywords): import cx_Oracle as db if "pw" in keywords: keywords["password"] = keywords.pop("pw") # @@ TODO: use db.makedsn if host, port is specified keywords["dsn"] = keywords.pop("db") self.dbname = "oracle" db.paramstyle = "numeric" self.paramstyle = db.paramstyle # oracle doesn't support pooling keywords.pop("pooling", None) DB.__init__(self, db, keywords) def _process_insert_query(self, query, tablename, seqname): if seqname is None: # It is not possible to get seq name from table name in Oracle return query else: return query + "; SELECT %s.currval FROM dual" % seqname def dburl2dict(url): """ Takes a URL to a database and parses it into an equivalent dictionary. >>> dburl2dict('postgres:///mygreatdb') == {'pw': None, 'dbn': 'postgres', 'db': 'mygreatdb', 'host': None, 'user': None, 'port': None} True >>> dburl2dict('postgres://james:day@serverfarm.example.net:5432/mygreatdb') == {'pw': 'day', 'dbn': 'postgres', 'db': 'mygreatdb', 'host': 'serverfarm.example.net', 'user': 'james', 'port': 5432} True >>> dburl2dict('postgres://james:day@serverfarm.example.net/mygreatdb') == {'pw': 'day', 'dbn': 'postgres', 'db': 'mygreatdb', 'host': 'serverfarm.example.net', 'user': 'james', 'port': None} True >>> dburl2dict('postgres://james:d%40y@serverfarm.example.net/mygreatdb') == {'pw': 'd@y', 'dbn': 'postgres', 'db': 'mygreatdb', 'host': 'serverfarm.example.net', 'user': 'james', 'port': None} True >>> dburl2dict('mysql://james:d%40y@serverfarm.example.net/mygreatdb') == {'pw': 'd@y', 'dbn': 'mysql', 'db': 'mygreatdb', 'host': 'serverfarm.example.net', 'user': 'james', 'port': None} True >>> dburl2dict('sqlite:///mygreatdb.db') {'db': 'mygreatdb.db', 'dbn': 'sqlite'} >>> dburl2dict('sqlite:////absolute/path/mygreatdb.db') {'db': '/absolute/path/mygreatdb.db', 'dbn': 'sqlite'} """ parts = urlparse.urlparse(unquote(url)) if parts.scheme == "sqlite": return {"dbn": parts.scheme, "db": parts.path[1:]} else: return { "dbn": parts.scheme, "user": parts.username, "pw": parts.password, "db": parts.path[1:], "host": parts.hostname, "port": parts.port, } _databases = {} def database(dburl=None, **params): """Creates appropriate database using params. Pooling will be enabled if DBUtils module is available. Pooling can be disabled by passing pooling=False in params. """ if not dburl and not params: dburl = os.environ["DATABASE_URL"] if dburl: params = dburl2dict(dburl) dbn = params.pop("dbn") if dbn in _databases: return _databases[dbn](**params) else: raise UnknownDB(dbn) def register_database(name, clazz): """ Register a database. >>> class LegacyDB(DB): ... def __init__(self, **params): ... pass ... >>> register_database('legacy', LegacyDB) >>> db = database(dbn='legacy', db='test', user='joe', passwd='secret') """ _databases[name] = clazz register_database("mysql", MySQLDB) register_database("postgres", PostgresDB) register_database("sqlite", SqliteDB) register_database("firebird", FirebirdDB) register_database("mssql", MSSQLDB) register_database("oracle", OracleDB) def _interpolate(format): """ Takes a format string and returns a list of 2-tuples of the form (boolean, string) where boolean says whether string should be evaled or not. from (public domain, Ka-Ping Yee) """ def matchorfail(text, pos): match = tokenprog.match(text, pos) if match is None: raise _ItplError(text, pos) return match, match.end() namechars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_" chunks = [] pos = 0 while 1: dollar = format.find("$", pos) if dollar < 0: break nextchar = format[dollar + 1] if nextchar == "{": chunks.append((0, format[pos:dollar])) pos, level = dollar + 2, 1 while level: match, pos = matchorfail(format, pos) tstart, tend = match.regs[3] token = format[tstart:tend] if token == "{": level = level + 1 elif token == "}": level = level - 1 chunks.append((1, format[dollar + 2 : pos - 1])) elif nextchar in namechars: chunks.append((0, format[pos:dollar])) match, pos = matchorfail(format, dollar + 1) while pos < len(format): if ( format[pos] == "." and pos + 1 < len(format) and format[pos + 1] in namechars ): match, pos = matchorfail(format, pos + 1) elif format[pos] in "([": pos, level = pos + 1, 1 while level: match, pos = matchorfail(format, pos) tstart, tend = match.regs[3] token = format[tstart:tend] if token[0] in "([": level = level + 1 elif token[0] in ")]": level = level - 1 else: break chunks.append((1, format[dollar + 1 : pos])) else: chunks.append((0, format[pos : dollar + 1])) pos = dollar + 1 + (nextchar == "$") if pos < len(format): chunks.append((0, format[pos:])) return chunks class _Node(object): def __init__(self, type, first, second=None): self.type = type self.first = first self.second = second def __eq__(self, other): return ( isinstance(other, _Node) and self.type == other.type and self.first == other.first and self.second == other.second ) def __repr__(self): return "Node(%r, %r, %r)" % (self.type, self.first, self.second) class Parser: """Parser to parse string templates like "Hello $name". Loosely based on (public domain, Ka-Ping Yee) """ namechars = "abcdefghijklmnopqrstuvwxyz" "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_" def __init__(self): self.reset() def reset(self): self.pos = 0 self.level = 0 self.text = "" def parse(self, text): """Parses the given text and returns a parse tree. """ self.reset() self.text = text return self.parse_all() def parse_all(self): while True: dollar = self.text.find("$", self.pos) if dollar < 0: break nextchar = self.text[dollar + 1] if nextchar in self.namechars: yield _Node("text", self.text[self.pos : dollar]) self.pos = dollar + 1 yield self.parse_expr() # for supporting ${x.id}, for backward compatibility elif nextchar == "{": saved_pos = self.pos self.pos = dollar + 2 # skip "${" expr = self.parse_expr() if self.text[self.pos] == "}": self.pos += 1 yield _Node("text", self.text[self.pos : dollar]) yield expr else: self.pos = saved_pos break else: yield _Node("text", self.text[self.pos : dollar + 1]) self.pos = dollar + 1 # $$ is used to escape $ if nextchar == "$": self.pos += 1 if self.pos < len(self.text): yield _Node("text", self.text[self.pos :]) def match(self): match = tokenprog.match(self.text, self.pos) if match is None: raise _ItplError(self.text, self.pos) return match, match.end() def is_literal(self, text): return text and text[0] in "0123456789\"'" def parse_expr(self): match, pos = self.match() if self.is_literal(match.group()): expr = _Node("literal", match.group()) else: expr = _Node("param", self.text[self.pos : pos]) self.pos = pos while self.pos < len(self.text): if ( self.text[self.pos] == "." and self.pos + 1 < len(self.text) and self.text[self.pos + 1] in self.namechars ): self.pos += 1 match, pos = self.match() attr = match.group() expr = _Node("getattr", expr, attr) self.pos = pos elif self.text[self.pos] == "[": saved_pos = self.pos self.pos += 1 key = self.parse_expr() if self.text[self.pos] == "]": self.pos += 1 expr = _Node("getitem", expr, key) else: self.pos = saved_pos break else: break return expr class SafeEval(object): """Safe evaluator for binding params to db queries. """ def safeeval(self, text, mapping): nodes = Parser().parse(text) return SQLQuery.join([self.eval_node(node, mapping) for node in nodes], "") def eval_node(self, node, mapping): if node.type == "text": return node.first else: return sqlquote(self.eval_expr(node, mapping)) def eval_expr(self, node, mapping): if node.type == "literal": return ast.literal_eval(node.first) elif node.type == "getattr": return getattr(self.eval_expr(node.first, mapping), node.second) elif node.type == "getitem": return self.eval_expr(node.first, mapping)[ self.eval_expr(node.second, mapping) ] elif node.type == "param": return mapping[node.first] def test_parser(): def f(text, expected): p = Parser() nodes = list(p.parse(text)) print(repr(text), nodes) assert nodes == expected, "Expected %r" % expected f("Hello", [_Node("text", "Hello")]) f("Hello $name", [_Node("text", "Hello "), _Node("param", "name")]) f( "Hello $name.foo", [_Node("text", "Hello "), _Node("getattr", _Node("param", "name"), "foo")], ) f( "WHERE id=$self.id LIMIT 1", [ _Node("text", "WHERE id="), _Node("getattr", _Node("param", "self", None), "id"), _Node("text", " LIMIT 1"), ], ) f( "WHERE id=$self['id'] LIMIT 1", [ _Node("text", "WHERE id="), _Node("getitem", _Node("param", "self", None), _Node("literal", "'id'")), _Node("text", " LIMIT 1"), ], ) def test_safeeval(): def f(q, vars): return SafeEval().safeeval(q, vars) print(f("WHERE id=$id", {"id": 1}).items) assert f("WHERE id=$id", {"id": 1}).items == ["WHERE id=", sqlparam(1)] if __name__ == "__main__": import doctest doctest.testmod() test_parser() test_safeeval() webpy-0.61/web/debugerror.py000066400000000000000000000304231370675551300160730ustar00rootroot00000000000000""" pretty debug errors (part of web.py) portions adapted from Django Copyright (c) 2005, the Lawrence Journal-World Used under the modified BSD license: http://www.xfree86.org/3.3.6/COPYRIGHT2.html#5 """ __all__ = ["debugerror", "djangoerror", "emailerrors"] import os import os.path import pprint import sys import traceback from . import webapi as web from .net import websafe from .template import Template from .utils import safestr, sendmail def update_globals_template(t, globals): t.t.__globals__.update(globals) whereami = os.path.join(os.getcwd(), __file__) whereami = os.path.sep.join(whereami.split(os.path.sep)[:-1]) djangoerror_t = """\ $def with (exception_type, exception_value, frames) $exception_type at $ctx.path $def dicttable (d, kls='req', id=None): $ items = d and list(d.items()) or [] $items.sort() $:dicttable_items(items, kls, id) $def dicttable_items(items, kls='req', id=None): $if items: $for k, v in items:
VariableValue
$k
$prettify(v)
$else:

No data.

$exception_type at $ctx.path

$exception_value

Python $frames[0].filename in $frames[0].function, line $frames[0].lineno
Web $ctx.method $ctx.home$ctx.path

Traceback (innermost first)

    $for frame in frames:
  • $frame.filename in $frame.function $if frame.context_line is not None:
    $if frame.pre_context:
      $for line in frame.pre_context:
    1. $line
    1. $frame.context_line ...
    $if frame.post_context:
      $for line in frame.post_context:
    1. $line
    $if frame.vars:
    ▶ Local vars $# $inspect.formatargvalues(*inspect.getargvalues(frame['tb'].tb_frame))
    $:dicttable(frame.vars, kls='vars', id=('v' + str(frame.id)))
$if ctx.output or ctx.headers:

Response so far

HEADERS

$:dicttable_items(ctx.headers)

BODY

$ctx.output

Request information

INPUT

$:dicttable(web.input(_unicode=False)) $:dicttable(web.cookies())

META

$ newctx = [(k, v) for (k, v) in ctx.iteritems() if not k.startswith('_') and not isinstance(v, dict)] $:dicttable(dict(newctx))

ENVIRONMENT

$:dicttable(ctx.env)

You're seeing this error because you have web.config.debug set to True. Set that to False if you don't want to see this.

""" # noqa: W605 djangoerror_r = None def djangoerror(): def _get_lines_from_file(filename, lineno, context_lines): """ Returns context_lines before and after lineno from file. Returns (pre_context_lineno, pre_context, context_line, post_context). """ try: source = open(filename).readlines() lower_bound = max(0, lineno - context_lines) upper_bound = lineno + context_lines pre_context = [line.strip("\n") for line in source[lower_bound:lineno]] context_line = source[lineno].strip("\n") post_context = [ line.strip("\n") for line in source[lineno + 1 : upper_bound] ] return lower_bound, pre_context, context_line, post_context except (OSError, IOError, IndexError): return None, [], None, [] exception_type, exception_value, tback = sys.exc_info() frames = [] while tback is not None: filename = tback.tb_frame.f_code.co_filename function = tback.tb_frame.f_code.co_name lineno = tback.tb_lineno - 1 # hack to get correct line number for templates lineno += tback.tb_frame.f_locals.get("__lineoffset__", 0) ( pre_context_lineno, pre_context, context_line, post_context, ) = _get_lines_from_file(filename, lineno, 7) if "__hidetraceback__" not in tback.tb_frame.f_locals: frames.append( web.storage( { "tback": tback, "filename": filename, "function": function, "lineno": lineno, "vars": tback.tb_frame.f_locals, "id": id(tback), "pre_context": pre_context, "context_line": context_line, "post_context": post_context, "pre_context_lineno": pre_context_lineno, } ) ) tback = tback.tb_next frames.reverse() def prettify(x): try: out = pprint.pformat(x) except Exception as e: out = "[could not display: <" + e.__class__.__name__ + ": " + str(e) + ">]" return out global djangoerror_r if djangoerror_r is None: djangoerror_r = Template(djangoerror_t, filename=__file__, filter=websafe) t = djangoerror_r globals = { "ctx": web.ctx, "web": web, "dict": dict, "str": str, "prettify": prettify, } update_globals_template(t, globals) return t(exception_type, exception_value, frames) def debugerror(): """ A replacement for `internalerror` that presents a nice page with lots of debug information for the programmer. (Based on the beautiful 500 page from [Django](http://djangoproject.com/), designed by [Wilson Miner](http://wilsonminer.com/).) """ return web._InternalError(djangoerror()) def emailerrors(to_address, olderror, from_address=None): """ Wraps the old `internalerror` handler (pass as `olderror`) to additionally email all errors to `to_address`, to aid in debugging production websites. Emails contain a normal text traceback as well as an attachment containing the nice `debugerror` page. """ from_address = from_address or to_address def emailerrors_internal(): error = olderror() tb = sys.exc_info() error_name = tb[0] error_value = tb[1] tb_txt = "".join(traceback.format_exception(*tb)) path = web.ctx.path request = web.ctx.method + " " + web.ctx.home + web.ctx.fullpath message = "\n%s\n\n%s\n\n" % (request, tb_txt) sendmail( "your buggy site <%s>" % from_address, "the bugfixer <%s>" % to_address, "bug: %(error_name)s: %(error_value)s (%(path)s)" % locals(), message, attachments=[dict(filename="bug.html", content=safestr(djangoerror()))], ) return error return emailerrors_internal if __name__ == "__main__": urls = ("/", "index") from .application import application app = application(urls, globals()) app.internalerror = debugerror class index: def GET(self): thisdoesnotexist # noqa: F821 app.run() webpy-0.61/web/form.py000066400000000000000000000514071370675551300147030ustar00rootroot00000000000000""" HTML forms (part of web.py) """ import copy import re from . import net, utils from . import webapi as web def attrget(obj, attr, value=None): try: if hasattr(obj, "has_key") and attr in obj: return obj[attr] except TypeError: # Handle the case where has_key takes different number of arguments. # This is the case with Model objects on appengine. See #134 pass if ( hasattr(obj, "keys") and attr in obj ): # needed for Py3, has_key doesn't exist anymore return obj[attr] elif hasattr(obj, attr): return getattr(obj, attr) return value class Form(object): r""" HTML form. >>> f = Form(Textbox("x")) >>> f.render() u'\n \n
' >>> f.fill(x="42") True >>> f.render() u'\n \n
' """ def __init__(self, *inputs, **kw): self.inputs = inputs self.valid = True self.note = None self.validators = kw.pop("validators", []) def __call__(self, x=None): o = copy.deepcopy(self) if x: o.validates(x) return o def render(self): out = "" out += self.rendernote(self.note) out += "\n" for i in self.inputs: html = ( utils.safeunicode(i.pre) + i.render() + self.rendernote(i.note) + utils.safeunicode(i.post) ) if i.is_hidden(): out += ' \n' % ( html ) else: out += ( ' \n' % (net.websafe(i.id), net.websafe(i.description), html) ) out += "
%s
%s
" return out def render_css(self): out = [] out.append(self.rendernote(self.note)) for i in self.inputs: if not i.is_hidden(): out.append( '' % (net.websafe(i.id), net.websafe(i.description)) ) out.append(i.pre) out.append(i.render()) out.append(self.rendernote(i.note)) out.append(i.post) out.append("\n") return "".join(out) def rendernote(self, note): if note: return '%s' % net.websafe(note) else: return "" def validates(self, source=None, _validate=True, **kw): source = source or kw or web.input() out = True for i in self.inputs: v = attrget(source, i.name) if _validate: out = i.validate(v) and out else: i.set_value(v) if _validate: out = out and self._validate(source) self.valid = out return out def _validate(self, value): self.value = value for v in self.validators: if not v.valid(value): self.note = v.msg return False return True def fill(self, source=None, **kw): return self.validates(source, _validate=False, **kw) def __getitem__(self, i): for x in self.inputs: if x.name == i: return x raise KeyError(i) def __getattr__(self, name): # don't interfere with deepcopy inputs = self.__dict__.get("inputs") or [] for x in inputs: if x.name == name: return x raise AttributeError(name) def get(self, i, default=None): try: return self[i] except KeyError: return default def _get_d(self): # @@ should really be form.attr, no? return utils.storage([(i.name, i.get_value()) for i in self.inputs]) d = property(_get_d) class Input(object): """Generic input. Type attribute must be specified when called directly. See also: Currently only types which can be written inside one `` tag are supported. - For checkbox, please use `Checkbox` class for better control. - For radiobox, please use `Radio` class for better control. >>> Input(name='foo', type='email', value="user@domain.com").render() u'' >>> Input(name='foo', type='number', value="bar").render() u'' >>> Input(name='num', type="number", min='0', max='10', step='2', value='5').render() u'' >>> Input(name='foo', type="tel", value='55512345').render() u'' >>> Input(name='search', type="search", value='Search').render() u'' >>> Input(name='search', type="search", value='Search', required='required', pattern='[a-z0-9]{2,30}', placeholder='Search...').render() u'' >>> Input(name='url', type="url", value='url').render() u'' >>> Input(name='range', type="range", min='0', max='10', step='2', value='5').render() u'' >>> Input(name='color', type="color").render() u'' >>> Input(name='f', type="file", accept=".doc,.docx,.xml").render() u'' """ def __init__(self, name, *validators, **attrs): self.name = name self.validators = validators self.attrs = attrs = AttributeList(attrs) self.type = attrs.pop("type", None) self.description = attrs.pop("description", name) self.value = attrs.pop("value", None) self.pre = attrs.pop("pre", "") self.post = attrs.pop("post", "") self.note = None self.id = attrs.setdefault("id", self.get_default_id()) if "class_" in attrs: attrs["class"] = attrs["class_"] del attrs["class_"] def is_hidden(self): return False def get_type(self): if self.type is not None: return self.type else: raise AttributeError("missing attribute 'type'") def get_default_id(self): return self.name def validate(self, value): self.set_value(value) for v in self.validators: if not v.valid(value): self.note = v.msg return False return True def set_value(self, value): self.value = value def get_value(self): return self.value def render(self): attrs = self.attrs.copy() attrs["type"] = self.get_type() if self.value is not None: attrs["value"] = self.value attrs["name"] = self.name attrs["id"] = self.id return "" % attrs def rendernote(self, note): if note: return '%s' % net.websafe(note) else: return "" def addatts(self): # add leading space for backward-compatibility return " " + str(self.attrs) class AttributeList(dict): """List of attributes of input. >>> a = AttributeList(type='text', name='x', value=20) >>> a """ def copy(self): return AttributeList(self) def __str__(self): return " ".join( ['%s="%s"' % (k, net.websafe(v)) for k, v in sorted(self.items())] ) def __repr__(self): return "" % repr(str(self)) class Textbox(Input): """Textbox input. >>> Textbox(name='foo', value='bar').render() u'' >>> Textbox(name='foo', value=0).render() u'' """ def get_type(self): return "text" class Password(Input): """Password input. >>> Password(name='password', value='secret').render() u'' """ def get_type(self): return "password" class Textarea(Input): """Textarea input. >>> Textarea(name='foo', value='bar').render() u'' """ def render(self): attrs = self.attrs.copy() attrs["name"] = self.name value = net.websafe(self.value or "") return "" % (attrs, value) class Dropdown(Input): r"""Dropdown/select input. >>> Dropdown(name='foo', args=['a', 'b', 'c'], value='b').render() u'\n' >>> Dropdown(name='foo', args=[('a', 'aa'), ('b', 'bb'), ('c', 'cc')], value='b').render() u'\n' """ def __init__(self, name, args, *validators, **attrs): self.args = args super(Dropdown, self).__init__(name, *validators, **attrs) def render(self): attrs = self.attrs.copy() attrs["name"] = self.name x = "\n" return x def _render_option(self, arg, indent=" "): if isinstance(arg, (tuple, list)): value, desc = arg else: value, desc = arg, arg value = utils.safestr(value) if isinstance(self.value, (tuple, list)): s_value = [utils.safestr(x) for x in self.value] else: s_value = utils.safestr(self.value) if s_value == value or (isinstance(s_value, list) and value in s_value): select_p = ' selected="selected"' else: select_p = "" return indent + '%s\n' % ( select_p, net.websafe(value), net.websafe(desc), ) class GroupedDropdown(Dropdown): r"""Grouped Dropdown/select input. >>> GroupedDropdown(name='car_type', args=(('Swedish Cars', ('Volvo', 'Saab')), ('German Cars', ('Mercedes', 'Audi'))), value='Audi').render() u'\n' >>> GroupedDropdown(name='car_type', args=(('Swedish Cars', (('v', 'Volvo'), ('s', 'Saab'))), ('German Cars', (('m', 'Mercedes'), ('a', 'Audi')))), value='a').render() u'\n' """ def __init__(self, name, args, *validators, **attrs): self.args = args super(Dropdown, self).__init__(name, *validators, **attrs) def render(self): attrs = self.attrs.copy() attrs["name"] = self.name x = "\n" return x class Radio(Input): def __init__(self, name, args, *validators, **attrs): self.args = args super(Radio, self).__init__(name, *validators, **attrs) def render(self): x = "" for idx, arg in enumerate(self.args, start=1): if isinstance(arg, (tuple, list)): value, desc = arg else: value, desc = arg, arg attrs = self.attrs.copy() attrs["name"] = self.name attrs["type"] = "radio" attrs["value"] = value attrs["id"] = self.name + str(idx) if self.value == value: attrs["checked"] = "checked" x += " %s" % (attrs, net.websafe(desc)) x += "" return x class Checkbox(Input): """Checkbox input. >>> Checkbox('foo', value='bar', checked=True).render() u'' >>> Checkbox('foo', value='bar').render() u'' >>> c = Checkbox('foo', value='bar') >>> c.validate('on') True >>> c.render() u'' """ def __init__(self, name, *validators, **attrs): self.checked = attrs.pop("checked", False) Input.__init__(self, name, *validators, **attrs) def get_default_id(self): value = utils.safestr(self.value or "") return self.name + "_" + value.replace(" ", "_") def render(self): attrs = self.attrs.copy() attrs["type"] = "checkbox" attrs["name"] = self.name attrs["value"] = self.value if self.checked: attrs["checked"] = "checked" return "" % attrs def set_value(self, value): self.checked = bool(value) def get_value(self): return self.checked class Button(Input): """HTML Button. >>> Button("save").render() u'' >>> Button("action", value="save", html="Save Changes").render() u'' """ def __init__(self, name, *validators, **attrs): super(Button, self).__init__(name, *validators, **attrs) self.description = "" def render(self): attrs = self.attrs.copy() attrs["name"] = self.name if self.value is not None: attrs["value"] = self.value html = attrs.pop("html", None) or net.websafe(self.name) return "" % (attrs, html) class Hidden(Input): """Hidden Input. >>> Hidden(name='foo', value='bar').render() u'' """ def is_hidden(self): return True def get_type(self): return "hidden" class File(Input): """File input. >>> File(name='f', accept=".doc,.docx,.xml").render() u'' """ def get_type(self): return "file" class Telephone(Input): """Telephone input. See: >>> Telephone(name='tel', value='55512345').render() u'' """ def get_type(self): return "tel" class Email(Input): """Email input. See: >>> Email(name='email', value='me@example.org').render() u'' """ def get_type(self): return "email" class Date(Input): """Date input. Note: Not supported by desktop Safari, Internet Explorer, or Opera Mini See: >>> Date(name='date', value='2020-04-01').render() u'' """ def get_type(self): return "date" class Time(Input): """Time input. Note: Not supported by desktop Safari, Internet Explorer, or Opera Mini See: >>> Time(name='time', value='07:00').render() u'' """ def get_type(self): return "time" class Search(Input): """Search input. See: >> Search(name='search', value='Search').render() u'' >>> Search(name='search', value='Search', required='required', pattern='[a-z0-9]{2,30}', placeholder='Search...').render() u'' """ def get_type(self): return "search" class Url(Input): """URL input. See: >>> Url(name='url', value='url').render() u'' """ def get_type(self): return "url" class Number(Input): """Number input. See: >>> Number(name='num', min='0', max='10', step='2', value='5').render() u'' """ def get_type(self): return "number" class Range(Input): """Range input. See: >>> Range(name='range', min='0', max='10', step='2', value='5').render() u'' """ def get_type(self): return "range" class Color(Input): """Color input. Note: Not supported by Internet Explorer or Opera Mini See: >>> Color(name='color').render() u'' """ def get_type(self): return "color" class Datalist(Input): """Datalist input. This is currently supported by Chrome, Firefox, Edge, and Opera. It is not supported on Safari or Internet Explorer. Use it with caution. Datalist cannot be used separately. It must be bound to an input. >>> Datalist(name='list', args=[('a', 'b'), ('c', 'd')]).render() u'' >>> Datalist(name='list', args=[['a', 'b'], ['c', 'd']]).render() u'' >>> Datalist(name='list', args=['a', 'b', 'c', 'd']).render() u'' """ def __init__(self, name, args, *validators, **kwargs): self.args = args super(Datalist, self).__init__(name, *validators, **kwargs) def render(self): attrs = self.attrs.copy() attrs["name"] = self.name label_p = "" x = "" % attrs for arg in self.args: if isinstance(arg, (tuple, list)): label_p = ' label="%s"' % net.websafe(arg[0]) label = net.websafe(arg[1]) else: label = net.websafe(arg) x += '' % (label_p, label) x += "" return x class Validator: def __deepcopy__(self, memo): return copy.copy(self) def __init__(self, msg, test, jstest=None): utils.autoassign(self, locals()) def valid(self, value): try: return self.test(value) except: return False notnull = Validator("Required", bool) class regexp(Validator): def __init__(self, rexp, msg): self.rexp = re.compile(rexp) self.msg = msg def valid(self, value): return bool(self.rexp.match(value)) if __name__ == "__main__": import doctest doctest.testmod() webpy-0.61/web/http.py000066400000000000000000000107211370675551300147110ustar00rootroot00000000000000""" HTTP Utilities (from web.py) """ __all__ = [ "expires", "lastmodified", "prefixurl", "modified", "changequery", "url", "profiler", ] import datetime from . import net, utils from . import webapi as web from .py3helpers import iteritems try: from urllib.parse import urlencode as urllib_urlencode except ImportError: from urllib import urlencode as urllib_urlencode try: xrange # Python 2 except NameError: xrange = range # Python 3 def prefixurl(base=""): """ Sorry, this function is really difficult to explain. Maybe some other time. """ url = web.ctx.path.lstrip("/") for i in xrange(url.count("/")): base += "../" if not base: base = "./" return base def expires(delta): """ Outputs an `Expires` header for `delta` from now. `delta` is a `timedelta` object or a number of seconds. """ if isinstance(delta, int): delta = datetime.timedelta(seconds=delta) date_obj = datetime.datetime.utcnow() + delta web.header("Expires", net.httpdate(date_obj)) def lastmodified(date_obj): """Outputs a `Last-Modified` header for `datetime`.""" web.header("Last-Modified", net.httpdate(date_obj)) def modified(date=None, etag=None): """ Checks to see if the page has been modified since the version in the requester's cache. When you publish pages, you can include `Last-Modified` and `ETag` with the date the page was last modified and an opaque token for the particular version, respectively. When readers reload the page, the browser sends along the modification date and etag value for the version it has in its cache. If the page hasn't changed, the server can just return `304 Not Modified` and not have to send the whole page again. This function takes the last-modified date `date` and the ETag `etag` and checks the headers to see if they match. If they do, it returns `True`, or otherwise it raises NotModified error. It also sets `Last-Modified` and `ETag` output headers. """ n = set( [x.strip('" ') for x in web.ctx.env.get("HTTP_IF_NONE_MATCH", "").split(",")] ) m = net.parsehttpdate(web.ctx.env.get("HTTP_IF_MODIFIED_SINCE", "").split(";")[0]) validate = False if etag: if "*" in n or etag in n: validate = True if date and m: # we subtract a second because # HTTP dates don't have sub-second precision if date - datetime.timedelta(seconds=1) <= m: validate = True if date: lastmodified(date) if etag: web.header("ETag", '"' + etag + '"') if validate: raise web.notmodified() else: return True def urlencode(query, doseq=0): """ Same as urllib.urlencode, but supports unicode strings. >>> urlencode({'text':'foo bar'}) 'text=foo+bar' >>> urlencode({'x': [1, 2]}, doseq=True) 'x=1&x=2' """ def convert(value, doseq=False): if doseq and isinstance(value, list): return [convert(v) for v in value] else: return utils.safestr(value) query = dict([(k, convert(v, doseq)) for k, v in query.items()]) return urllib_urlencode(query, doseq=doseq) def changequery(query=None, **kw): """ Imagine you're at `/foo?a=1&b=2`. Then `changequery(a=3)` will return `/foo?a=3&b=2` -- the same URL but with the arguments you requested changed. """ if query is None: query = web.rawinput(method="get") for k, v in iteritems(kw): if v is None: query.pop(k, None) else: query[k] = v out = web.ctx.path if query: out += "?" + urlencode(query, doseq=True) return out def url(path=None, doseq=False, **kw): """ Makes url by concatenating web.ctx.homepath and path and the query string created using the arguments. """ if path is None: path = web.ctx.path if path.startswith("/"): out = web.ctx.homepath + path else: out = path if kw: out += "?" + urlencode(kw, doseq=doseq) return out def profiler(app): """Outputs basic profiling information at the bottom of each response.""" from utils import profile def profile_internal(e, o): out, result = profile(app)(e, o) return list(out) + ["
" + net.websafe(result) + "
"] return profile_internal if __name__ == "__main__": import doctest doctest.testmod() webpy-0.61/web/httpserver.py000066400000000000000000000241451370675551300161450ustar00rootroot00000000000000import os import posixpath import sys from . import utils from . import webapi as web try: from io import BytesIO except ImportError: from StringIO import BytesIO try: from http.server import HTTPServer, SimpleHTTPRequestHandler, BaseHTTPRequestHandler except ImportError: from SimpleHTTPServer import SimpleHTTPRequestHandler from BaseHTTPServer import HTTPServer, BaseHTTPRequestHandler try: from urllib import parse as urlparse from urllib.parse import unquote except ImportError: import urlparse from urllib import unquote __all__ = ["runsimple"] def runbasic(func, server_address=("0.0.0.0", 8080)): """ Runs a simple HTTP server hosting WSGI app `func`. The directory `static/` is hosted statically. Based on [WsgiServer][ws] from [Colin Stewart][cs]. [ws]: http://www.owlfish.com/software/wsgiutils/documentation/wsgi-server-api.html [cs]: http://www.owlfish.com/ """ # Copyright (c) 2004 Colin Stewart (http://www.owlfish.com/) # Modified somewhat for simplicity # Used under the modified BSD license: # http://www.xfree86.org/3.3.6/COPYRIGHT2.html#5 import SocketServer import socket import errno import traceback class WSGIHandler(SimpleHTTPRequestHandler): def run_wsgi_app(self): protocol, host, path, parameters, query, fragment = urlparse.urlparse( "http://dummyhost%s" % self.path ) # we only use path, query env = { "wsgi.version": (1, 0), "wsgi.url_scheme": "http", "wsgi.input": self.rfile, "wsgi.errors": sys.stderr, "wsgi.multithread": 1, "wsgi.multiprocess": 0, "wsgi.run_once": 0, "REQUEST_METHOD": self.command, "REQUEST_URI": self.path, "PATH_INFO": path, "QUERY_STRING": query, "CONTENT_TYPE": self.headers.get("Content-Type", ""), "CONTENT_LENGTH": self.headers.get("Content-Length", ""), "REMOTE_ADDR": self.client_address[0], "SERVER_NAME": self.server.server_address[0], "SERVER_PORT": str(self.server.server_address[1]), "SERVER_PROTOCOL": self.request_version, } for http_header, http_value in self.headers.items(): env["HTTP_%s" % http_header.replace("-", "_").upper()] = http_value # Setup the state self.wsgi_sent_headers = 0 self.wsgi_headers = [] try: # We have there environment, now invoke the application result = self.server.app(env, self.wsgi_start_response) try: try: for data in result: if data: self.wsgi_write_data(data) finally: if hasattr(result, "close"): result.close() except socket.error as socket_err: # Catch common network errors and suppress them if socket_err.args[0] in (errno.ECONNABORTED, errno.EPIPE): return except socket.timeout: return except: print(traceback.format_exc(), file=web.debug) if not self.wsgi_sent_headers: # We must write out something! self.wsgi_write_data(" ") return do_POST = run_wsgi_app do_PUT = run_wsgi_app do_DELETE = run_wsgi_app def do_GET(self): if self.path.startswith("/static/"): SimpleHTTPRequestHandler.do_GET(self) else: self.run_wsgi_app() def wsgi_start_response(self, response_status, response_headers, exc_info=None): if self.wsgi_sent_headers: raise Exception("Headers already sent and start_response called again!") # Should really take a copy to avoid changes in the application.... self.wsgi_headers = (response_status, response_headers) return self.wsgi_write_data def wsgi_write_data(self, data): if not self.wsgi_sent_headers: status, headers = self.wsgi_headers # Need to send header prior to data status_code = status[: status.find(" ")] status_msg = status[status.find(" ") + 1 :] self.send_response(int(status_code), status_msg) for header, value in headers: self.send_header(header, value) self.end_headers() self.wsgi_sent_headers = 1 # Send the data self.wfile.write(data) class WSGIServer(SocketServer.ThreadingMixIn, HTTPServer): def __init__(self, func, server_address): HTTPServer.HTTPServer.__init__(self, server_address, WSGIHandler) self.app = func self.serverShuttingDown = 0 print("http://%s:%d/" % server_address) WSGIServer(func, server_address).serve_forever() # The WSGIServer instance. # Made global so that it can be stopped in embedded mode. server = None def runsimple(func, server_address=("0.0.0.0", 8080)): """ Runs [CherryPy][cp] WSGI server hosting WSGI app `func`. The directory `static/` is hosted statically. [cp]: http://www.cherrypy.org """ global server func = StaticMiddleware(func) func = LogMiddleware(func) server = WSGIServer(server_address, func) if "/" in server_address[0]: print("%s" % server_address) else: if server.ssl_adapter: print("https://%s:%d/" % server_address) else: print("http://%s:%d/" % server_address) try: server.start() except (KeyboardInterrupt, SystemExit): server.stop() server = None def WSGIServer(server_address, wsgi_app): """Creates CherryPy WSGI server listening at `server_address` to serve `wsgi_app`. This function can be overwritten to customize the webserver or use a different webserver. """ from cheroot import wsgi server = wsgi.Server(server_address, wsgi_app, server_name="localhost") server.nodelay = not sys.platform.startswith( "java" ) # TCP_NODELAY isn't supported on the JVM return server class StaticApp(SimpleHTTPRequestHandler): """WSGI application for serving static files.""" def __init__(self, environ, start_response): self.headers = [] self.environ = environ self.start_response = start_response self.directory = os.getcwd() def send_response(self, status, msg=""): # the int(status) call is needed because in Py3 status is an enum.IntEnum and we need the integer behind self.status = str(int(status)) + " " + msg def send_header(self, name, value): self.headers.append((name, value)) def end_headers(self): pass def log_message(*a): pass def __iter__(self): environ = self.environ self.path = environ.get("PATH_INFO", "") self.client_address = ( environ.get("REMOTE_ADDR", "-"), environ.get("REMOTE_PORT", "-"), ) self.command = environ.get("REQUEST_METHOD", "-") self.wfile = BytesIO() # for capturing error try: path = self.translate_path(self.path) etag = '"%s"' % os.path.getmtime(path) client_etag = environ.get("HTTP_IF_NONE_MATCH") self.send_header("ETag", etag) if etag == client_etag: self.send_response(304, "Not Modified") self.start_response(self.status, self.headers) return except OSError: pass # Probably a 404 f = self.send_head() self.start_response(self.status, self.headers) if f: block_size = 16 * 1024 while True: buf = f.read(block_size) if not buf: break yield buf f.close() else: value = self.wfile.getvalue() yield value class StaticMiddleware: """WSGI middleware for serving static files.""" def __init__(self, app, prefix="/static/"): self.app = app self.prefix = prefix def __call__(self, environ, start_response): path = environ.get("PATH_INFO", "") path = self.normpath(path) if path.startswith(self.prefix): return StaticApp(environ, start_response) else: return self.app(environ, start_response) def normpath(self, path): path2 = posixpath.normpath(unquote(path)) if path.endswith("/"): path2 += "/" return path2 class LogMiddleware: """WSGI middleware for logging the status.""" def __init__(self, app): self.app = app self.format = '%s - - [%s] "%s %s %s" - %s' f = BytesIO() class FakeSocket: def makefile(self, *a): return f # take log_date_time_string method from BaseHTTPRequestHandler self.log_date_time_string = BaseHTTPRequestHandler( FakeSocket(), None, None ).log_date_time_string def __call__(self, environ, start_response): def xstart_response(status, response_headers, *args): out = start_response(status, response_headers, *args) self.log(status, environ) return out return self.app(environ, xstart_response) def log(self, status, environ): outfile = environ.get("wsgi.errors", web.debug) req = environ.get("PATH_INFO", "_") protocol = environ.get("ACTUAL_SERVER_PROTOCOL", "-") method = environ.get("REQUEST_METHOD", "-") host = "%s:%s" % ( environ.get("REMOTE_ADDR", "-"), environ.get("REMOTE_PORT", "-"), ) time = self.log_date_time_string() msg = self.format % (host, time, protocol, method, req, status) print(utils.safestr(msg), file=outfile) webpy-0.61/web/net.py000066400000000000000000000143601370675551300145230ustar00rootroot00000000000000""" Network Utilities (from web.py) """ import datetime import re import socket import time try: from urllib.parse import quote except ImportError: from urllib import quote __all__ = [ "validipaddr", "validip6addr", "validipport", "validip", "validaddr", "urlquote", "httpdate", "parsehttpdate", "htmlquote", "htmlunquote", "websafe", ] def validip6addr(address): """ Returns True if `address` is a valid IPv6 address. >>> validip6addr('::') True >>> validip6addr('aaaa:bbbb:cccc:dddd::1') True >>> validip6addr('1:2:3:4:5:6:7:8:9:10') False >>> validip6addr('12:10') False """ try: socket.inet_pton(socket.AF_INET6, address) except (socket.error, AttributeError, ValueError): return False return True def validipaddr(address): """ Returns True if `address` is a valid IPv4 address. >>> validipaddr('192.168.1.1') True >>> validipaddr('192.168. 1.1') False >>> validipaddr('192.168.1.800') False >>> validipaddr('192.168.1') False """ try: octets = address.split(".") if len(octets) != 4: return False for x in octets: if " " in x: return False if not (0 <= int(x) <= 255): return False except ValueError: return False return True def validipport(port): """ Returns True if `port` is a valid IPv4 port. >>> validipport('9000') True >>> validipport('foo') False >>> validipport('1000000') False """ try: if not (0 <= int(port) <= 65535): return False except ValueError: return False return True def validip(ip, defaultaddr="0.0.0.0", defaultport=8080): """ Returns `(ip_address, port)` from string `ip_addr_port` >>> validip('1.2.3.4') ('1.2.3.4', 8080) >>> validip('80') ('0.0.0.0', 80) >>> validip('192.168.0.1:85') ('192.168.0.1', 85) >>> validip('::') ('::', 8080) >>> validip('[::]:88') ('::', 88) >>> validip('[::1]:80') ('::1', 80) """ addr = defaultaddr port = defaultport # Matt Boswell's code to check for ipv6 first match = re.search(r"^\[([^]]+)\](?::(\d+))?$", ip) # check for [ipv6]:port if match: if validip6addr(match.group(1)): if match.group(2): if validipport(match.group(2)): return (match.group(1), int(match.group(2))) else: return (match.group(1), port) else: if validip6addr(ip): return (ip, port) # end ipv6 code ip = ip.split(":", 1) if len(ip) == 1: if not ip[0]: pass elif validipaddr(ip[0]): addr = ip[0] elif validipport(ip[0]): port = int(ip[0]) else: raise ValueError(":".join(ip) + " is not a valid IP address/port") elif len(ip) == 2: addr, port = ip if not validipaddr(addr) or not validipport(port): raise ValueError(":".join(ip) + " is not a valid IP address/port") port = int(port) else: raise ValueError(":".join(ip) + " is not a valid IP address/port") return (addr, port) def validaddr(string_): """ Returns either (ip_address, port) or "/path/to/socket" from string_ >>> validaddr('/path/to/socket') '/path/to/socket' >>> validaddr('8000') ('0.0.0.0', 8000) >>> validaddr('127.0.0.1') ('127.0.0.1', 8080) >>> validaddr('127.0.0.1:8000') ('127.0.0.1', 8000) >>> validip('[::1]:80') ('::1', 80) >>> validaddr('fff') Traceback (most recent call last): ... ValueError: fff is not a valid IP address/port """ if "/" in string_: return string_ else: return validip(string_) def urlquote(val): """ Quotes a string for use in a URL. >>> urlquote('://?f=1&j=1') '%3A//%3Ff%3D1%26j%3D1' >>> urlquote(None) '' >>> urlquote(u'\u203d') '%E2%80%BD' """ if val is None: return "" val = str(val).encode("utf-8") return quote(val) def httpdate(date_obj): """ Formats a datetime object for use in HTTP headers. >>> import datetime >>> httpdate(datetime.datetime(1970, 1, 1, 1, 1, 1)) 'Thu, 01 Jan 1970 01:01:01 GMT' """ return date_obj.strftime("%a, %d %b %Y %H:%M:%S GMT") def parsehttpdate(string_): """ Parses an HTTP date into a datetime object. >>> parsehttpdate('Thu, 01 Jan 1970 01:01:01 GMT') datetime.datetime(1970, 1, 1, 1, 1, 1) """ try: t = time.strptime(string_, "%a, %d %b %Y %H:%M:%S %Z") except ValueError: return None return datetime.datetime(*t[:6]) def htmlquote(text): r""" Encodes `text` for raw use in HTML. >>> htmlquote(u"<'&\">") u'<'&">' """ text = text.replace(u"&", u"&") # Must be done first! text = text.replace(u"<", u"<") text = text.replace(u">", u">") text = text.replace(u"'", u"'") text = text.replace(u'"', u""") return text def htmlunquote(text): r""" Decodes `text` that's HTML quoted. >>> htmlunquote(u'<'&">') u'<\'&">' """ text = text.replace(u""", u'"') text = text.replace(u"'", u"'") text = text.replace(u">", u">") text = text.replace(u"<", u"<") text = text.replace(u"&", u"&") # Must be done last! return text def websafe(val): r""" Converts `val` so that it is safe for use in Unicode HTML. >>> websafe("<'&\">") u'<'&">' >>> websafe(None) u'' >>> websafe(u'\u203d') == u'\u203d' True """ if val is None: return u"" if isinstance(val, bytes): val = val.decode("utf-8") elif not isinstance(val, str): val = str(val) return htmlquote(val) if __name__ == "__main__": import doctest doctest.testmod() webpy-0.61/web/py3helpers.py000066400000000000000000000003161370675551300160270ustar00rootroot00000000000000"""Utilities for make the code run both on Python2 and Python3. """ # Dictionary iteration iterkeys = lambda d: iter(d.keys()) itervalues = lambda d: iter(d.values()) iteritems = lambda d: iter(d.items()) webpy-0.61/web/session.py000066400000000000000000000312671370675551300154250ustar00rootroot00000000000000""" Session Management (from web.py) """ import datetime import os import os.path import shutil import threading import time from copy import deepcopy from hashlib import sha1 from . import utils from . import webapi as web from .py3helpers import iteritems try: import cPickle as pickle except ImportError: import pickle from base64 import encodebytes, decodebytes __all__ = ["Session", "SessionExpired", "Store", "DiskStore", "DBStore"] web.config.session_parameters = utils.storage( { "cookie_name": "webpy_session_id", "cookie_domain": None, "cookie_path": None, "samesite": None, "timeout": 86400, # 24 * 60 * 60, # 24 hours in seconds "ignore_expiry": True, "ignore_change_ip": True, "secret_key": "fLjUfxqXtfNoIldA0A0J", "expired_message": "Session expired", "httponly": True, "secure": False, } ) class SessionExpired(web.HTTPError): def __init__(self, message): web.HTTPError.__init__(self, "200 OK", {}, data=message) class Session(object): """Session management for web.py """ __slots__ = [ "store", "_initializer", "_last_cleanup_time", "_config", "_data", "__getitem__", "__setitem__", "__delitem__", ] def __init__(self, app, store, initializer=None): self.store = store self._initializer = initializer self._last_cleanup_time = 0 self._config = utils.storage(web.config.session_parameters) self._data = utils.threadeddict() self.__getitem__ = self._data.__getitem__ self.__setitem__ = self._data.__setitem__ self.__delitem__ = self._data.__delitem__ if app: app.add_processor(self._processor) def __contains__(self, name): return name in self._data def __getattr__(self, name): return getattr(self._data, name) def __setattr__(self, name, value): if name in self.__slots__: object.__setattr__(self, name, value) else: setattr(self._data, name, value) def __delattr__(self, name): delattr(self._data, name) def _processor(self, handler): """Application processor to setup session for every request""" self._cleanup() self._load() try: return handler() finally: self._save() def _load(self): """Load the session from the store, by the id from cookie""" cookie_name = self._config.cookie_name self.session_id = web.cookies().get(cookie_name) # protection against session_id tampering if self.session_id and not self._valid_session_id(self.session_id): self.session_id = None self._check_expiry() if self.session_id: d = self.store[self.session_id] self.update(d) self._validate_ip() if not self.session_id: self.session_id = self._generate_session_id() if self._initializer: if isinstance(self._initializer, dict): self.update(deepcopy(self._initializer)) elif hasattr(self._initializer, "__call__"): self._initializer() self.ip = web.ctx.ip def _check_expiry(self): # check for expiry if self.session_id and self.session_id not in self.store: if self._config.ignore_expiry: self.session_id = None else: return self.expired() def _validate_ip(self): # check for change of IP if self.session_id and self.get("ip", None) != web.ctx.ip: if not self._config.ignore_change_ip: return self.expired() def _save(self): current_values = dict(self._data) del current_values["session_id"] del current_values["ip"] if not self.get("_killed") and current_values != self._initializer: self._setcookie(self.session_id) self.store[self.session_id] = dict(self._data) else: if web.cookies().get(self._config.cookie_name): self._setcookie( self.session_id, expires=self._config.timeout, samesite=self._config.get("samesite"), ) def _setcookie(self, session_id, expires="", **kw): cookie_name = self._config.cookie_name cookie_domain = self._config.cookie_domain cookie_path = self._config.cookie_path httponly = self._config.httponly secure = self._config.secure samesite = kw.get("samesite", self._config.get("samesite", None)) web.setcookie( cookie_name, session_id, expires=expires or self._config.timeout, domain=cookie_domain, httponly=httponly, secure=secure, path=cookie_path, samesite=samesite, ) def _generate_session_id(self): """Generate a random id for session""" while True: rand = os.urandom(16) now = time.time() secret_key = self._config.secret_key hashable = "%s%s%s%s" % (rand, now, utils.safestr(web.ctx.ip), secret_key) session_id = sha1(hashable.encode("utf-8")).hexdigest() if session_id not in self.store: break return session_id def _valid_session_id(self, session_id): rx = utils.re_compile("^[0-9a-fA-F]+$") return rx.match(session_id) def _cleanup(self): """Cleanup the stored sessions""" current_time = time.time() timeout = self._config.timeout if current_time - self._last_cleanup_time > timeout: self.store.cleanup(timeout) self._last_cleanup_time = current_time def expired(self): """Called when an expired session is atime""" self._killed = True self._save() raise SessionExpired(self._config.expired_message) def kill(self): """Kill the session, make it no longer available""" del self.store[self.session_id] self._killed = True class Store: """Base class for session stores""" def __contains__(self, key): raise NotImplementedError() def __getitem__(self, key): raise NotImplementedError() def __setitem__(self, key, value): raise NotImplementedError() def cleanup(self, timeout): """removes all the expired sessions""" raise NotImplementedError() def encode(self, session_dict): """encodes session dict as a string""" pickled = pickle.dumps(session_dict) return encodebytes(pickled) def decode(self, session_data): """decodes the data to get back the session dict """ if isinstance(session_data, str): session_data = session_data.encode() pickled = decodebytes(session_data) return pickle.loads(pickled) class DiskStore(Store): """ Store for saving a session on disk. >>> import tempfile >>> root = tempfile.mkdtemp() >>> s = DiskStore(root) >>> s['a'] = 'foo' >>> s['a'] 'foo' >>> time.sleep(0.01) >>> s.cleanup(0.01) >>> s['a'] Traceback (most recent call last): ... KeyError: 'a' """ def __init__(self, root): # if the storage root doesn't exists, create it. if not os.path.exists(root): os.makedirs(os.path.abspath(root)) self.root = root def _get_path(self, key): if os.path.sep in key: raise ValueError("Bad key: %s" % repr(key)) return os.path.join(self.root, key) def __contains__(self, key): path = self._get_path(key) return os.path.exists(path) def __getitem__(self, key): path = self._get_path(key) if os.path.exists(path): with open(path, "rb") as fh: pickled = fh.read() return self.decode(pickled) else: raise KeyError(key) def __setitem__(self, key, value): path = self._get_path(key) pickled = self.encode(value) try: tname = path + "." + threading.current_thread().getName() f = open(tname, "wb") try: f.write(pickled) finally: f.close() shutil.move(tname, path) # atomary operation except IOError: pass def __delitem__(self, key): path = self._get_path(key) if os.path.exists(path): os.remove(path) def cleanup(self, timeout): if not os.path.isdir(self.root): return now = time.time() for f in os.listdir(self.root): path = self._get_path(f) atime = os.stat(path).st_atime if now - atime > timeout: shutil.rmtree(path) class DBStore(Store): """Store for saving a session in database Needs a table with the following columns: session_id CHAR(128) UNIQUE NOT NULL, atime DATETIME NOT NULL default current_timestamp, data TEXT """ def __init__(self, db, table_name): self.db = db self.table = table_name def __contains__(self, key): data = self.db.select(self.table, where="session_id=$key", vars=locals()) return bool(list(data)) def __getitem__(self, key): now = datetime.datetime.now() try: s = self.db.select(self.table, where="session_id=$key", vars=locals())[0] self.db.update( self.table, where="session_id=$key", atime=now, vars=locals() ) except IndexError: raise KeyError(key) else: return self.decode(s.data) def __setitem__(self, key, value): # Remove the leading `b` of bytes object (`b"..."`), otherwise encoded # value is invalid base64 format. pickled = self.encode(value).decode() now = datetime.datetime.now() if key in self: self.db.update( self.table, where="session_id=$key", data=pickled, atime=now, vars=locals(), ) else: self.db.insert(self.table, False, session_id=key, atime=now, data=pickled) def __delitem__(self, key): self.db.delete(self.table, where="session_id=$key", vars=locals()) def cleanup(self, timeout): timeout = datetime.timedelta( timeout / (24.0 * 60 * 60) ) # timedelta takes numdays as arg last_allowed_time = datetime.datetime.now() - timeout self.db.delete(self.table, where="$last_allowed_time > atime", vars=locals()) class ShelfStore: """Store for saving session using `shelve` module. import shelve store = ShelfStore(shelve.open('session.shelf')) XXX: is shelve thread-safe? """ def __init__(self, shelf): self.shelf = shelf def __contains__(self, key): return key in self.shelf def __getitem__(self, key): atime, v = self.shelf[key] self[key] = v # update atime return v def __setitem__(self, key, value): self.shelf[key] = time.time(), value def __delitem__(self, key): try: del self.shelf[key] except KeyError: pass def cleanup(self, timeout): now = time.time() for k in self.shelf: atime, v = self.shelf[k] if now - atime > timeout: del self[k] class MemoryStore(Store): """Store for saving a session in memory. Useful where there is limited fs writes on the disk, like flash memories Data will be saved into a dict: k: (time, pydata) """ def __init__(self, d_store=None): if d_store is None: d_store = {} self.d_store = d_store def __contains__(self, key): return key in self.d_store def __getitem__(self, key): """ Return the value and update the last seen value """ t, value = self.d_store[key] self.d_store[key] = (time.time(), value) return value def __setitem__(self, key, value): self.d_store[key] = (time.time(), value) def __delitem__(self, key): del self.d_store[key] def cleanup(self, timeout): now = time.time() to_del = [] for k, (atime, value) in iteritems(self.d_store): if now - atime > timeout: to_del.append(k) # to avoid exception on "dict change during iterations" for k in to_del: del self.d_store[k] if __name__ == "__main__": import doctest doctest.testmod() webpy-0.61/web/template.py000066400000000000000000001410561370675551300155530ustar00rootroot00000000000000""" simple, elegant templating (part of web.py) Template design: Template string is split into tokens and the tokens are combined into nodes. Parse tree is a nodelist. TextNode and ExpressionNode are simple nodes and for-loop, if-loop etc are block nodes, which contain multiple child nodes. Each node can emit some python string. python string emitted by the root node is validated for safeeval and executed using python in the given environment. Enough care is taken to make sure the generated code and the template has line to line match, so that the error messages can point to exact line number in template. (It doesn't work in some cases still.) Grammar: template -> defwith sections defwith -> '$def with (' arguments ')' | '' sections -> section* section -> block | assignment | line assignment -> '$ ' line -> (text|expr)* text -> expr -> '$' pyexpr | '$(' pyexpr ')' | '${' pyexpr '}' pyexpr -> """ import ast import glob import os import sys import tokenize from io import open import builtins from .net import websafe from .utils import re_compile, safestr, safeunicode, storage from .webapi import config __all__ = [ "Template", "Render", "render", "frender", "ParseError", "SecurityError", "test", ] from collections.abc import MutableMapping def splitline(text): r""" Splits the given text at newline. >>> splitline('foo\nbar') ('foo\n', 'bar') >>> splitline('foo') ('foo', '') >>> splitline('') ('', '') """ index = text.find("\n") + 1 if index: return text[:index], text[index:] else: return text, "" class Parser: """Parser Base. """ def __init__(self): self.statement_nodes = STATEMENT_NODES self.keywords = KEYWORDS def parse(self, text, name="