Genshi-0.7/0000775000175000017500000000000012101253106013062 5ustar simonsimon00000000000000Genshi-0.7/PKG-INFO0000664000175000017500000000231412101253106014157 0ustar simonsimon00000000000000Metadata-Version: 1.1 Name: Genshi Version: 0.7 Summary: A toolkit for generation of output for the web Home-page: http://genshi.edgewall.org/ Author: Edgewall Software Author-email: info@edgewall.org License: BSD Download-URL: http://genshi.edgewall.org/wiki/Download Description: Genshi is a Python library that provides an integrated set of components for parsing, generating, and processing HTML, XML or other textual content for output generation on the web. The major feature is a template language, which is heavily inspired by Kid. Keywords: python.templating.engines Platform: UNKNOWN Classifier: Development Status :: 4 - Beta Classifier: Environment :: Web Environment Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: BSD License Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 2 Classifier: Programming Language :: Python :: 3 Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content Classifier: Topic :: Software Development :: Libraries :: Python Modules Classifier: Topic :: Text Processing :: Markup :: HTML Classifier: Topic :: Text Processing :: Markup :: XML Genshi-0.7/MANIFEST.in0000664000175000017500000000063112101246063014624 0ustar simonsimon00000000000000include ChangeLog include COPYING recursive-include genshi *.py *.c *.txt *.html recursive-include scripts *.py recursive-include fixes *.py recursive-include examples * recursive-include doc *.html *.css *.txt *.png *.gif *.py *.ini COPYING recursive-exclude doc/logo.lineform * exclude doc/2000ft.graffle global-exclude *.pyc include fixes/*.* recursive-include genshi/template/tests/templates *.html *.txt Genshi-0.7/COPYING0000664000175000017500000000261112101246063014121 0ustar simonsimon00000000000000Copyright (C) 2006-2010 Edgewall Software All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. The name of the author may not be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. Genshi-0.7/doc/0000775000175000017500000000000012101253106013627 5ustar simonsimon00000000000000Genshi-0.7/doc/plugin.txt0000664000175000017500000002477412101246063015710 0ustar simonsimon00000000000000.. -*- mode: rst; encoding: utf-8 -*- =========================== Using the Templating Plugin =========================== While you can easily use Genshi templating through the APIs provided directly by Genshi, in some situations you may want to use Genshi through the template engine plugin API. Note though that this considerably limits the power and flexibility of Genshi templates (for example, there's no good way to use filters such as Genshi's `HTMLFormFiller`_ when the plugin API is sitting between your code and Genshi). .. _`HTMLFormFiller`: filters.html>#html-form-filler .. contents:: Contents :depth: 2 .. sectnum:: Introduction ============ Some Python web frameworks support a variety of different templating engines through the `Template Engine Plugin API`_, which was first developed by the Buffet_ and TurboGears_ projects. .. _`Template Engine Plugin API`: http://docs.turbogears.org/1.0/TemplatePlugins .. _`Buffet`: http://projects.dowski.com/projects/buffet .. _`TurboGears`: http://www.turbogears.org/ Genshi supports this API out of the box, so you can use it in frameworks like TurboGears or `Pylons`_ without installing any additional packages. A small example TurboGears application is included in the ``examples`` directory of source distributions of Genshi. .. _`Pylons`: http://pylonshq.com/ Usage ===== The way you use Genshi through the plugin API depends very much on the framework you're using. In general, the approach will look something like the following: (1) Configure Genshi as the default (or an additional) template engine (2) Optionally specify Genshi-specific `configuration options`_ (3) For any given *view* or *controller* (or whatever those are called in your framework of choice), specify the name of the template to use and which data should be made available to it. For point 1, you'll have to specify the *name* of the template engine plugin. For Genshi, this is **"genshi"**. However, because Genshi supports both markup and text templates, it also provides two separate plugins, namely **"genshi-markup"** and **"genshi-text"** (the "genshi" name is just an alias for "genshi-markup"). Usually, you can choose a default template engine, but also use a different engine on a per-request basis. So to use markup templates in general, but a text template in a specific controller, you'd configure "genshi" as the default template engine, and specify "genshi-text" for the controllers that should use text templates. How exactly this works depends on the framework in use. When rendering a specific template in a controller (point 3 above), you may also be able to pass additional options to the plugin. This includes the ``format`` keyword argument, which Genshi will use to override the configured default serialization method. In combination with specifying the "genshi-text" engine name as explained above, you would use this to specify the "text" serialization method when you want to use a text template. Or you'd specify "xml" for the format when you want to produce an Atom feed or other XML content. Template Paths -------------- How you specify template paths depends on whether you have a `search path`_ set up or not. The search path is a list of directories that Genshi should load templates from. Now when you request a template using a relative path such as ``mytmpl.html`` or ``foo/mytmpl.html``, Genshi will look for that file in the directories on the search path. For mostly historical reasons, the Genshi template engine plugin uses a different approach when you **haven't** configured the template search path: you now load templates using *dotted notation*, for example ``mytmpl`` or ``foo.mytmpl``. Note how you've lost the ability to explicitly specify the file extension: you now have to use ``.html`` for markup templates, and ``.txt`` for text templates. Using the search path is recommended for a number of reasons: First, it's the native Genshi model and is thus more robust and better supported. Second, a search path gives you much more flexibility for organizing your application templates. And as noted above, you aren't forced to use hardcoded filename extensions for your template files. Extra Implicit Objects ---------------------- The "genshi-markup" template engine plugin adds some extra functions that are made available to all templates implicitly, namely: ``HTML(string)`` Parses the given string as HTML and returns a markup stream. ``XML(string)`` Parses the given string as XML and returns a markup stream. ``ET(tree)`` Adapts the given `ElementTree`_ object to a markup stream. The framework may make additional objects available by default. Consult the documentation of your framework for more information. .. _elementtree: http://effbot.org/zone/element-index.htm .. _`configuration options`: Configuration Options ===================== The plugin API allows plugins to be configured using a dictionary of strings. The following is a list of configuration options that Genshi supports. These may or may not be made available by your framework. TurboGears 1.0, for example, only passes a fixed set of options to all plugins. ``genshi.allow_exec`` -------------------------- Whether the Python code blocks should be permitted in templates. Specify "yes" to allow code blocks (which is the default), or "no" otherwise. Please note that disallowing code blocks in templates does not turn Genshi into a sandboxable template engine; there are sufficient ways to do harm even using plain expressions. ``genshi.auto_reload`` ---------------------- Whether the template loader should check the last modification time of template files, and automatically reload them if they have been changed. Specify "yes" to enable this reloading (which is the default), or "no" to turn it off. You probably want to disable reloading in a production environment to improve performance of both templating loading and the processing of includes. But remember that you'll then have to manually restart the server process anytime the templates are updated. ``genshi.default_doctype`` -------------------------- The default ``DOCTYPE`` declaration to use in generated markup. Valid values are: **html-strict** (or just **html**) HTML 4.01 Strict **html-transitional** HTML 4.01 Transitional **xhtml-strict** (or just **xhtml**) XHTML 1.0 Strict **xhtml-transitional** XHTML 1.0 Transitional **html5** HTML5 (as `proposed`_ by the WHAT-WG) .. _proposed: http://www.whatwg.org/specs/web-apps/current-work/ .. note:: While using the Genshi API directly allows you to specify document types not in that list, the *dictionary-of-strings* based configuration utilized by the plugin API unfortunately limits your choices to those listed above. The default behavior is to not do any prepending/replacing of a ``DOCTYPE``, but rather pass through those defined in the templates (if any). If this option is set, however, any ``DOCTYPE`` declarations in the templates are replaced by the specified document type. Note that with (X)HTML, the presence and choice of the ``DOCTYPE`` can have a more or less dramatic impact on how modern browsers render pages that use CSS style sheets. In particular, browsers may switch to *quirks rendering mode* for certain document types, or when the ``DOCTYPE`` declaration is missing completely. For more information on the choice of the appropriate ``DOCTYPE``, see: * `Recommended DTDs to use in your Web document `_ * `Choosing a DOCTYPE `_ ``genshi.default_encoding`` --------------------------- The default output encoding to use when serializing a template. By default, Genshi uses UTF-8. If you need to, you can choose a different charset by specifying this option, although that rarely makes sense. As Genshi is not in control over what HTTP headers are being sent together with the template output, make sure that you (or the framework you're using) specify the chosen encoding as part of the outgoing ``Content-Type`` header. For example:: Content-Type: text/html; charset=utf-8 .. note:: Browsers commonly use ISO-8859-1 by default for ``text/html``, so even if you use Genshi's default UTF-8 encoding, you'll have to let the browser know about that explicitly ``genshi.default_format`` ------------------------- Determines the default serialization method to use. Valid options are: **xml** Serialization to XML **xhtml** Serialization to XHTML in a way that should be compatible with HTML (i.e. the result can be sent using the ``text/html`` MIME type, but can also be handled by XML parsers if you're careful). **html** Serialization to HTML **text** Plain text serialization See `Understanding HTML, XML and XHTML`_ for an excellent description of the subtle differences between the three different markup serialization options. As a general recommendation, if you don't have a special requirement to produce well-formed XML, you should probably use the **html** option for your web sites. .. _`Understanding HTML, XML and XHTML`: http://webkit.org/blog/?p=68 ``genshi.loader_callback`` -------------------------- The callback function that should be invoked whenever the template loader loads a new template. .. note:: Unlike the other options, this option can **not** be passed as a string value, but rather must be a reference to the actual function. That effectively means it can not be set from (non-Python) configuration files. ``genshi.lookup_errors`` ------------------------ The error handling style to use in template expressions. Can be either **lenient** (the default) or **strict**. See the `Error Handling`_ section for detailled information on the differences between these two modes. .. _`Error Handling`: templates.html#template-expressions-and-code-blocks ``genshi.max_cache_size`` ------------------------- The maximum number of templates that the template loader will cache in memory. The default value is **25**. You may want to choose a higher value if your web site uses a larger number of templates, and you have enough memory to spare. ``genshi.new_text_syntax`` -------------------------- Whether the new syntax for text templates should be used. Specify "yes" to enable the new syntax, or "no" to use the old syntax. In the version of Genshi, the default is to use the old syntax for backwards-compatibility, but that will change in a future release. .. _`search path`: ``genshi.search_path`` ---------------------- A colon-separated list of file-system path names that the template loader should use to search for templates. Genshi-0.7/doc/filters.html0000664000175000017500000005110412101251412016164 0ustar simonsimon00000000000000 Genshi: Stream Filters

Stream Filters

Markup Streams showed how to write filters and how they are applied to markup streams. This page describes the features of the various filters that come with Genshi itself.

1   HTML Form Filler

The filter genshi.filters.html.HTMLFormFiller can automatically populate an HTML form from values provided as a simple dictionary. When using this filter, you can basically omit any value, selected, or checked attributes from form controls in your templates, and let the filter do all that work for you.

HTMLFormFiller takes a dictionary of data to populate the form with, where the keys should match the names of form elements, and the values determine the values of those controls. For example:

>>> from genshi.filters import HTMLFormFiller
>>> from genshi.template import MarkupTemplate

>>> template = MarkupTemplate("""<form>
...   <p>
...     <label>User name:
...       <input type="text" name="username" />
...     </label><br />
...     <label>Password:
...       <input type="password" name="password" />
...     </label><br />
...     <label>
...       <input type="checkbox" name="remember" /> Remember me
...     </label>
...   </p>
... </form>""")
>>> filler = HTMLFormFiller(data=dict(username='john', remember=True))
>>> print(template.generate() | filler)
<form>
  <p>
    <label>User name:
      <input type="text" name="username" value="john"/>
    </label><br/>
    <label>Password:
      <input type="password" name="password"/>
    </label><br/>
    <label>
      <input type="checkbox" name="remember" checked="checked"/> Remember me
    </label>
  </p>
</form>

Note

This processing is done without in any way reparsing the template output. As any stream filter it operates after the template output is generated but before that output is actually serialized.

The filter will of course also handle radio buttons as well as <select> and <textarea> elements. For radio buttons to be marked as checked, the value in the data dictionary needs to match the value attribute of the <input> element, or evaluate to a truth value if the element has no such attribute. For options in a <select> box to be marked as selected, the value in the data dictionary needs to match the value attribute of the <option> element, or the text content of the option if it has no value attribute. Password and file input fields are not populated, as most browsers would ignore that anyway for security reasons.

You'll want to make sure that the values in the data dictionary have already been converted to strings. While the filter may be able to deal with non-string data in some cases (such as check boxes), in most cases it will either not attempt any conversion or not produce the desired results.

You can restrict the form filler to operate only on a specific <form> by passing either the id or the name keyword argument to the initializer. If either of those is specified, the filter will only apply to form tags with an attribute matching the specified value.

2   HTML Sanitizer

The filter genshi.filters.html.HTMLSanitizer filter can be used to clean up user-submitted HTML markup, removing potentially dangerous constructs that could be used for various kinds of abuse, such as cross-site scripting (XSS) attacks:

>>> from genshi.filters import HTMLSanitizer
>>> from genshi.input import HTML

>>> html = HTML(u"""<div>
...   <p>Innocent looking text.</p>
...   <script>alert("Danger: " + document.cookie)</script>
... </div>""")
>>> sanitize = HTMLSanitizer()
>>> print(html | sanitize)
<div>
  <p>Innocent looking text.</p>
</div>

In this example, the <script> tag was removed from the output.

You can determine which tags and attributes should be allowed by initializing the filter with corresponding sets. See the API documentation for more information.

Inline style attributes are forbidden by default. If you allow them, the filter will still perform sanitization on the contents any encountered inline styles: the proprietary expression() function (supported only by Internet Explorer) is removed, and any property using an url() which a potentially dangerous URL scheme (such as javascript:) are also stripped out:

>>> from genshi.filters import HTMLSanitizer
>>> from genshi.input import HTML

>>> html = HTML(u"""<div>
...   <br style="background: url(javascript:alert(document.cookie); color: #000" />
... </div>""")
>>> sanitize = HTMLSanitizer(safe_attrs=HTMLSanitizer.SAFE_ATTRS | set(['style']))
>>> print(html | sanitize)
<div>
  <br style="color: #000"/>
</div>

Warning

You should probably not rely on the style filtering, as sanitizing mixed HTML, CSS, and Javascript is very complicated and suspect to various browser bugs. If you can somehow get away with not allowing inline styles in user-submitted content, that would definitely be the safer route to follow.

3   Transformer

The filter genshi.filters.transform.Transformer provides a convenient way to transform or otherwise work with markup event streams. It allows you to specify which parts of the stream you're interested in with XPath expressions, and then attach a variety of transformations to the parts that match:

>>> from genshi.builder import tag
>>> from genshi.core import TEXT
>>> from genshi.filters import Transformer
>>> from genshi.input import HTML

>>> html = HTML(u'''<html>
...   <head><title>Some Title</title></head>
...   <body>
...     Some <em>body</em> text.
...   </body>
... </html>''')

>>> print(html | Transformer('body/em').map(unicode.upper, TEXT)
...                                    .unwrap().wrap(tag.u).end()
...                                    .select('body/u')
...                                    .prepend('underlined '))
<html>
  <head><title>Some Title</title></head>
  <body>
    Some <u>underlined BODY</u> text.
  </body>
</html>

This example sets up a transformation that:

  1. matches any <em> element anywhere in the body,
  2. uppercases any text nodes in the element,
  3. strips off the <em> start and close tags,
  4. wraps the content in a <u> tag, and
  5. inserts the text underlined inside the <u> tag.

A number of commonly useful transformations are available for this filter. Please consult the API documentation a complete list.

In addition, you can also perform custom transformations. For example, the following defines a transformation that changes the name of a tag:

>>> from genshi import QName
>>> from genshi.filters.transform import ENTER, EXIT

>>> class RenameTransformation(object):
...    def __init__(self, name):
...        self.name = QName(name)
...    def __call__(self, stream):
...        for mark, (kind, data, pos) in stream:
...            if mark is ENTER:
...                data = self.name, data[1]
...            elif mark is EXIT:
...                data = self.name
...            yield mark, (kind, data, pos)

A transformation can be any callable object that accepts an augmented event stream. In this case we define a class, so that we can initialize it with the tag name.

Custom transformations can be applied using the apply() method of a transformer instance:

>>> xform = Transformer('body//em').map(unicode.upper, TEXT) \
>>> xform = xform.apply(RenameTransformation('u'))
>>> print(html | xform)
<html>
  <head><title>Some Title</title></head>
  <body>
    Some <u>BODY</u> text.
  </body>
</html>

Note

The transformation filter was added in Genshi 0.5.

4   Translator

The genshi.filters.i18n.Translator filter implements basic support for internationalizing and localizing templates. When used as a filter, it translates a configurable set of text nodes and attribute values using a gettext-style translation function.

The Translator class also defines the extract class method, which can be used to extract localizable messages from a template.

Please refer to the API documentation for more information on this filter.

Note

The translation filter was added in Genshi 0.4.

Genshi-0.7/doc/i18n.txt0000664000175000017500000004436512101246063015167 0ustar simonsimon00000000000000.. -*- mode: rst; encoding: utf-8 -*- ===================================== Internationalization and Localization ===================================== Genshi provides comprehensive supporting infrastructure for internationalizing and localizing templates. That includes functionality for extracting localizable strings from templates, as well as a template filter and special directives that can apply translations to templates as they get rendered. This support is based on `gettext`_ message catalogs and the `gettext Python module`_. The extraction process can be used from the API level, or through the front-ends implemented by the `Babel`_ project, for which Genshi provides a plugin. .. _`gettext`: http://www.gnu.org/software/gettext/ .. _`gettext python module`: http://docs.python.org/lib/module-gettext.html .. _`babel`: http://babel.edgewall.org/ .. contents:: Contents :depth: 2 .. sectnum:: Basics ====== The simplest way to internationalize and translate templates would be to wrap all localizable strings in a ``gettext()`` function call (which is often aliased to ``_()`` for brevity). In that case, no extra template filter is required. .. code-block:: genshi

${_("Hello, world!")}

However, this approach results in significant “character noise” in templates, making them harder to read and preview. The ``genshi.filters.Translator`` filter allows you to get rid of the explicit `gettext`_ function calls, so you can (often) just continue to write: .. code-block:: genshi

Hello, world!

This text will still be extracted and translated as if you had wrapped it in a ``_()`` call. .. note:: For parameterized or pluralizable messages, you need to use the special `template directives`_ described below, or use the corresponding ``gettext`` function in embedded Python expressions. You can control which tags should be ignored by this process; for example, it doesn't really make sense to translate the content of the HTML ```` element. Both ``') >>> print(tmpl.generate()) On the other hand, Genshi will always replace two dollar signs in text with a single dollar sign, so you'll need to use three dollar signs to get two in the output: .. code-block:: pycon >>> tmpl = MarkupTemplate('') >>> print(tmpl.generate()) .. _`code blocks`: Code Blocks =========== Templates also support full Python code blocks, using the ```` processing instruction in XML templates: .. code-block:: genshi
${greeting('world')}
This will produce the following output: .. code-block:: xml
Hello, world!
In text templates (although only those using the new syntax introduced in Genshi 0.5), code blocks use the special ``{% python %}`` directive: .. code-block:: genshitext {% python from genshi.builder import tag def greeting(name): return 'Hello, %s!' % name %} ${greeting('world')} This will produce the following output:: Hello, world! Code blocks can import modules, define classes and functions, and basically do anything you can do in normal Python code. What code blocks can *not* do is to produce content that is emitted directly tp the generated output. .. note:: Using the ``print`` statement will print to the standard output stream, just as it does for other Python code in your application. Unlike expressions, Python code in ```` processing instructions can not use item and attribute access in an interchangeable manner. That means that “dotted notation” is always attribute access, and vice-versa. The support for Python code blocks in templates is not supposed to encourage mixing application code into templates, which is generally considered bad design. If you're using many code blocks, that may be a sign that you should move such code into separate Python modules. If you'd rather not allow the use of Python code blocks in templates, you can simply set the ``allow_exec`` parameter (available on the ``Template`` and the ``TemplateLoader`` initializers) to ``False``. In that case Genshi will raise a syntax error when a ```` processing instruction is encountered. But please note that disallowing code blocks in templates does not turn Genshi into a sandboxable template engine; there are sufficient ways to do harm even using plain expressions. .. _`error handling`: Error Handling ============== By default, Genshi raises an ``UndefinedError`` if a template expression attempts to access a variable that is not defined: .. code-block:: pycon >>> from genshi.template import MarkupTemplate >>> tmpl = MarkupTemplate('

${doh}

') >>> tmpl.generate().render('xhtml') Traceback (most recent call last): ... UndefinedError: "doh" not defined You can change this behavior by setting the variable lookup mode to "lenient". In that case, accessing undefined variables returns an `Undefined` object, meaning that the expression does not fail immediately. See below for details. If you need to check whether a variable exists in the template context, use the defined_ or the value_of_ function described below. To check for existence of attributes on an object, or keys in a dictionary, use the ``hasattr()``, ``getattr()`` or ``get()`` functions, or the ``in`` operator, just as you would in regular Python code: >>> from genshi.template import MarkupTemplate >>> tmpl = MarkupTemplate('

${defined("doh")}

') >>> print(tmpl.generate().render('xhtml'))

False

.. note:: Lenient error handling was the default in Genshi prior to version 0.5. Strict mode was introduced in version 0.4, and became the default in 0.5. The reason for this change was that the lenient error handling was masking actual errors in templates, thereby also making it harder to debug some problems. .. _`lenient`: Lenient Mode ------------ If you instruct Genshi to use the lenient variable lookup mode, it allows you to access variables that are not defined, without raising an ``UndefinedError``. This mode can be chosen by passing the ``lookup='lenient'`` keyword argument to the template initializer, or by passing the ``variable_lookup='lenient'`` keyword argument to the ``TemplateLoader`` initializer: .. code-block:: pycon >>> from genshi.template import MarkupTemplate >>> tmpl = MarkupTemplate('

${doh}

', lookup='lenient') >>> print(tmpl.generate().render('xhtml'))

You *will* however get an exception if you try to call an undefined variable, or do anything else with it, such as accessing its attributes: .. code-block:: pycon >>> from genshi.template import MarkupTemplate >>> tmpl = MarkupTemplate('

${doh.oops}

', lookup='lenient') >>> print(tmpl.generate().render('xhtml')) Traceback (most recent call last): ... UndefinedError: "doh" not defined If you need to know whether a variable is defined, you can check its type against the ``Undefined`` class, for example in a conditional directive: .. code-block:: pycon >>> from genshi.template import MarkupTemplate >>> tmpl = MarkupTemplate('

${type(doh) is not Undefined}

', ... lookup='lenient') >>> print(tmpl.generate().render('xhtml'))

False

Alternatively, the built-in functions defined_ or value_of_ can be used in this case. Custom Modes ------------ In addition to the built-in "lenient" and "strict" modes, it is also possible to use a custom error handling mode. For example, you could use lenient error handling in a production environment, while also logging a warning when an undefined variable is referenced. See the API documentation of the ``genshi.template.eval`` module for details. Built-in Functions & Types ========================== The following functions and types are available by default in template code, in addition to the standard built-ins that are available to all Python code. .. _`defined`: ``defined(name)`` ----------------- This function determines whether a variable of the specified name exists in the context data, and returns ``True`` if it does. .. _`value_of`: ``value_of(name, default=None)`` -------------------------------- This function returns the value of the variable with the specified name if such a variable is defined, and returns the value of the ``default`` parameter if no such variable is defined. .. _`Markup`: ``Markup(text)`` ---------------- The ``Markup`` type marks a given string as being safe for inclusion in markup, meaning it will *not* be escaped in the serialization stage. Use this with care, as not escaping a user-provided string may allow malicious users to open your web site to cross-site scripting attacks. .. _`Undefined`: ``Undefined`` ---------------- The ``Undefined`` type can be used to check whether a reference variable is defined, as explained in `error handling`_. .. _`directives`: ------------------- Template Directives ------------------- Directives provide control flow functionality for templates, such as conditions or iteration. As the syntax for directives depends on whether you're using markup or text templates, refer to the `XML Template Language `_ or `Text Template Language `_ pages for information. Genshi-0.7/doc/plugin.html0000664000175000017500000004007512101251411016016 0ustar simonsimon00000000000000 Genshi: Using the Templating Plugin

Using the Templating Plugin

While you can easily use Genshi templating through the APIs provided directly by Genshi, in some situations you may want to use Genshi through the template engine plugin API. Note though that this considerably limits the power and flexibility of Genshi templates (for example, there's no good way to use filters such as Genshi's HTMLFormFiller when the plugin API is sitting between your code and Genshi).

1   Introduction

Some Python web frameworks support a variety of different templating engines through the Template Engine Plugin API, which was first developed by the Buffet and TurboGears projects.

Genshi supports this API out of the box, so you can use it in frameworks like TurboGears or Pylons without installing any additional packages. A small example TurboGears application is included in the examples directory of source distributions of Genshi.

2   Usage

The way you use Genshi through the plugin API depends very much on the framework you're using. In general, the approach will look something like the following:

  1. Configure Genshi as the default (or an additional) template engine
  2. Optionally specify Genshi-specific configuration options
  3. For any given view or controller (or whatever those are called in your framework of choice), specify the name of the template to use and which data should be made available to it.

For point 1, you'll have to specify the name of the template engine plugin. For Genshi, this is "genshi". However, because Genshi supports both markup and text templates, it also provides two separate plugins, namely "genshi-markup" and "genshi-text" (the "genshi" name is just an alias for "genshi-markup").

Usually, you can choose a default template engine, but also use a different engine on a per-request basis. So to use markup templates in general, but a text template in a specific controller, you'd configure "genshi" as the default template engine, and specify "genshi-text" for the controllers that should use text templates. How exactly this works depends on the framework in use.

When rendering a specific template in a controller (point 3 above), you may also be able to pass additional options to the plugin. This includes the format keyword argument, which Genshi will use to override the configured default serialization method. In combination with specifying the "genshi-text" engine name as explained above, you would use this to specify the "text" serialization method when you want to use a text template. Or you'd specify "xml" for the format when you want to produce an Atom feed or other XML content.

2.1   Template Paths

How you specify template paths depends on whether you have a search path set up or not. The search path is a list of directories that Genshi should load templates from. Now when you request a template using a relative path such as mytmpl.html or foo/mytmpl.html, Genshi will look for that file in the directories on the search path.

For mostly historical reasons, the Genshi template engine plugin uses a different approach when you haven't configured the template search path: you now load templates using dotted notation, for example mytmpl or foo.mytmpl. Note how you've lost the ability to explicitly specify the file extension: you now have to use .html for markup templates, and .txt for text templates.

Using the search path is recommended for a number of reasons: First, it's the native Genshi model and is thus more robust and better supported. Second, a search path gives you much more flexibility for organizing your application templates. And as noted above, you aren't forced to use hardcoded filename extensions for your template files.

2.2   Extra Implicit Objects

The "genshi-markup" template engine plugin adds some extra functions that are made available to all templates implicitly, namely:

HTML(string)
Parses the given string as HTML and returns a markup stream.
XML(string)
Parses the given string as XML and returns a markup stream.
ET(tree)
Adapts the given ElementTree object to a markup stream.

The framework may make additional objects available by default. Consult the documentation of your framework for more information.

3   Configuration Options

The plugin API allows plugins to be configured using a dictionary of strings. The following is a list of configuration options that Genshi supports. These may or may not be made available by your framework. TurboGears 1.0, for example, only passes a fixed set of options to all plugins.

3.1   genshi.allow_exec

Whether the Python code blocks should be permitted in templates. Specify "yes" to allow code blocks (which is the default), or "no" otherwise. Please note that disallowing code blocks in templates does not turn Genshi into a sandboxable template engine; there are sufficient ways to do harm even using plain expressions.

3.2   genshi.auto_reload

Whether the template loader should check the last modification time of template files, and automatically reload them if they have been changed. Specify "yes" to enable this reloading (which is the default), or "no" to turn it off.

You probably want to disable reloading in a production environment to improve performance of both templating loading and the processing of includes. But remember that you'll then have to manually restart the server process anytime the templates are updated.

3.3   genshi.default_doctype

The default DOCTYPE declaration to use in generated markup. Valid values are:

html-strict (or just html)
HTML 4.01 Strict
html-transitional
HTML 4.01 Transitional
xhtml-strict (or just xhtml)
XHTML 1.0 Strict
xhtml-transitional
XHTML 1.0 Transitional
html5
HTML5 (as proposed by the WHAT-WG)

Note

While using the Genshi API directly allows you to specify document types not in that list, the dictionary-of-strings based configuration utilized by the plugin API unfortunately limits your choices to those listed above.

The default behavior is to not do any prepending/replacing of a DOCTYPE, but rather pass through those defined in the templates (if any). If this option is set, however, any DOCTYPE declarations in the templates are replaced by the specified document type.

Note that with (X)HTML, the presence and choice of the DOCTYPE can have a more or less dramatic impact on how modern browsers render pages that use CSS style sheets. In particular, browsers may switch to quirks rendering mode for certain document types, or when the DOCTYPE declaration is missing completely.

For more information on the choice of the appropriate DOCTYPE, see:

3.4   genshi.default_encoding

The default output encoding to use when serializing a template. By default, Genshi uses UTF-8. If you need to, you can choose a different charset by specifying this option, although that rarely makes sense.

As Genshi is not in control over what HTTP headers are being sent together with the template output, make sure that you (or the framework you're using) specify the chosen encoding as part of the outgoing Content-Type header. For example:

Content-Type: text/html; charset=utf-8

Note

Browsers commonly use ISO-8859-1 by default for text/html, so even if you use Genshi's default UTF-8 encoding, you'll have to let the browser know about that explicitly

3.5   genshi.default_format

Determines the default serialization method to use. Valid options are:

xml
Serialization to XML
xhtml
Serialization to XHTML in a way that should be compatible with HTML (i.e. the result can be sent using the text/html MIME type, but can also be handled by XML parsers if you're careful).
html
Serialization to HTML
text
Plain text serialization

See Understanding HTML, XML and XHTML for an excellent description of the subtle differences between the three different markup serialization options. As a general recommendation, if you don't have a special requirement to produce well-formed XML, you should probably use the html option for your web sites.

3.6   genshi.loader_callback

The callback function that should be invoked whenever the template loader loads a new template.

Note

Unlike the other options, this option can not be passed as a string value, but rather must be a reference to the actual function. That effectively means it can not be set from (non-Python) configuration files.

3.7   genshi.lookup_errors

The error handling style to use in template expressions. Can be either lenient (the default) or strict. See the Error Handling section for detailled information on the differences between these two modes.

3.8   genshi.max_cache_size

The maximum number of templates that the template loader will cache in memory. The default value is 25. You may want to choose a higher value if your web site uses a larger number of templates, and you have enough memory to spare.

3.9   genshi.new_text_syntax

Whether the new syntax for text templates should be used. Specify "yes" to enable the new syntax, or "no" to use the old syntax.

In the version of Genshi, the default is to use the old syntax for backwards-compatibility, but that will change in a future release.

3.10   genshi.search_path

A colon-separated list of file-system path names that the template loader should use to search for templates.

Genshi-0.7/doc/i18n.html0000664000175000017500000007721012101251411015300 0ustar simonsimon00000000000000 Genshi: Internationalization and Localization

Internationalization and Localization

Genshi provides comprehensive supporting infrastructure for internationalizing and localizing templates. That includes functionality for extracting localizable strings from templates, as well as a template filter and special directives that can apply translations to templates as they get rendered.

This support is based on gettext message catalogs and the gettext Python module. The extraction process can be used from the API level, or through the front-ends implemented by the Babel project, for which Genshi provides a plugin.

1   Basics

The simplest way to internationalize and translate templates would be to wrap all localizable strings in a gettext() function call (which is often aliased to _() for brevity). In that case, no extra template filter is required.

<p>${_("Hello, world!")}</p>

However, this approach results in significant “character noise” in templates, making them harder to read and preview.

The genshi.filters.Translator filter allows you to get rid of the explicit gettext function calls, so you can (often) just continue to write:

<p>Hello, world!</p>

This text will still be extracted and translated as if you had wrapped it in a _() call.

Note

For parameterized or pluralizable messages, you need to use the special template directives described below, or use the corresponding gettext function in embedded Python expressions.

You can control which tags should be ignored by this process; for example, it doesn't really make sense to translate the content of the HTML <script></script> element. Both <script> and <style> are excluded by default.

Attribute values can also be automatically translated. The default is to consider the attributes abbr, alt, label, prompt, standby, summary, and title, which is a list that makes sense for HTML documents. Of course, you can tell the translator to use a different set of attribute names, or none at all.

1.1   Language Tagging

You can control automatic translation in your templates using the xml:lang attribute. If the value of that attribute is a literal string, the contents and attributes of the element will be ignored:

<p xml:lang="en">Hello, world!</p>

On the other hand, if the value of the xml:lang attribute contains a Python expression, the element contents and attributes are still considered for automatic translation:

<html xml:lang="$locale">
  ...
</html>

2   Template Directives

Sometimes localizable strings in templates may contain dynamic parameters, or they may depend on the numeric value of some variable to choose a proper plural form. Sometimes the strings contain embedded markup, such as tags for emphasis or hyperlinks, and you don't want to rely on the people doing the translations to know the syntax and escaping rules of HTML and XML.

In those cases the simple text extraction and translation process described above is not sufficient. You could just use gettext API functions in embedded Python expressions for parameters and pluralization, but that does not help when messages contain embedded markup. Genshi provides special template directives for internationalization that attempt to provide a comprehensive solution for this problem space.

To enable these directives, you'll need to register them with the templates they are used in. You can do this by adding them manually via the Template.add_directives(namespace, factory) (where namespace would be “http://genshi.edgewall.org/i18n” and factory would be an instance of the Translator class). Or you can just call the Translator.setup(template) class method, which both registers the directives and adds the translation filter.

After the directives have been registered with the template engine on the Python side of your application, you need to declare the corresponding directive namespace in all markup templates that use them. For example:

<html xmlns:py="http://genshi.edgewall.org/"
      xmlns:i18n="http://genshi.edgewall.org/i18n">
  …
</html>

These directives only make sense in the context of markup templates. For text templates, you can just use the corresponding gettext API calls as needed.

Note

The internationalization directives are still somewhat experimental and have some known issues. However, the attribute language they implement should be stable and is not subject to change substantially in future versions.

2.1   Messages

2.1.1   i18n:msg

This is the basic directive for defining localizable text passages that contain parameters and/or markup.

For example, consider the following template snippet:

<p>
  Please visit <a href="${site.url}">${site.name}</a> for help.
</p>

Without further annotation, the translation filter would treat this sentence as two separate messages (“Please visit” and “for help”), and the translator would have no control over the position of the link in the sentence.

However, when you use the Genshi internationalization directives, you simply add an i18n:msg attribute to the enclosing <p> element:

<p i18n:msg="name">
  Please visit <a href="${site.url}">${site.name}</a> for help.
</p>

Genshi is then able to identify the text in the <p> element as a single message for translation purposes. You'll see the following string in your message catalog:

Please visit [1:%(name)s] for help.

The <a> element with its attribute has been replaced by a part in square brackets, which does not include the tag name or the attributes of the element.

The value of the i18n:msg attribute is a comma-separated list of parameter names, which serve as simplified aliases for the actual Python expressions the message contains. The order of the paramer names in the list must correspond to the order of the expressions in the text. In this example, there is only one parameter: its alias for translation is “name”, while the corresponding expression is ${site.name}.

The translator now has complete control over the structure of the sentence. He or she certainly does need to make sure that any bracketed parts are not removed, and that the name parameter is preserved correctly. But those are things that can be easily checked by validating the message catalogs. The important thing is that the translator can change the sentence structure, and has no way to break the application by forgetting to close a tag, for example.

So if the German translator of this snippet decided to translate it to:

Um Hilfe zu erhalten, besuchen Sie bitte [1:%(name)s]

The resulting output might be:

<p>
  Um Hilfe zu erhalten, besuchen Sie bitte
  <a href="http://example.com/">Example</a>
</p>

Messages may contain multiple tags, and they may also be nested. For example:

<p i18n:msg="name">
  <i>Please</i> visit <b>the site <a href="${site.url}">${site.name}</a></b>
  for help.
</p>

This would result in the following message ID:

[1:Please] visit [2:the site [3:%(name)s]] for help.

Again, the translator has full control over the structure of the sentence. So the German translation could actually look like this:

Um Hilfe zu erhalten besuchen Sie [1:bitte]
[3:%(name)s], [2:das ist eine Web-Site]

Which Genshi would recompose into the following outout:

<p>
  Um Hilfe zu erhalten besuchen Sie <i>bitte</i>
  <a href="http://example.com/">Example</a>, <b>das ist eine Web-Site</b>
</p>

Note how the translation has changed the order and even the nesting of the tags.

Warning

Please note that i18n:msg directives do not support other nested directives. Directives commonly change the structure of the generated markup dynamically, which often would result in the structure of the text changing, thus making translation as a single message ineffective.

2.1.2   i18n:choose, i18n:singular, i18n:plural

Translatable strings that vary based on some number of objects, such as “You have 1 new message” or “You have 3 new messages”, present their own challenge, in particular when you consider that different languages have different rules for pluralization. For example, while English and most western languages have two plural forms (one for n=1 and an other for n<>1), Welsh has five different plural forms, while Hungarian only has one.

The gettext framework has long supported this via the ngettext() family of functions. You specify two default messages, one singular and one plural, and the number of items. The translations however may contain any number of plural forms for the message, depending on how many are commonly used in the language. ngettext will choose the correct plural form of the translated message based on the specified number of items.

Genshi provides a variant of the i18n:msg directive described above that allows choosing the proper plural form based on the numeric value of a given variable. The pluralization support is implemented in a set of three directives that must be used together: i18n:choose, i18n:singular, and i18n:plural.

The i18n:choose directive is used to set up the context of the message: it simply wraps the singular and plural variants.

The value of this directive is split into two parts: the first is the numeral, a Python expression that evaluates to a number to determine which plural form should be chosen. The second part, separated by a semicolon, lists the parameter names. This part is equivalent to the value of the i18n:msg directive.

For example:

<p i18n:choose="len(messages); num">
  <i18n:singular>You have <b>${len(messages)}</b> new message.</i18n:singular>
  <i18n:plural>You have <b>${len(messages)}</b> new messages.</i18n:plural>
</p>

All three directives can be used either as elements or attribute. So the above example could also be written as follows:

<i18n:choose numeral="len(messages)" params="num">
  <p i18n:singular="">You have <b>${len(messages)}</b> new message.</p>
  <p i18n:plural="">You have <b>${len(messages)}</b> new messages.</p>
</i18n:choose>

When used as an element, the two parts of the i18n:choose value are split into two different attributes: numeral and params. The i18n:singular and i18n:plural directives do not require or support any value (or any extra attributes).

2.2   Comments and Domains

2.2.1   i18n:comment

The i18n:comment directive can be used to supply a comment for the translator. For example, if a template snippet is not easily understood outside of its context, you can add a translator comment to help the translator understand in what context the message will be used:

<p i18n:msg="name" i18n:comment="Link to the relevant support site">
  Please visit <a href="${site.url}">${site.name}</a> for help.
</p>

This comment will be extracted together with the message itself, and will commonly be placed along the message in the message catalog, so that it is easily visible to the person doing the translation.

This directive has no impact on how the template is rendered, and is ignored outside of the extraction process.

2.2.2   i18n:domain

In larger projects, message catalogs are commonly split up into different domains. For example, you might have a core application domain, and then separate domains for extensions or libraries.

Genshi provides a directive called i18n:domain that lets you choose the translation domain for a particular scope. For example:

<div i18n:domain="examples">
  <p>Hello, world!</p>
</div>

3   Extraction

The Translator class provides a class method called extract, which is a generator yielding all localizable strings found in a template or markup stream. This includes both literal strings in text nodes and attribute values, as well as strings in gettext() calls in embedded Python code. See the API documentation for details on how to use this method directly.

3.1   Babel Integration

This functionality is integrated with the message extraction framework provided by the Babel project. Babel provides a command-line interface as well as commands that can be used from setup.py scripts using Setuptools or Distutils.

The first thing you need to do to make Babel extract messages from Genshi templates is to let Babel know which files are Genshi templates. This is done using a “mapping configuration”, which can be stored in a configuration file, or specified directly in your setup.py.

In a configuration file, the mapping may look like this:

# Python souce
[python:**.py]

# Genshi templates
[genshi:**/templates/**.html]
include_attrs = title

[genshi:**/templates/**.txt]
template_class = genshi.template.TextTemplate
encoding = latin-1

Please consult the Babel documentation for details on configuration.

If all goes well, running the extraction with Babel should create a POT file containing the strings from your Genshi templates and your Python source files.

3.2   Configuration Options

The Genshi extraction plugin for Babel supports the following options:

3.2.1   template_class

The concrete Template class that the file should be loaded with. Specify the package/module name and the class name, separated by a colon.

The default is to use genshi.template:MarkupTemplate, and you'll want to set it to genshi.template:TextTemplate for text templates.

3.2.2   encoding

The encoding of the template file. This is only used for text templates. The default is to assume “utf-8”.

3.2.3   include_attrs

Comma-separated list of attribute names that should be considered to have localizable values. Only used for markup templates.

3.2.4   ignore_tags

Comma-separated list of tag names that should be ignored. Only used for markup templates.

3.2.5   extract_text

Whether text outside explicit gettext function calls should be extracted. By default, any text nodes not inside ignored tags, and values of attribute in the include_attrs list are extracted. If this option is disabled, only strings in gettext function calls are extracted.

Note

If you disable this option, and do not make use of the internationalization directives, it's not necessary to add the translation filter as described above. You only need to make sure that the template has access to the gettext functions it uses.

4   Translation

If you have prepared MO files for use with Genshi using the appropriate tools, you can access the message catalogs with the gettext Python module. You'll probably want to create a gettext.GNUTranslations instance, and make the translation functions it provides available to your templates by putting them in the template context.

The Translator filter needs to be added to the filters of the template (applying it as a stream filter will likely not have the desired effect). Furthermore it needs to be the first filter in the list, including the internal filters that Genshi adds itself:

from genshi.filters import Translator
from genshi.template import MarkupTemplate

template = MarkupTemplate("...")
template.filters.insert(0, Translator(translations.ugettext))

The Translator class also provides the convenience method setup(), which will both add the filter and register the i18n directives:

from genshi.filters import Translator
from genshi.template import MarkupTemplate

template = MarkupTemplate("...")
translator = Translator(translations.ugettext)
translator.setup(template)

Warning

If you're using TemplateLoader, you should specify a callback function in which you add the filter. That ensures that the filter is not added everytime the template is rendered, thereby being applied multiple times.

Genshi-0.7/doc/loader.txt0000664000175000017500000002203312101246063015642 0ustar simonsimon00000000000000.. -*- mode: rst; encoding: utf-8 -*- ================= Loading Templates ================= Genshi comes with a simple but flexible implementation of a template loader in the ``genshi.template.loader`` module. The loader provides caching of templates so they do not need to be reparsed when used, support for multiple template directories that together form a virtual search path, as well as support for different template loading strategies. .. contents:: Contents :depth: 3 .. sectnum:: ----- Usage ----- The basic usage pattern is simple: instantiate one ``TemplateLoader`` object and keep it around, then ask it to load a template whenever you need to load one: .. code-block:: python from genshi.template import TemplateLoader loader = TemplateLoader(['/path/to/dir1', '/path/to/dir2'], auto_reload=True) tmpl = loader.load('test.html') When you try to load a template that can't be found, you get a ``TemplateNotFound`` error. The default template class used by the loader is ``MarkupTemplate``, but that can be overridden both with a different default (as a keyword argument to the ``TemplateLoader`` constructor), as well as on invocation of the ``load()`` method: .. code-block:: python from genshi.template.text import NewTextTemplate tmpl = loader.load('mail.txt', cls=NewTextTemplate) ------- Caching ------- The ``TemplateLoader`` class provides a simple in-memory cache for parsed template objects. This improves performance, because templates do not need to be reparsed every time they are rendered. The size of this cache can be adjusted using the `max_cache_size` option on the ``TemplateLoader`` constructor. The value of that option determines the maximum number of template objects kept in the cache. When this limit is reached, any templates that haven't been used in a while get purged. Technically, this is a least-recently-used (LRU) cache, the default limit is set to 25 templates. Automatic Reloading =================== Once a template has been cached, it will normally not get reparsed until it has been purged from the cache. This means that any changes to the template file are not taken into consideration as long as it is still found in the cache. As this is inconvenient in development scenarios, the ``auto_reload`` option allows for automatic cache invalidation based on whether the template source has changed. .. code-block:: python from genshi.template import TemplateLoader loader = TemplateLoader('templates', auto_reload=True, max_cache_size=100) In production environments, automatic reloading should be disabled, as it does affect performance negatively. Callback Interface ================== Sometimes you need to make sure that templates get properly configured after they have been loaded, but you only want to do that when the template is actually loaded and parsed, not when it is returned from the cache. For such cases, the ``TemplateLoader`` provides a way to specify a callback function that gets invoked whenever a template is loaded. You can specify that callback by passing it into the loader constructor via the ``callback`` keyword argument, or later by setting the attribute of the same name. The callback function should expect a single argument, the template object. For example, to properly inject the `translation filter`_ into any loaded template, you'd use code similar to this: .. code-block:: python from genshi.filters import Translator from genshi.template import TemplateLoader def template_loaded(template): Translator(translations.ugettext).setup(template) loader = TemplateLoader('templates', callback=template_loaded) .. _`translation filter`: i18n.html -------------------- Template Search Path -------------------- The template loader can be configured with a list of multiple directories to search for templates. The loader maps these directories to a single logical directory for locating templates by file name. The order of the directories making up the search path is significant: the loader will first try to locate a requested template in the first directory on the path, then in the second, and so on. If there are two templates with the same file name in multiple directories on the search path, whatever file is found first gets used. Based on this design, an application could, for example, configure a search path consisting of a directory containing the default templates, as well as a directory where site-specific templates can be stored that will override the default templates. Load Functions ============== Usually the search path consists of strings representing directory paths, but it may also contain “load functions”: functions that are basically invoked with the file name, and return the template content. Genshi comes with three builtin load functions: ``directory(path)`` ------------------- The equivalent of just using a string containing the directory path: looks up the file name in a specific directory. .. code-block:: python from genshi.template import TemplateLoader, loader tl = TemplateLoader([loader.directory('/path/to/dir/')]) That is the same as: .. code-block:: python tl = TemplateLoader(['/path/to/dir/']) ``package(name, path)`` ----------------------- Uses the ``pkg_resources`` API to locate files in Python package data (which may be inside a ZIP archive). .. code-block:: python from genshi.template import TemplateLoader, loader tl = TemplateLoader([loader.package('myapp', 'templates')]) This will look for templates in the ``templates`` directory of the Python package ``myapp``. ``prefixed(**delegates)`` ------------------------- Delegates load requests to different load functions based on the path prefix. .. code-block:: python from genshi.template import TemplateLoader, loader tl = TemplateLoader(loader.prefixed( core = '/tmp/dir1', plugin1 = loader.package('plugin1', 'templates'), plugin2 = loader.package('plugin2', 'templates'), )) tmpl = tl.load('core/index.html') This example sets up a loader with three delegates, under the prefixes “core”, “plugin1”, and “plugin2”. When a template is requested, the ``prefixed`` load function looks for a delegate with a corresponding prefix, removes the prefix from the path and asks the delegate to load the template. In this case, assuming the directory ``/path/to/dir`` contains a file named ``index.html``, that file will be used when we load ``core/index.html``. The other delegates are not checked as their prefix does not match. .. note:: These builtin load functions are available both as class methods of the ``TemplateLoader`` class as well as on the module level Custom Load Functions --------------------- You can easily use your own load function with the template loader, for example to load templates from a database. All that is needed is a callable object that accepts a ``filename`` (a string) and returns a tuple of the form ``(filepath, filename, fileobj, uptodate_fun)``, where: ``filepath`` is the absolute path to the template. This is primarily used for output in tracebacks, and does not need to map to an actual path on the file system. ``filename`` is the base name of the template file ``fileobj`` is a readable file-like object that provides the content of the template ``uptodate_fun`` is a function that the loader can invoke to check whether the cached version of the template is still up-to-date, or ``None`` if the load function is not able to provide such a check. If provided, the function should not expect any parameters (so you'll definitely want to use a closure here), and should return ``True`` if the template has not changed since it was last loaded. When the requested template can not be found, the function should raise an ``IOError`` or ``TemplateNotFound`` exception. ------------------ Customized Loading ------------------ If you require a completely different implementation of template loading, you can extend or even replace the builtin ``TemplateLoader`` class. Protocol ======== The protocol between the template loader and the ``Template`` class is simple and only used for processing includes. The only required part of that protocol is that the object assigned to ``Template.loader`` implements a ``load`` method compatible to that of the ``TemplateLoader`` class, at the minimum with the signature ``load(filename, relative_to=None, cls=None)``. In addition, templates currently check for the existence and value of a boolean ``auto_reload`` property. If the property does not exist or evaluates to a non-truth value, inlining of included templates is disabled. Inlining is a small optimization that removes some overhead in the processing of includes. Subclassing ``TemplateLoader`` ============================== You can also adjust the behavior of the ``TemplateLoader`` class by subclassing it. You can of course override anything needed, but the class also provides the ``_instantiate()`` hook, which is intended for use by subclasses to customize the creation of the template object from the file name and content. Please consult the code and the API documentation for more detail. Genshi-0.7/doc/upgrade.txt0000664000175000017500000001507512101246063016033 0ustar simonsimon00000000000000================ Upgrading Genshi ================ .. contents:: Contents :depth: 2 .. sectnum:: ------------------------------------------------------ Upgrading from Genshi 0.6.x to the development version ------------------------------------------------------ The Genshi development version now supports both Python 2 and Python 3. The most noticable API change in the Genshi development version is that the default encoding in numerous places is now None (i.e. unicode) instead of UTF-8. This change was made in order to ease the transition to Python 3 where strings are unicode strings by default. If your application relies on the default UTF-8 encoding a simple way to have it work both with Genshi 0.6.x and the development version is to specify the encoding explicitly in calls to the following classes, methods and functions: * genshi.HTML * genshi.Stream.render * genshi.input.HTMLParser * genshi.template.MarkupTemplate * genshi.template.TemplateLoader * genshi.template.TextTemplate (and genshi.template.NewTextTemplate) * genshi.template.OldTextTemplate Whether you explicitly specify UTF-8 or explicitly specify None (unicode) is a matter of personal taste, although working with unicode may make porting your own application to Python 3 easier. ------------------------------------ Upgrading from Genshi 0.5.x to 0.6.x ------------------------------------ Required Python Version ----------------------- Support for Python 2.3 has been dropped in this release. Python 2.4 is now the minimum version of Python required to run Genshi. The XPath engine has been completely overhauled for this version. Some patterns that previously matched incorrectly will no longer match, and the other way around. In such cases, the XPath expressions need to be fixed in your application and templates. ------------------------------------ Upgrading from Genshi 0.4.x to 0.5.x ------------------------------------ Error Handling -------------- The default error handling mode has been changed to "strict". This means that accessing variables not defined in the template data will now generate an immediate exception, as will accessing object attributes or dictionary keys that don't exist. If your templates rely on the old lenient behavior, you can configure Genshi to use that instead. See the documentation for details on how to do that. But be warned that lenient error handling may be removed completely in a future release. Match Template Processing ------------------------- There has also been a subtle change to how ``py:match`` templates are processed: in previous versions, all match templates would be applied to the content generated by the matching template, and only the matching template itself was applied recursively to the original content. This behavior resulted in problems with many kinds of recursive matching, and hence was changed for 0.5: now, all match templates declared before the matching template are applied to the original content, and match templates declared after the matching template are applied to the generated content. This change should not have any effect on most applications, but you may want to check your use of match templates to make sure. Text Templates -------------- Genshi 0.5 introduces a new, alternative syntax for text templates, which is more flexible and powerful compared to the old syntax. For backwards compatibility, this new syntax is not used by default, though it will be in a future version. It is recommended that you migrate to using this new syntax. To do so, simply rename any references in your code to ``TextTemplate`` to ``NewTextTemplate``. To explicitly use the old syntax, use ``OldTextTemplate`` instead, so that you can be sure you'll be using the same language when the default in Genshi is changed (at least until the old implementation is completely removed). ``Markup`` Constructor ---------------------- The ``Markup`` class no longer has a specialized constructor. The old (undocumented) constructor provided a shorthand for doing positional substitutions. If you have code like this: .. code-block:: python Markup('%s', name) You must replace it by the more explicit: .. code-block:: python Markup('%s') % name ``Template`` Constructor ------------------------ The constructor of the ``Template`` class and its subclasses has changed slightly: instead of the optional ``basedir`` parameter, it now expects an (also optional) ``filepath`` parameter, which specifies the absolute path to the template. You probably aren't using those constructors directly, anyway, but using the ``TemplateLoader`` API instead. ------------------------------------ Upgrading from Genshi 0.3.x to 0.4.x ------------------------------------ The modules ``genshi.filters`` and ``genshi.template`` have been refactored into packages containing multiple modules. While code using the regular APIs should continue to work without problems, you should make sure to remove any leftover traces of the files ``filters.py`` and ``template.py`` in the ``genshi`` package on the installation path (including the corresponding ``.pyc`` files). This is not necessary when Genshi was installed as a Python egg. Results of evaluating template expressions are no longer implicitly called if they are callable. If you have been using that feature, you will need to add the parenthesis to actually call the function. Instances of ``genshi.core.Attrs`` are now immutable. Filters manipulating the attributes in a stream may need to be updated. Also, the ``Attrs`` class no longer automatically wraps all attribute names in ``QName`` objects, so users of the ``Attrs`` class need to do this themselves. See the documentation of the ``Attrs`` class for more information. --------------------- Upgrading from Markup --------------------- Prior to version 0.3, the name of the Genshi project was "Markup". The name change means that you will have to adjust your import statements and the namespace URI of XML templates, among other things: * The package name was changed from "markup" to "genshi". Please adjust any import statements referring to the old package name. * The namespace URI for directives in Genshi XML templates has changed from ``http://markup.edgewall.org/`` to ``http://genshi.edgewall.org/``. Please update the ``xmlns:py`` declaration in your template files accordingly. Furthermore, due to the inclusion of a text-based template language, the class:: markup.template.Template has been renamed to:: genshi.template.MarkupTemplate If you've been using the Template class directly, you'll need to update your code (a simple find/replace should do—the API itself did not change). Genshi-0.7/doc/xpath.html0000664000175000017500000001367012101251412015646 0ustar simonsimon00000000000000 Genshi: Using XPath in Genshi

Using XPath in Genshi

Genshi provides basic XPath support for matching and querying event streams.

1   Limitations

Due to the streaming nature of the processing model, Genshi uses only a subset of the XPath 1.0 language.

In particular, only the following axes are supported:

  • attribute
  • child
  • descendant
  • descendant-or-self
  • self

This means you can't use the parent, ancestor, or sibling axes in Genshi (the namespace axis isn't supported either, but what you'd ever need that for I don't know). Basically, any path expression that would require buffering of the stream is not supported.

Predicates are of course supported, but path expressions inside predicates are restricted to attribute lookups (again due to the lack of buffering).

Most of the XPath functions and operators are supported, however they (currently) only work inside predicates. The following functions are not supported:

  • count()
  • id()
  • lang()
  • last()
  • position()
  • string()
  • sum()

The mathematical operators (+, -, *, div, and mod) are not yet supported, whereas sub-expressions and the various comparison and logical operators should work as expected.

You can also use XPath variable references ($var) inside predicates.

2   Querying Streams

The Stream class provides a select(path) function that can be used to retrieve subsets of the stream:

>>> from genshi.input import XML

>>> doc = XML('''<doc>
...  <items count="4">
...       <item status="new">
...         <summary>Foo</summary>
...       </item>
...       <item status="closed">
...         <summary>Bar</summary>
...       </item>
...       <item status="closed" resolution="invalid">
...         <summary>Baz</summary>
...       </item>
...       <item status="closed" resolution="fixed">
...         <summary>Waz</summary>
...       </item>
...   </items>
... </doc>''')

>>> print(doc.select('items/item[@status="closed" and '
...     '(@resolution="invalid" or not(@resolution))]/summary/text()'))
BarBaz

3   Matching in Templates

See the directive py:match in the XML Template Language Specification.

Genshi-0.7/doc/xml-templates.html0000664000175000017500000011414612101251412017316 0ustar simonsimon00000000000000 Genshi: Genshi XML Template Language

Genshi XML Template Language

Genshi provides a XML-based template language that is heavily inspired by Kid, which in turn was inspired by a number of existing template languages, namely XSLT, TAL, and PHP.

This document describes the template language and will be most useful as reference to those developing Genshi XML templates. Templates are XML files of some kind (such as XHTML) that include processing directives (elements or attributes identified by a separate namespace) that affect how the template is rendered, and template expressions that are dynamically substituted by variable data.

See Genshi Templating Basics for general information on embedding Python code in templates.

1   Template Directives

Directives are elements and/or attributes in the template that are identified by the namespace http://genshi.edgewall.org/. They can affect how the template is rendered in a number of ways: Genshi provides directives for conditionals and looping, among others.

To use directives in a template, the namespace must be declared, which is usually done on the root element:

<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:py="http://genshi.edgewall.org/"
      lang="en">
  ...
</html>

In this example, the default namespace is set to the XHTML namespace, and the namespace for Genshi directives is bound to the prefix “py”.

All directives can be applied as attributes, and some can also be used as elements. The if directives for conditionals, for example, can be used in both ways:

<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:py="http://genshi.edgewall.org/"
      lang="en">
  ...
  <div py:if="foo">
    <p>Bar</p>
  </div>
  ...
</html>

This is basically equivalent to the following:

<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:py="http://genshi.edgewall.org/"
      lang="en">
  ...
  <py:if test="foo">
    <div>
      <p>Bar</p>
    </div>
  </py:if>
  ...
</html>

The rationale behind the second form is that directives do not always map naturally to elements in the template. In such cases, the py:strip directive can be used to strip off the unwanted element, or the directive can simply be used as an element.

1.1   Conditional Sections

1.1.1   py:if

The element and its content is only rendered if the expression evaluates to a truth value:

<div>
  <b py:if="foo">${bar}</b>
</div>

Given the data foo=True and bar='Hello' in the template context, this would produce:

<div>
  <b>Hello</b>
</div>

But setting foo=False would result in the following output:

<div>
</div>

This directive can also be used as an element:

<div>
  <py:if test="foo">
    <b>${bar}</b>
  </py:if>
</div>

1.1.2   py:choose

The py:choose directive, in combination with the directives py:when and py:otherwise provides advanced conditional processing for rendering one of several alternatives. The first matching py:when branch is rendered, or, if no py:when branch matches, the py:otherwise branch is rendered.

If the py:choose directive is empty the nested py:when directives will be tested for truth:

<div py:choose="">
  <span py:when="0 == 1">0</span>
  <span py:when="1 == 1">1</span>
  <span py:otherwise="">2</span>
</div>

This would produce the following output:

<div>
  <span>1</span>
</div>

If the py:choose directive contains an expression the nested py:when directives will be tested for equality to the parent py:choose value:

<div py:choose="1">
  <span py:when="0">0</span>
  <span py:when="1">1</span>
  <span py:otherwise="">2</span>
</div>

This would produce the following output:

<div>
  <span>1</span>
</div>

These directives can also be used as elements:

<py:choose test="1">
  <py:when test="0">0</py:when>
  <py:when test="1">1</py:when>
  <py:otherwise>2</py:otherwise>
</py:choose>

1.2   Looping

1.2.1   py:for

The element is repeated for every item in an iterable:

<ul>
  <li py:for="item in items">${item}</li>
</ul>

Given items=[1, 2, 3] in the context data, this would produce:

<ul>
  <li>1</li><li>2</li><li>3</li>
</ul>

This directive can also be used as an element:

<ul>
  <py:for each="item in items">
    <li>${item}</li>
  </py:for>
</ul>

1.3   Snippet Reuse

1.3.1   py:def

The py:def directive can be used to create macros, i.e. snippets of template code that have a name and optionally some parameters, and that can be inserted in other places:

<div>
  <p py:def="greeting(name)" class="greeting">
    Hello, ${name}!
  </p>
  ${greeting('world')}
  ${greeting('everyone else')}
</div>

The above would be rendered to:

<div>
  <p class="greeting">
    Hello, world!
  </p>
  <p class="greeting">
    Hello, everyone else!
  </p>
</div>

If a macro doesn't require parameters, it can be defined without the parenthesis. For example:

<div>
  <p py:def="greeting" class="greeting">
    Hello, world!
  </p>
  ${greeting()}
</div>

The above would be rendered to:

<div>
  <p class="greeting">
    Hello, world!
  </p>
</div>

This directive can also be used as an element:

<div>
  <py:def function="greeting(name)">
    <p class="greeting">Hello, ${name}!</p>
  </py:def>
</div>

1.3.2   py:match

This directive defines a match template: given an XPath expression, it replaces any element in the template that matches the expression with its own content.

For example, the match template defined in the following template matches any element with the tag name “greeting”:

<div>
  <span py:match="greeting">
    Hello ${select('@name')}
  </span>
  <greeting name="Dude" />
</div>

This would result in the following output:

<div>
  <span>
    Hello Dude
  </span>
</div>

Inside the body of a py:match directive, the select(path) function is made available so that parts or all of the original element can be incorporated in the output of the match template. See Using XPath for more information about this function.

Match templates are applied both to the original markup as well to the generated markup. The order in which they are applied depends on the order they are declared in the template source: a match template defined after another match template is applied to the output generated by the first match template. The match templates basically form a pipeline.

This directive can also be used as an element:

<div>
  <py:match path="greeting">
    <span>Hello ${select('@name')}</span>
  </py:match>
  <greeting name="Dude" />
</div>

When used this way, the py:match directive can also be annotated with a couple of optimization hints. For example, the following informs the matching engine that the match should only be applied once:

<py:match path="body" once="true">
  <body py:attrs="select('@*')">
    <div id="header">...</div>
    ${select("*|text()")}
    <div id="footer">...</div>
  </body>
</py:match>

The following optimization hints are recognized:

Attribute Default Description
buffer true Whether the matched content should be buffered in memory. Buffering can improve performance a bit at the cost of needing more memory during rendering. Buffering is ''required'' for match templates that contain more than one invocation of the select() function. If there is only one call, and the matched content can potentially be very long, consider disabling buffering to avoid excessive memory use.
once false Whether the engine should stop looking for more matching elements after the first match. Use this on match templates that match elements that can only occur once in the stream, such as the <head> or <body> elements in an HTML template, or elements with a specific ID.
recursive true Whether the match template should be applied to its own output. Note that once implies non-recursive behavior, so this attribute only needs to be set for match templates that don't also have once set.

Note

The py:match optimization hints were added in the 0.5 release. In earlier versions, the attributes have no effect.

1.4   Variable Binding

1.4.1   py:with

The py:with directive lets you assign expressions to variables, which can be used to make expressions inside the directive less verbose and more efficient. For example, if you need use the expression author.posts more than once, and that actually results in a database query, assigning the results to a variable using this directive would probably help.

For example:

<div>
  <span py:with="y=7; z=x+10">$x $y $z</span>
</div>

Given x=42 in the context data, this would produce:

<div>
  <span>42 7 52</span>
</div>

This directive can also be used as an element:

<div>
  <py:with vars="y=7; z=x+10">$x $y $z</py:with>
</div>

Note that if a variable of the same name already existed outside of the scope of the py:with directive, it will not be overwritten. Instead, it will have the same value it had prior to the py:with assignment. Effectively, this means that variables are immutable in Genshi.

1.5   Structure Manipulation

1.5.1   py:attrs

This directive adds, modifies or removes attributes from the element:

<ul>
  <li py:attrs="foo">Bar</li>
</ul>

Given foo={'class': 'collapse'} in the template context, this would produce:

<ul>
  <li class="collapse">Bar</li>
</ul>

Attributes with the value None are omitted, so given foo={'class': None} in the context for the same template this would produce:

<ul>
  <li>Bar</li>
</ul>

This directive can only be used as an attribute.

1.5.2   py:content

This directive replaces any nested content with the result of evaluating the expression:

<ul>
  <li py:content="bar">Hello</li>
</ul>

Given bar='Bye' in the context data, this would produce:

<ul>
  <li>Bye</li>
</ul>

This directive can only be used as an attribute.

1.5.3   py:replace

This directive replaces the element itself with the result of evaluating the expression:

<div>
  <span py:replace="bar">Hello</span>
</div>

Given bar='Bye' in the context data, this would produce:

<div>
  Bye
</div>

This directive can also be used as an element (since version 0.5):

<div>
  <py:replace value="title">Placeholder</py:replace>
</div>

1.5.4   py:strip

This directive conditionally strips the top-level element from the output. When the value of the py:strip attribute evaluates to True, the element is stripped from the output:

<div>
  <div py:strip="True"><b>foo</b></div>
</div>

This would be rendered as:

<div>
  <b>foo</b>
</div>

As a shorthand, if the value of the py:strip attribute is empty, that has the same effect as using a truth value (i.e. the element is stripped).

1.6   Processing Order

It is possible to attach multiple directives to a single element, although not all combinations make sense. When multiple directives are encountered, they are processed in the following order:

  1. py:def
  2. py:match
  3. py:when
  4. py:otherwise
  5. py:for
  6. py:if
  7. py:choose
  8. py:with
  9. py:replace
  10. py:content
  11. py:attrs
  12. py:strip

2   Includes

To reuse common snippets of template code, you can include other files using XInclude.

For this, you need to declare the XInclude namespace (commonly bound to the prefix “xi”) and use the <xi:include> element where you want the external file to be pulled in:

<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:py="http://genshi.edgewall.org/"
      xmlns:xi="http://www.w3.org/2001/XInclude">
  <xi:include href="base.html" />
  ...
</html>

Include paths are relative to the filename of the template currently being processed. So if the example above was in the file "myapp/index.html" (relative to the template search path), the XInclude processor would look for the included file at "myapp/base.html". You can also use Unix-style relative paths, for example "../base.html" to look in the parent directory.

Any content included this way is inserted into the generated output instead of the <xi:include> element. The included template sees the same context data. Match templates and macros in the included template are also available to the including template after the point it was included.

By default, an error will be raised if an included file is not found. If that's not what you want, you can specify fallback content that should be used if the include fails. For example, to to make the include above fail silently, you'd write:

<xi:include href="base.html"><xi:fallback /></xi:include>

See the XInclude specification for more about fallback content. Note though that Genshi currently only supports a small subset of XInclude.

2.1   Dynamic Includes

Incudes in Genshi are fully dynamic: Just like normal attributes, the href attribute accepts expressions, and directives can be used on the <xi:include /> element just as on any other element, meaning you can do things like conditional includes:

<xi:include href="${name}.html" py:if="not in_popup"
            py:for="name in ('foo', 'bar', 'baz')" />

2.2   Including Text Templates

The parse attribute of the <xi:include> element can be used to specify whether the included template is an XML template or a text template (using the new syntax added in Genshi 0.5):

<xi:include href="myscript.js" parse="text" />

This example would load the myscript.js file as a NewTextTemplate. See text templates for details on the syntax of text templates.

3   Comments

Normal XML/HTML comment syntax can be used in templates:

<!-- this is a comment -->

However, such comments get passed through the processing pipeline and are by default included in the final output. If that's not desired, prefix the comment text with an exclamation mark:

<!-- !this is a comment too, but one that will be stripped from the output -->

Note that it does not matter whether there's whitespace before or after the exclamation mark, so the above could also be written as follows:

<!--! this is a comment too, but one that will be stripped from the output -->
Genshi-0.7/doc/upgrade.html0000664000175000017500000002506712101251412016154 0ustar simonsimon00000000000000 Genshi: Upgrading Genshi

Upgrading Genshi

1   Upgrading from Genshi 0.6.x to the development version

The Genshi development version now supports both Python 2 and Python 3.

The most noticable API change in the Genshi development version is that the default encoding in numerous places is now None (i.e. unicode) instead of UTF-8. This change was made in order to ease the transition to Python 3 where strings are unicode strings by default.

If your application relies on the default UTF-8 encoding a simple way to have it work both with Genshi 0.6.x and the development version is to specify the encoding explicitly in calls to the following classes, methods and functions:

  • genshi.HTML
  • genshi.Stream.render
  • genshi.input.HTMLParser
  • genshi.template.MarkupTemplate
  • genshi.template.TemplateLoader
  • genshi.template.TextTemplate (and genshi.template.NewTextTemplate)
  • genshi.template.OldTextTemplate

Whether you explicitly specify UTF-8 or explicitly specify None (unicode) is a matter of personal taste, although working with unicode may make porting your own application to Python 3 easier.

2   Upgrading from Genshi 0.5.x to 0.6.x

2.1   Required Python Version

Support for Python 2.3 has been dropped in this release. Python 2.4 is now the minimum version of Python required to run Genshi.

The XPath engine has been completely overhauled for this version. Some patterns that previously matched incorrectly will no longer match, and the other way around. In such cases, the XPath expressions need to be fixed in your application and templates.

3   Upgrading from Genshi 0.4.x to 0.5.x

3.1   Error Handling

The default error handling mode has been changed to "strict". This means that accessing variables not defined in the template data will now generate an immediate exception, as will accessing object attributes or dictionary keys that don't exist. If your templates rely on the old lenient behavior, you can configure Genshi to use that instead. See the documentation for details on how to do that. But be warned that lenient error handling may be removed completely in a future release.

3.2   Match Template Processing

There has also been a subtle change to how py:match templates are processed: in previous versions, all match templates would be applied to the content generated by the matching template, and only the matching template itself was applied recursively to the original content. This behavior resulted in problems with many kinds of recursive matching, and hence was changed for 0.5: now, all match templates declared before the matching template are applied to the original content, and match templates declared after the matching template are applied to the generated content. This change should not have any effect on most applications, but you may want to check your use of match templates to make sure.

3.3   Text Templates

Genshi 0.5 introduces a new, alternative syntax for text templates, which is more flexible and powerful compared to the old syntax. For backwards compatibility, this new syntax is not used by default, though it will be in a future version. It is recommended that you migrate to using this new syntax. To do so, simply rename any references in your code to TextTemplate to NewTextTemplate. To explicitly use the old syntax, use OldTextTemplate instead, so that you can be sure you'll be using the same language when the default in Genshi is changed (at least until the old implementation is completely removed).

3.4   Markup Constructor

The Markup class no longer has a specialized constructor. The old (undocumented) constructor provided a shorthand for doing positional substitutions. If you have code like this:

Markup('<b>%s</b>', name)

You must replace it by the more explicit:

Markup('<b>%s</b>') % name

3.5   Template Constructor

The constructor of the Template class and its subclasses has changed slightly: instead of the optional basedir parameter, it now expects an (also optional) filepath parameter, which specifies the absolute path to the template. You probably aren't using those constructors directly, anyway, but using the TemplateLoader API instead.

4   Upgrading from Genshi 0.3.x to 0.4.x

The modules genshi.filters and genshi.template have been refactored into packages containing multiple modules. While code using the regular APIs should continue to work without problems, you should make sure to remove any leftover traces of the files filters.py and template.py in the genshi package on the installation path (including the corresponding .pyc files). This is not necessary when Genshi was installed as a Python egg.

Results of evaluating template expressions are no longer implicitly called if they are callable. If you have been using that feature, you will need to add the parenthesis to actually call the function.

Instances of genshi.core.Attrs are now immutable. Filters manipulating the attributes in a stream may need to be updated. Also, the Attrs class no longer automatically wraps all attribute names in QName objects, so users of the Attrs class need to do this themselves. See the documentation of the Attrs class for more information.

5   Upgrading from Markup

Prior to version 0.3, the name of the Genshi project was "Markup". The name change means that you will have to adjust your import statements and the namespace URI of XML templates, among other things:

  • The package name was changed from "markup" to "genshi". Please adjust any import statements referring to the old package name.
  • The namespace URI for directives in Genshi XML templates has changed from http://markup.edgewall.org/ to http://genshi.edgewall.org/. Please update the xmlns:py declaration in your template files accordingly.

Furthermore, due to the inclusion of a text-based template language, the class:

markup.template.Template

has been renamed to:

genshi.template.MarkupTemplate

If you've been using the Template class directly, you'll need to update your code (a simple find/replace should do—the API itself did not change).

Genshi-0.7/doc/filters.txt0000664000175000017500000002071212101246063016046 0ustar simonsimon00000000000000.. -*- mode: rst; encoding: utf-8 -*- ============== Stream Filters ============== `Markup Streams`_ showed how to write filters and how they are applied to markup streams. This page describes the features of the various filters that come with Genshi itself. .. _`Markup Streams`: streams.html .. contents:: Contents :depth: 1 .. sectnum:: HTML Form Filler ================ The filter ``genshi.filters.html.HTMLFormFiller`` can automatically populate an HTML form from values provided as a simple dictionary. When using this filter, you can basically omit any ``value``, ``selected``, or ``checked`` attributes from form controls in your templates, and let the filter do all that work for you. ``HTMLFormFiller`` takes a dictionary of data to populate the form with, where the keys should match the names of form elements, and the values determine the values of those controls. For example: .. code-block:: pycon >>> from genshi.filters import HTMLFormFiller >>> from genshi.template import MarkupTemplate >>> template = MarkupTemplate("""
...

...
...
... ...

...
""") >>> filler = HTMLFormFiller(data=dict(username='john', remember=True)) >>> print(template.generate() | filler)



.. note:: This processing is done without in any way reparsing the template output. As any stream filter it operates after the template output is generated but *before* that output is actually serialized. The filter will of course also handle radio buttons as well as ``

""") | HTMLFormFiller() self.assertEquals("""

""") | HTMLFormFiller(data={'foo': 'bar'}) self.assertEquals("""

""", html.render()) def test_fill_textarea_multi_value(self): html = HTML(u"""

""") | HTMLFormFiller(data={'foo': ['bar']}) self.assertEquals("""

""", html.render()) def test_fill_textarea_multiple(self): # Ensure that the subsequent textarea doesn't get the data from the # first html = HTML(u"""

""") | HTMLFormFiller(data={'foo': 'Some text'}) self.assertEquals("""

""") | HTMLFormFiller(data={'foo': 'Some text'}) self.assertEquals("""

""", html.render()) def test_fill_input_checkbox_single_value_auto_no_value(self): html = HTML(u"""

""") | HTMLFormFiller() self.assertEquals("""

""", html.render()) def test_fill_input_checkbox_single_value_auto(self): html = HTML(u"""

""") self.assertEquals("""

""", (html | HTMLFormFiller(data={'foo': ''})).render()) self.assertEquals("""

""", (html | HTMLFormFiller(data={'foo': 'on'})).render()) def test_fill_input_checkbox_single_value_defined(self): html = HTML("""

""", encoding='ascii') self.assertEquals("""

""", (html | HTMLFormFiller(data={'foo': '1'})).render()) self.assertEquals("""

""", (html | HTMLFormFiller(data={'foo': '2'})).render()) def test_fill_input_checkbox_multi_value_auto(self): html = HTML("""

""", encoding='ascii') self.assertEquals("""

""", (html | HTMLFormFiller(data={'foo': []})).render()) self.assertEquals("""

""", (html | HTMLFormFiller(data={'foo': ['on']})).render()) def test_fill_input_checkbox_multi_value_defined(self): html = HTML(u"""

""") self.assertEquals("""

""", (html | HTMLFormFiller(data={'foo': ['1']})).render()) self.assertEquals("""

""", (html | HTMLFormFiller(data={'foo': ['2']})).render()) def test_fill_input_radio_no_value(self): html = HTML(u"""

""") | HTMLFormFiller() self.assertEquals("""

""", html.render()) def test_fill_input_radio_single_value(self): html = HTML(u"""

""") self.assertEquals("""

""", (html | HTMLFormFiller(data={'foo': '1'})).render()) self.assertEquals("""

""", (html | HTMLFormFiller(data={'foo': '2'})).render()) def test_fill_input_radio_multi_value(self): html = HTML(u"""

""") self.assertEquals("""

""", (html | HTMLFormFiller(data={'foo': ['1']})).render()) self.assertEquals("""

""", (html | HTMLFormFiller(data={'foo': ['2']})).render()) def test_fill_input_radio_empty_string(self): html = HTML(u"""

""") self.assertEquals("""

""", (html | HTMLFormFiller(data={'foo': ''})).render()) def test_fill_input_radio_multi_empty_string(self): html = HTML(u"""

""") self.assertEquals("""

""", (html | HTMLFormFiller(data={'foo': ['']})).render()) def test_fill_select_no_value_auto(self): html = HTML(u"""

""") | HTMLFormFiller() self.assertEquals("""

""", html.render()) def test_fill_select_no_value_defined(self): html = HTML(u"""

""") | HTMLFormFiller() self.assertEquals("""

""", html.render()) def test_fill_select_single_value_auto(self): html = HTML(u"""

""") | HTMLFormFiller(data={'foo': '1'}) self.assertEquals("""

""", html.render()) def test_fill_select_single_value_defined(self): html = HTML(u"""

""") | HTMLFormFiller(data={'foo': '1'}) self.assertEquals("""

""", html.render()) def test_fill_select_multi_value_auto(self): html = HTML(u"""

""") | HTMLFormFiller(data={'foo': ['1', '3']}) self.assertEquals("""

""", html.render()) def test_fill_select_multi_value_defined(self): html = HTML(u"""

""") | HTMLFormFiller(data={'foo': ['1', '3']}) self.assertEquals("""

""", html.render()) def test_fill_option_segmented_text(self): html = MarkupTemplate(u"""
""").generate(x=1) | HTMLFormFiller(data={'foo': '1'}) self.assertEquals(u"""
""", html.render()) def test_fill_option_segmented_text_no_value(self): html = MarkupTemplate("""
""").generate(x=1) | HTMLFormFiller(data={'foo': 'foo 1 bar'}) self.assertEquals("""
""", html.render()) def test_fill_option_unicode_value(self): html = HTML(u"""
""") | HTMLFormFiller(data={'foo': u'ö'}) self.assertEquals(u"""
""", html.render(encoding=None)) def test_fill_input_password_disabled(self): html = HTML(u"""

""") | HTMLFormFiller(data={'pass': 'bar'}) self.assertEquals("""

""", html.render()) def test_fill_input_password_enabled(self): html = HTML(u"""

""") | HTMLFormFiller(data={'pass': '1234'}, passwords=True) self.assertEquals("""

""", html.render()) def StyleSanitizer(): safe_attrs = HTMLSanitizer.SAFE_ATTRS | frozenset(['style']) return HTMLSanitizer(safe_attrs=safe_attrs) class HTMLSanitizerTestCase(unittest.TestCase): def assert_parse_error_or_equal(self, expected, exploit): try: html = HTML(exploit) except ParseError: return self.assertEquals(expected, (html | HTMLSanitizer()).render()) def test_sanitize_unchanged(self): html = HTML(u'fo
o') self.assertEquals('fo
o', (html | HTMLSanitizer()).render()) html = HTML(u'foo') self.assertEquals('foo', (html | HTMLSanitizer()).render()) def test_sanitize_escape_text(self): html = HTML(u'fo&') self.assertEquals('fo&', (html | HTMLSanitizer()).render()) html = HTML(u'<foo>') self.assertEquals('<foo>', (html | HTMLSanitizer()).render()) def test_sanitize_entityref_text(self): html = HTML(u'foö') self.assertEquals(u'foö', (html | HTMLSanitizer()).render(encoding=None)) def test_sanitize_escape_attr(self): html = HTML(u'
') self.assertEquals('
', (html | HTMLSanitizer()).render()) def test_sanitize_close_empty_tag(self): html = HTML(u'fo
o') self.assertEquals('fo
o', (html | HTMLSanitizer()).render()) def test_sanitize_invalid_entity(self): html = HTML(u'&junk;') self.assertEquals('&junk;', (html | HTMLSanitizer()).render()) def test_sanitize_remove_script_elem(self): html = HTML(u'') self.assertEquals('', (html | HTMLSanitizer()).render()) html = HTML(u'') self.assertEquals('', (html | HTMLSanitizer()).render()) src = u'alert("foo")' self.assert_parse_error_or_equal('<SCR\x00IPT>alert("foo")', src) src = u'' self.assert_parse_error_or_equal('<SCRIPT&XYZ; ' 'SRC="http://example.com/">', src) def test_sanitize_remove_onclick_attr(self): html = HTML(u'
') self.assertEquals('
', (html | HTMLSanitizer()).render()) def test_sanitize_remove_input_password(self): html = HTML(u'
') self.assertEquals('
', (html | HTMLSanitizer()).render()) def test_sanitize_remove_comments(self): html = HTML(u'''
''') self.assertEquals('
', (html | HTMLSanitizer()).render()) def test_sanitize_remove_style_scripts(self): sanitizer = StyleSanitizer() # Inline style with url() using javascript: scheme html = HTML(u'
') self.assertEquals('
', (html | sanitizer).render()) # Inline style with url() using javascript: scheme, using control char html = HTML(u'
') self.assertEquals('
', (html | sanitizer).render()) # Inline style with url() using javascript: scheme, in quotes html = HTML(u'
') self.assertEquals('
', (html | sanitizer).render()) # IE expressions in CSS not allowed html = HTML(u'
') self.assertEquals('
', (html | sanitizer).render()) html = HTML(u'
') self.assertEquals('
', (html | sanitizer).render()) html = HTML(u'
') self.assertEquals('
', (html | sanitizer).render()) # Inline style with url() using javascript: scheme, using unicode # escapes html = HTML(u'
') self.assertEquals('
', (html | sanitizer).render()) html = HTML(u'
') self.assertEquals('
', (html | sanitizer).render()) html = HTML(u'
') self.assertEquals('
', (html | sanitizer).render()) html = HTML(u'
') self.assertEquals('
', (html | sanitizer).render()) html = HTML(u'
') self.assertEquals('
', (html | sanitizer).render()) def test_sanitize_remove_style_phishing(self): sanitizer = StyleSanitizer() # The position property is not allowed html = HTML(u'
') self.assertEquals('
', (html | sanitizer).render()) # Normal margins get passed through html = HTML(u'
') self.assertEquals('
', (html | sanitizer).render()) # But not negative margins html = HTML(u'
') self.assertEquals('
', (html | sanitizer).render()) html = HTML(u'
') self.assertEquals('
', (html | sanitizer).render()) html = HTML(u'
') self.assertEquals('
', (html | sanitizer).render()) def test_sanitize_remove_src_javascript(self): html = HTML(u'') self.assertEquals('', (html | HTMLSanitizer()).render()) # Case-insensitive protocol matching html = HTML(u'') self.assertEquals('', (html | HTMLSanitizer()).render()) # Grave accents (not parsed) src = u'' self.assert_parse_error_or_equal('', src) # Protocol encoded using UTF-8 numeric entities html = HTML(u'') self.assertEquals('', (html | HTMLSanitizer()).render()) # Protocol encoded using UTF-8 numeric entities without a semicolon # (which is allowed because the max number of digits is used) html = HTML(u'') self.assertEquals('', (html | HTMLSanitizer()).render()) # Protocol encoded using UTF-8 numeric hex entities without a semicolon # (which is allowed because the max number of digits is used) html = HTML(u'') self.assertEquals('', (html | HTMLSanitizer()).render()) # Embedded tab character in protocol html = HTML(u'') self.assertEquals('', (html | HTMLSanitizer()).render()) # Embedded tab character in protocol, but encoded this time html = HTML(u'') self.assertEquals('', (html | HTMLSanitizer()).render()) def test_sanitize_expression(self): html = HTML(ur'
XSS
') self.assertEqual('
XSS
', unicode(html | StyleSanitizer())) def test_capital_expression(self): html = HTML(ur'
XSS
') self.assertEqual('
XSS
', unicode(html | StyleSanitizer())) def test_sanitize_url_with_javascript(self): html = HTML(u'
' u'XSS
') self.assertEqual('
XSS
', unicode(html | StyleSanitizer())) def test_sanitize_capital_url_with_javascript(self): html = HTML(u'
' u'XSS
') self.assertEqual('
XSS
', unicode(html | StyleSanitizer())) def test_sanitize_unicode_escapes(self): html = HTML(ur'
' ur'XSS
') self.assertEqual('
XSS
', unicode(html | StyleSanitizer())) def test_sanitize_backslash_without_hex(self): html = HTML(ur'
XSS
') self.assertEqual('
XSS
', unicode(html | StyleSanitizer())) html = HTML(ur'
XSS
') self.assertEqual(r'
' 'XSS
', unicode(html | StyleSanitizer())) def test_sanitize_unsafe_props(self): html = HTML(u'
XSS
') self.assertEqual('
XSS
', unicode(html | StyleSanitizer())) html = HTML(u'
XSS
') self.assertEqual('
XSS
', unicode(html | StyleSanitizer())) html = HTML(u'
' u'XSS
') self.assertEqual('
XSS
', unicode(html | StyleSanitizer())) html = HTML(u"""
XSS
""") self.assertEqual('
XSS
', unicode(html | StyleSanitizer())) html = HTML(u"""
XSS
""") self.assertEqual('
XSS
', unicode(html | StyleSanitizer())) def test_sanitize_negative_margin(self): html = HTML(u'
XSS
') self.assertEqual('
XSS
', unicode(html | StyleSanitizer())) html = HTML(u'
XSS
') self.assertEqual('
XSS
', unicode(html | StyleSanitizer())) def test_sanitize_css_hack(self): html = HTML(u'
XSS
') self.assertEqual('
XSS
', unicode(html | StyleSanitizer())) html = HTML(u'
XSS
') self.assertEqual('
XSS
', unicode(html | StyleSanitizer())) def test_sanitize_property_name(self): html = HTML(u'
prop
') self.assertEqual('
prop
', unicode(html | StyleSanitizer())) def test_sanitize_unicode_expression(self): # Fullwidth small letters html = HTML(u'
' u'XSS
') self.assertEqual('
XSS
', unicode(html | StyleSanitizer())) # Fullwidth capital letters html = HTML(u'
' u'XSS
') self.assertEqual('
XSS
', unicode(html | StyleSanitizer())) # IPA extensions html = HTML(u'
' u'XSS
') self.assertEqual('
XSS
', unicode(html | StyleSanitizer())) def test_sanitize_unicode_url(self): # IPA extensions html = HTML(u'
' u'XSS
') self.assertEqual('
XSS
', unicode(html | StyleSanitizer())) def suite(): suite = unittest.TestSuite() suite.addTest(doctest.DocTestSuite(HTMLFormFiller.__module__)) suite.addTest(unittest.makeSuite(HTMLFormFillerTestCase, 'test')) suite.addTest(unittest.makeSuite(HTMLSanitizerTestCase, 'test')) return suite if __name__ == '__main__': unittest.main(defaultTest='suite') Genshi-0.7/genshi/filters/tests/__init__.py0000664000175000017500000000156212101246064021273 0ustar simonsimon00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2007-2008 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which # you should have received as part of this distribution. The terms # are also available at http://genshi.edgewall.org/wiki/License. # # This software consists of voluntary contributions made by many # individuals. For the exact contribution history, see the revision # history and logs, available at http://genshi.edgewall.org/log/. import doctest import unittest def suite(): from genshi.filters.tests import test_html, i18n, transform suite = unittest.TestSuite() suite.addTest(test_html.suite()) suite.addTest(i18n.suite()) if hasattr(doctest, 'NORMALIZE_WHITESPACE'): suite.addTest(transform.suite()) return suite if __name__ == '__main__': unittest.main(defaultTest='suite') Genshi-0.7/genshi/filters/__init__.py0000664000175000017500000000134712101246064020132 0ustar simonsimon00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2007-2009 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which # you should have received as part of this distribution. The terms # are also available at http://genshi.edgewall.org/wiki/License. # # This software consists of voluntary contributions made by many # individuals. For the exact contribution history, see the revision # history and logs, available at http://genshi.edgewall.org/log/. """Implementation of a number of stream filters.""" from genshi.filters.html import HTMLFormFiller, HTMLSanitizer from genshi.filters.i18n import Translator from genshi.filters.transform import Transformer __docformat__ = 'restructuredtext en' Genshi-0.7/genshi/filters/html.py0000664000175000017500000005441512101246064017343 0ustar simonsimon00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2006-2009 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which # you should have received as part of this distribution. The terms # are also available at http://genshi.edgewall.org/wiki/License. # # This software consists of voluntary contributions made by many # individuals. For the exact contribution history, see the revision # history and logs, available at http://genshi.edgewall.org/log/. """Implementation of a number of stream filters.""" try: any except NameError: from genshi.util import any import re from genshi.core import Attrs, QName, stripentities from genshi.core import END, START, TEXT, COMMENT __all__ = ['HTMLFormFiller', 'HTMLSanitizer'] __docformat__ = 'restructuredtext en' class HTMLFormFiller(object): """A stream filter that can populate HTML forms from a dictionary of values. >>> from genshi.input import HTML >>> html = HTML(''' ...

... ''', encoding='utf-8') >>> filler = HTMLFormFiller(data={'foo': 'bar'}) >>> print(html | filler)

""" # TODO: only select the first radio button, and the first select option # (if not in a multiple-select) # TODO: only apply to elements in the XHTML namespace (or no namespace)? def __init__(self, name=None, id=None, data=None, passwords=False): """Create the filter. :param name: The name of the form that should be populated. If this parameter is given, only forms where the ``name`` attribute value matches the parameter are processed. :param id: The ID of the form that should be populated. If this parameter is given, only forms where the ``id`` attribute value matches the parameter are processed. :param data: The dictionary of form values, where the keys are the names of the form fields, and the values are the values to fill in. :param passwords: Whether password input fields should be populated. This is off by default for security reasons (for example, a password may end up in the browser cache) :note: Changed in 0.5.2: added the `passwords` option """ self.name = name self.id = id if data is None: data = {} self.data = data self.passwords = passwords def __call__(self, stream): """Apply the filter to the given stream. :param stream: the markup event stream to filter """ in_form = in_select = in_option = in_textarea = False select_value = option_value = textarea_value = None option_start = None option_text = [] no_option_value = False for kind, data, pos in stream: if kind is START: tag, attrs = data tagname = tag.localname if tagname == 'form' and ( self.name and attrs.get('name') == self.name or self.id and attrs.get('id') == self.id or not (self.id or self.name)): in_form = True elif in_form: if tagname == 'input': type = attrs.get('type', '').lower() if type in ('checkbox', 'radio'): name = attrs.get('name') if name and name in self.data: value = self.data[name] declval = attrs.get('value') checked = False if isinstance(value, (list, tuple)): if declval is not None: checked = declval in [unicode(v) for v in value] else: checked = any(value) else: if declval is not None: checked = declval == unicode(value) elif type == 'checkbox': checked = bool(value) if checked: attrs |= [(QName('checked'), 'checked')] elif 'checked' in attrs: attrs -= 'checked' elif type in ('', 'hidden', 'text') \ or type == 'password' and self.passwords: name = attrs.get('name') if name and name in self.data: value = self.data[name] if isinstance(value, (list, tuple)): value = value[0] if value is not None: attrs |= [ (QName('value'), unicode(value)) ] elif tagname == 'select': name = attrs.get('name') if name in self.data: select_value = self.data[name] in_select = True elif tagname == 'textarea': name = attrs.get('name') if name in self.data: textarea_value = self.data.get(name) if isinstance(textarea_value, (list, tuple)): textarea_value = textarea_value[0] in_textarea = True elif in_select and tagname == 'option': option_start = kind, data, pos option_value = attrs.get('value') if option_value is None: no_option_value = True option_value = '' in_option = True continue yield kind, (tag, attrs), pos elif in_form and kind is TEXT: if in_select and in_option: if no_option_value: option_value += data option_text.append((kind, data, pos)) continue elif in_textarea: continue yield kind, data, pos elif in_form and kind is END: tagname = data.localname if tagname == 'form': in_form = False elif tagname == 'select': in_select = False select_value = None elif in_select and tagname == 'option': if isinstance(select_value, (tuple, list)): selected = option_value in [unicode(v) for v in select_value] else: selected = option_value == unicode(select_value) okind, (tag, attrs), opos = option_start if selected: attrs |= [(QName('selected'), 'selected')] elif 'selected' in attrs: attrs -= 'selected' yield okind, (tag, attrs), opos if option_text: for event in option_text: yield event in_option = False no_option_value = False option_start = option_value = None option_text = [] elif in_textarea and tagname == 'textarea': if textarea_value: yield TEXT, unicode(textarea_value), pos textarea_value = None in_textarea = False yield kind, data, pos else: yield kind, data, pos class HTMLSanitizer(object): """A filter that removes potentially dangerous HTML tags and attributes from the stream. >>> from genshi import HTML >>> html = HTML('
', encoding='utf-8') >>> print(html | HTMLSanitizer())
The default set of safe tags and attributes can be modified when the filter is instantiated. For example, to allow inline ``style`` attributes, the following instantation would work: >>> html = HTML('
', encoding='utf-8') >>> sanitizer = HTMLSanitizer(safe_attrs=HTMLSanitizer.SAFE_ATTRS | set(['style'])) >>> print(html | sanitizer)
Note that even in this case, the filter *does* attempt to remove dangerous constructs from style attributes: >>> html = HTML('
', encoding='utf-8') >>> print(html | sanitizer)
This handles HTML entities, unicode escapes in CSS and Javascript text, as well as a lot of other things. However, the style tag is still excluded by default because it is very hard for such sanitizing to be completely safe, especially considering how much error recovery current web browsers perform. It also does some basic filtering of CSS properties that may be used for typical phishing attacks. For more sophisticated filtering, this class provides a couple of hooks that can be overridden in sub-classes. :warn: Note that this special processing of CSS is currently only applied to style attributes, **not** style elements. """ SAFE_TAGS = frozenset(['a', 'abbr', 'acronym', 'address', 'area', 'b', 'big', 'blockquote', 'br', 'button', 'caption', 'center', 'cite', 'code', 'col', 'colgroup', 'dd', 'del', 'dfn', 'dir', 'div', 'dl', 'dt', 'em', 'fieldset', 'font', 'form', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'hr', 'i', 'img', 'input', 'ins', 'kbd', 'label', 'legend', 'li', 'map', 'menu', 'ol', 'optgroup', 'option', 'p', 'pre', 'q', 's', 'samp', 'select', 'small', 'span', 'strike', 'strong', 'sub', 'sup', 'table', 'tbody', 'td', 'textarea', 'tfoot', 'th', 'thead', 'tr', 'tt', 'u', 'ul', 'var']) SAFE_ATTRS = frozenset(['abbr', 'accept', 'accept-charset', 'accesskey', 'action', 'align', 'alt', 'axis', 'bgcolor', 'border', 'cellpadding', 'cellspacing', 'char', 'charoff', 'charset', 'checked', 'cite', 'class', 'clear', 'cols', 'colspan', 'color', 'compact', 'coords', 'datetime', 'dir', 'disabled', 'enctype', 'for', 'frame', 'headers', 'height', 'href', 'hreflang', 'hspace', 'id', 'ismap', 'label', 'lang', 'longdesc', 'maxlength', 'media', 'method', 'multiple', 'name', 'nohref', 'noshade', 'nowrap', 'prompt', 'readonly', 'rel', 'rev', 'rows', 'rowspan', 'rules', 'scope', 'selected', 'shape', 'size', 'span', 'src', 'start', 'summary', 'tabindex', 'target', 'title', 'type', 'usemap', 'valign', 'value', 'vspace', 'width']) SAFE_CSS = frozenset([ # CSS 3 properties 'background', 'background-attachment', 'background-color', 'background-image', 'background-position', 'background-repeat', 'border', 'border-bottom', 'border-bottom-color', 'border-bottom-style', 'border-bottom-width', 'border-collapse', 'border-color', 'border-left', 'border-left-color', 'border-left-style', 'border-left-width', 'border-right', 'border-right-color', 'border-right-style', 'border-right-width', 'border-spacing', 'border-style', 'border-top', 'border-top-color', 'border-top-style', 'border-top-width', 'border-width', 'bottom', 'caption-side', 'clear', 'clip', 'color', 'content', 'counter-increment', 'counter-reset', 'cursor', 'direction', 'display', 'empty-cells', 'float', 'font', 'font-family', 'font-size', 'font-style', 'font-variant', 'font-weight', 'height', 'left', 'letter-spacing', 'line-height', 'list-style', 'list-style-image', 'list-style-position', 'list-style-type', 'margin', 'margin-bottom', 'margin-left', 'margin-right', 'margin-top', 'max-height', 'max-width', 'min-height', 'min-width', 'opacity', 'orphans', 'outline', 'outline-color', 'outline-style', 'outline-width', 'overflow', 'padding', 'padding-bottom', 'padding-left', 'padding-right', 'padding-top', 'page-break-after', 'page-break-before', 'page-break-inside', 'quotes', 'right', 'table-layout', 'text-align', 'text-decoration', 'text-indent', 'text-transform', 'top', 'unicode-bidi', 'vertical-align', 'visibility', 'white-space', 'widows', 'width', 'word-spacing', 'z-index', ]) SAFE_SCHEMES = frozenset(['file', 'ftp', 'http', 'https', 'mailto', None]) URI_ATTRS = frozenset(['action', 'background', 'dynsrc', 'href', 'lowsrc', 'src']) def __init__(self, safe_tags=SAFE_TAGS, safe_attrs=SAFE_ATTRS, safe_schemes=SAFE_SCHEMES, uri_attrs=URI_ATTRS, safe_css=SAFE_CSS): """Create the sanitizer. The exact set of allowed elements and attributes can be configured. :param safe_tags: a set of tag names that are considered safe :param safe_attrs: a set of attribute names that are considered safe :param safe_schemes: a set of URI schemes that are considered safe :param uri_attrs: a set of names of attributes that contain URIs """ self.safe_tags = safe_tags # The set of tag names that are considered safe. self.safe_attrs = safe_attrs # The set of attribute names that are considered safe. self.safe_css = safe_css # The set of CSS properties that are considered safe. self.uri_attrs = uri_attrs # The set of names of attributes that may contain URIs. self.safe_schemes = safe_schemes # The set of URI schemes that are considered safe. # IE6 _EXPRESSION_SEARCH = re.compile(u""" [eE \uFF25 # FULLWIDTH LATIN CAPITAL LETTER E \uFF45 # FULLWIDTH LATIN SMALL LETTER E ] [xX \uFF38 # FULLWIDTH LATIN CAPITAL LETTER X \uFF58 # FULLWIDTH LATIN SMALL LETTER X ] [pP \uFF30 # FULLWIDTH LATIN CAPITAL LETTER P \uFF50 # FULLWIDTH LATIN SMALL LETTER P ] [rR \u0280 # LATIN LETTER SMALL CAPITAL R \uFF32 # FULLWIDTH LATIN CAPITAL LETTER R \uFF52 # FULLWIDTH LATIN SMALL LETTER R ] [eE \uFF25 # FULLWIDTH LATIN CAPITAL LETTER E \uFF45 # FULLWIDTH LATIN SMALL LETTER E ] [sS \uFF33 # FULLWIDTH LATIN CAPITAL LETTER S \uFF53 # FULLWIDTH LATIN SMALL LETTER S ]{2} [iI \u026A # LATIN LETTER SMALL CAPITAL I \uFF29 # FULLWIDTH LATIN CAPITAL LETTER I \uFF49 # FULLWIDTH LATIN SMALL LETTER I ] [oO \uFF2F # FULLWIDTH LATIN CAPITAL LETTER O \uFF4F # FULLWIDTH LATIN SMALL LETTER O ] [nN \u0274 # LATIN LETTER SMALL CAPITAL N \uFF2E # FULLWIDTH LATIN CAPITAL LETTER N \uFF4E # FULLWIDTH LATIN SMALL LETTER N ] """, re.VERBOSE).search # IE6 # 7) Particular bit of Unicode characters _URL_FINDITER = re.compile( u'[Uu][Rr\u0280][Ll\u029F]\s*\(([^)]+)').finditer def __call__(self, stream): """Apply the filter to the given stream. :param stream: the markup event stream to filter """ waiting_for = None for kind, data, pos in stream: if kind is START: if waiting_for: continue tag, attrs = data if not self.is_safe_elem(tag, attrs): waiting_for = tag continue new_attrs = [] for attr, value in attrs: value = stripentities(value) if attr not in self.safe_attrs: continue elif attr in self.uri_attrs: # Don't allow URI schemes such as "javascript:" if not self.is_safe_uri(value): continue elif attr == 'style': # Remove dangerous CSS declarations from inline styles decls = self.sanitize_css(value) if not decls: continue value = '; '.join(decls) new_attrs.append((attr, value)) yield kind, (tag, Attrs(new_attrs)), pos elif kind is END: tag = data if waiting_for: if waiting_for == tag: waiting_for = None else: yield kind, data, pos elif kind is not COMMENT: if not waiting_for: yield kind, data, pos def is_safe_css(self, propname, value): """Determine whether the given css property declaration is to be considered safe for inclusion in the output. :param propname: the CSS property name :param value: the value of the property :return: whether the property value should be considered safe :rtype: bool :since: version 0.6 """ if propname not in self.safe_css: return False if propname.startswith('margin') and '-' in value: # Negative margins can be used for phishing return False return True def is_safe_elem(self, tag, attrs): """Determine whether the given element should be considered safe for inclusion in the output. :param tag: the tag name of the element :type tag: QName :param attrs: the element attributes :type attrs: Attrs :return: whether the element should be considered safe :rtype: bool :since: version 0.6 """ if tag not in self.safe_tags: return False if tag.localname == 'input': input_type = attrs.get('type', '').lower() if input_type == 'password': return False return True def is_safe_uri(self, uri): """Determine whether the given URI is to be considered safe for inclusion in the output. The default implementation checks whether the scheme of the URI is in the set of allowed URIs (`safe_schemes`). >>> sanitizer = HTMLSanitizer() >>> sanitizer.is_safe_uri('http://example.org/') True >>> sanitizer.is_safe_uri('javascript:alert(document.cookie)') False :param uri: the URI to check :return: `True` if the URI can be considered safe, `False` otherwise :rtype: `bool` :since: version 0.4.3 """ if '#' in uri: uri = uri.split('#', 1)[0] # Strip out the fragment identifier if ':' not in uri: return True # This is a relative URI chars = [char for char in uri.split(':', 1)[0] if char.isalnum()] return ''.join(chars).lower() in self.safe_schemes def sanitize_css(self, text): """Remove potentially dangerous property declarations from CSS code. In particular, properties using the CSS ``url()`` function with a scheme that is not considered safe are removed: >>> sanitizer = HTMLSanitizer() >>> sanitizer.sanitize_css(u''' ... background: url(javascript:alert("foo")); ... color: #000; ... ''') [u'color: #000'] Also, the proprietary Internet Explorer function ``expression()`` is always stripped: >>> sanitizer.sanitize_css(u''' ... background: #fff; ... color: #000; ... width: e/**/xpression(alert("foo")); ... ''') [u'background: #fff', u'color: #000'] :param text: the CSS text; this is expected to be `unicode` and to not contain any character or numeric references :return: a list of declarations that are considered safe :rtype: `list` :since: version 0.4.3 """ decls = [] text = self._strip_css_comments(self._replace_unicode_escapes(text)) for decl in text.split(';'): decl = decl.strip() if not decl: continue try: propname, value = decl.split(':', 1) except ValueError: continue if not self.is_safe_css(propname.strip().lower(), value.strip()): continue is_evil = False if self._EXPRESSION_SEARCH(value): is_evil = True for match in self._URL_FINDITER(value): if not self.is_safe_uri(match.group(1)): is_evil = True break if not is_evil: decls.append(decl.strip()) return decls _NORMALIZE_NEWLINES = re.compile(r'\r\n').sub _UNICODE_ESCAPE = re.compile( r"""\\([0-9a-fA-F]{1,6})\s?|\\([^\r\n\f0-9a-fA-F'"{};:()#*])""", re.UNICODE).sub def _replace_unicode_escapes(self, text): def _repl(match): t = match.group(1) if t: return unichr(int(t, 16)) t = match.group(2) if t == '\\': return r'\\' else: return t return self._UNICODE_ESCAPE(_repl, self._NORMALIZE_NEWLINES('\n', text)) _CSS_COMMENTS = re.compile(r'/\*.*?\*/').sub def _strip_css_comments(self, text): return self._CSS_COMMENTS('', text) Genshi-0.7/genshi/compat.py0000664000175000017500000000664212101246064016211 0ustar simonsimon00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2006-2009 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which # you should have received as part of this distribution. The terms # are also available at http://genshi.edgewall.org/wiki/License. # # This software consists of voluntary contributions made by many # individuals. For the exact contribution history, see the revision # history and logs, available at http://genshi.edgewall.org/log/. """Various Python version compatibility classes and functions.""" import sys from types import CodeType IS_PYTHON2 = (sys.version_info[0] == 2) # This function should only be called in Python 2, and will fail in Python 3 if IS_PYTHON2: def stringrepr(string): ascii = string.encode('ascii', 'backslashreplace') quoted = "'" + ascii.replace("'", "\\'") + "'" if len(ascii) > len(string): return 'u' + quoted return quoted else: def stringrepr(string): raise RuntimeError( 'Python 2 compatibility function. Not usable in Python 3.') # We need to differentiate between StringIO and BytesIO in places if IS_PYTHON2: from StringIO import StringIO try: from cStringIO import StringIO as BytesIO except ImportError: BytesIO = StringIO else: from io import StringIO, BytesIO # We want to test bytestring input to some stuff. if IS_PYTHON2: def wrapped_bytes(bstr): assert bstr.startswith('b') return bstr[1:] else: def wrapped_bytes(bstr): assert bstr.startswith('b') return bstr # We do some scary stuff with CodeType() in template/eval.py if IS_PYTHON2: def get_code_params(code): return (code.co_nlocals, code.co_stacksize, code.co_flags, code.co_code, code.co_consts, code.co_names, code.co_varnames, code.co_filename, code.co_name, code.co_firstlineno, code.co_lnotab, (), ()) def build_code_chunk(code, filename, name, lineno): return CodeType(0, code.co_nlocals, code.co_stacksize, code.co_flags | 0x0040, code.co_code, code.co_consts, code.co_names, code.co_varnames, filename, name, lineno, code.co_lnotab, (), ()) else: def get_code_params(code): return (code.co_nlocals, code.co_kwonlyargcount, code.co_stacksize, code.co_flags, code.co_code, code.co_consts, code.co_names, code.co_varnames, code.co_filename, code.co_name, code.co_firstlineno, code.co_lnotab, (), ()) def build_code_chunk(code, filename, name, lineno): return CodeType(0, code.co_nlocals, code.co_kwonlyargcount, code.co_stacksize, code.co_flags | 0x0040, code.co_code, code.co_consts, code.co_names, code.co_varnames, filename, name, lineno, code.co_lnotab, (), ()) # Compatibility fallback implementations for Python < 2.6 try: next = next except NameError: def next(iterator): return iterator.next() # Compatibility fallback implementations for Python < 2.5 try: all = all any = any except NameError: def any(S): for x in S: if x: return True return False def all(S): for x in S: if not x: return False return True Genshi-0.7/genshi/template/0000775000175000017500000000000012101253106016152 5ustar simonsimon00000000000000Genshi-0.7/genshi/template/directives.py0000664000175000017500000006526412101246064020707 0ustar simonsimon00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2006-2009 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which # you should have received as part of this distribution. The terms # are also available at http://genshi.edgewall.org/wiki/License. # # This software consists of voluntary contributions made by many # individuals. For the exact contribution history, see the revision # history and logs, available at http://genshi.edgewall.org/log/. """Implementation of the various template directives.""" from genshi.core import QName, Stream from genshi.path import Path from genshi.template.base import TemplateRuntimeError, TemplateSyntaxError, \ EXPR, _apply_directives, _eval_expr from genshi.template.eval import Expression, ExpressionASTTransformer, \ _ast, _parse __all__ = ['AttrsDirective', 'ChooseDirective', 'ContentDirective', 'DefDirective', 'ForDirective', 'IfDirective', 'MatchDirective', 'OtherwiseDirective', 'ReplaceDirective', 'StripDirective', 'WhenDirective', 'WithDirective'] __docformat__ = 'restructuredtext en' class DirectiveMeta(type): """Meta class for template directives.""" def __new__(cls, name, bases, d): d['tagname'] = name.lower().replace('directive', '') return type.__new__(cls, name, bases, d) class Directive(object): """Abstract base class for template directives. A directive is basically a callable that takes three positional arguments: ``ctxt`` is the template data context, ``stream`` is an iterable over the events that the directive applies to, and ``directives`` is is a list of other directives on the same stream that need to be applied. Directives can be "anonymous" or "registered". Registered directives can be applied by the template author using an XML attribute with the corresponding name in the template. Such directives should be subclasses of this base class that can be instantiated with the value of the directive attribute as parameter. Anonymous directives are simply functions conforming to the protocol described above, and can only be applied programmatically (for example by template filters). """ __metaclass__ = DirectiveMeta __slots__ = ['expr'] def __init__(self, value, template=None, namespaces=None, lineno=-1, offset=-1): self.expr = self._parse_expr(value, template, lineno, offset) @classmethod def attach(cls, template, stream, value, namespaces, pos): """Called after the template stream has been completely parsed. :param template: the `Template` object :param stream: the event stream associated with the directive :param value: the argument value for the directive; if the directive was specified as an element, this will be an `Attrs` instance with all specified attributes, otherwise it will be a `unicode` object with just the attribute value :param namespaces: a mapping of namespace URIs to prefixes :param pos: a ``(filename, lineno, offset)`` tuple describing the location where the directive was found in the source This class method should return a ``(directive, stream)`` tuple. If ``directive`` is not ``None``, it should be an instance of the `Directive` class, and gets added to the list of directives applied to the substream at runtime. `stream` is an event stream that replaces the original stream associated with the directive. """ return cls(value, template, namespaces, *pos[1:]), stream def __call__(self, stream, directives, ctxt, **vars): """Apply the directive to the given stream. :param stream: the event stream :param directives: a list of the remaining directives that should process the stream :param ctxt: the context data :param vars: additional variables that should be made available when Python code is executed """ raise NotImplementedError def __repr__(self): expr = '' if getattr(self, 'expr', None) is not None: expr = ' "%s"' % self.expr.source return '<%s%s>' % (type(self).__name__, expr) @classmethod def _parse_expr(cls, expr, template, lineno=-1, offset=-1): """Parses the given expression, raising a useful error message when a syntax error is encountered. """ try: return expr and Expression(expr, template.filepath, lineno, lookup=template.lookup) or None except SyntaxError, err: err.msg += ' in expression "%s" of "%s" directive' % (expr, cls.tagname) raise TemplateSyntaxError(err, template.filepath, lineno, offset + (err.offset or 0)) def _assignment(ast): """Takes the AST representation of an assignment, and returns a function that applies the assignment of a given value to a dictionary. """ def _names(node): if isinstance(node, _ast.Tuple): return tuple([_names(child) for child in node.elts]) elif isinstance(node, _ast.Name): return node.id def _assign(data, value, names=_names(ast)): if type(names) is tuple: for idx in range(len(names)): _assign(data, value[idx], names[idx]) else: data[names] = value return _assign class AttrsDirective(Directive): """Implementation of the ``py:attrs`` template directive. The value of the ``py:attrs`` attribute should be a dictionary or a sequence of ``(name, value)`` tuples. The items in that dictionary or sequence are added as attributes to the element: >>> from genshi.template import MarkupTemplate >>> tmpl = MarkupTemplate('''
    ...
  • Bar
    • ...
''') >>> print(tmpl.generate(foo={'class': 'collapse'}))
  • Bar
>>> print(tmpl.generate(foo=[('class', 'collapse')]))
  • Bar
If the value evaluates to ``None`` (or any other non-truth value), no attributes are added: >>> print(tmpl.generate(foo=None))
  • Bar
""" __slots__ = [] def __call__(self, stream, directives, ctxt, **vars): def _generate(): kind, (tag, attrib), pos = stream.next() attrs = _eval_expr(self.expr, ctxt, vars) if attrs: if isinstance(attrs, Stream): try: attrs = iter(attrs).next() except StopIteration: attrs = [] elif not isinstance(attrs, list): # assume it's a dict attrs = attrs.items() attrib |= [ (QName(n), v is not None and unicode(v).strip() or None) for n, v in attrs ] yield kind, (tag, attrib), pos for event in stream: yield event return _apply_directives(_generate(), directives, ctxt, vars) class ContentDirective(Directive): """Implementation of the ``py:content`` template directive. This directive replaces the content of the element with the result of evaluating the value of the ``py:content`` attribute: >>> from genshi.template import MarkupTemplate >>> tmpl = MarkupTemplate('''
    ...
  • Hello
    • ...
''') >>> print(tmpl.generate(bar='Bye'))
  • Bye
""" __slots__ = [] @classmethod def attach(cls, template, stream, value, namespaces, pos): if type(value) is dict: raise TemplateSyntaxError('The content directive can not be used ' 'as an element', template.filepath, *pos[1:]) expr = cls._parse_expr(value, template, *pos[1:]) return None, [stream[0], (EXPR, expr, pos), stream[-1]] class DefDirective(Directive): """Implementation of the ``py:def`` template directive. This directive can be used to create "Named Template Functions", which are template snippets that are not actually output during normal processing, but rather can be expanded from expressions in other places in the template. A named template function can be used just like a normal Python function from template expressions: >>> from genshi.template import MarkupTemplate >>> tmpl = MarkupTemplate('''
...

... ${greeting}, ${name}! ...

... ${echo('Hi', name='you')} ...
''') >>> print(tmpl.generate(bar='Bye'))

Hi, you!

If a function does not require parameters, the parenthesis can be omitted in the definition: >>> tmpl = MarkupTemplate('''
...

... Hello, world! ...

... ${helloworld()} ...
''') >>> print(tmpl.generate(bar='Bye'))

Hello, world!

""" __slots__ = ['name', 'args', 'star_args', 'dstar_args', 'defaults'] def __init__(self, args, template, namespaces=None, lineno=-1, offset=-1): Directive.__init__(self, None, template, namespaces, lineno, offset) ast = _parse(args).body self.args = [] self.star_args = None self.dstar_args = None self.defaults = {} if isinstance(ast, _ast.Call): self.name = ast.func.id for arg in ast.args: # only names self.args.append(arg.id) for kwd in ast.keywords: self.args.append(kwd.arg) exp = Expression(kwd.value, template.filepath, lineno, lookup=template.lookup) self.defaults[kwd.arg] = exp if getattr(ast, 'starargs', None): self.star_args = ast.starargs.id if getattr(ast, 'kwargs', None): self.dstar_args = ast.kwargs.id else: self.name = ast.id @classmethod def attach(cls, template, stream, value, namespaces, pos): if type(value) is dict: value = value.get('function') return super(DefDirective, cls).attach(template, stream, value, namespaces, pos) def __call__(self, stream, directives, ctxt, **vars): stream = list(stream) def function(*args, **kwargs): scope = {} args = list(args) # make mutable for name in self.args: if args: scope[name] = args.pop(0) else: if name in kwargs: val = kwargs.pop(name) else: val = _eval_expr(self.defaults.get(name), ctxt, vars) scope[name] = val if not self.star_args is None: scope[self.star_args] = args if not self.dstar_args is None: scope[self.dstar_args] = kwargs ctxt.push(scope) for event in _apply_directives(stream, directives, ctxt, vars): yield event ctxt.pop() function.__name__ = self.name # Store the function reference in the bottom context frame so that it # doesn't get popped off before processing the template has finished # FIXME: this makes context data mutable as a side-effect ctxt.frames[-1][self.name] = function return [] def __repr__(self): return '<%s "%s">' % (type(self).__name__, self.name) class ForDirective(Directive): """Implementation of the ``py:for`` template directive for repeating an element based on an iterable in the context data. >>> from genshi.template import MarkupTemplate >>> tmpl = MarkupTemplate('''
    ...
  • ${item}
    • ...
''') >>> print(tmpl.generate(items=[1, 2, 3]))
  • 1
  • 2
  • 3
""" __slots__ = ['assign', 'filename'] def __init__(self, value, template, namespaces=None, lineno=-1, offset=-1): if ' in ' not in value: raise TemplateSyntaxError('"in" keyword missing in "for" directive', template.filepath, lineno, offset) assign, value = value.split(' in ', 1) ast = _parse(assign, 'exec') value = 'iter(%s)' % value.strip() self.assign = _assignment(ast.body[0].value) self.filename = template.filepath Directive.__init__(self, value, template, namespaces, lineno, offset) @classmethod def attach(cls, template, stream, value, namespaces, pos): if type(value) is dict: value = value.get('each') return super(ForDirective, cls).attach(template, stream, value, namespaces, pos) def __call__(self, stream, directives, ctxt, **vars): iterable = _eval_expr(self.expr, ctxt, vars) if iterable is None: return assign = self.assign scope = {} stream = list(stream) for item in iterable: assign(scope, item) ctxt.push(scope) for event in _apply_directives(stream, directives, ctxt, vars): yield event ctxt.pop() def __repr__(self): return '<%s>' % type(self).__name__ class IfDirective(Directive): """Implementation of the ``py:if`` template directive for conditionally excluding elements from being output. >>> from genshi.template import MarkupTemplate >>> tmpl = MarkupTemplate('''
... ${bar} ...
''') >>> print(tmpl.generate(foo=True, bar='Hello'))
Hello
""" __slots__ = [] @classmethod def attach(cls, template, stream, value, namespaces, pos): if type(value) is dict: value = value.get('test') return super(IfDirective, cls).attach(template, stream, value, namespaces, pos) def __call__(self, stream, directives, ctxt, **vars): value = _eval_expr(self.expr, ctxt, vars) if value: return _apply_directives(stream, directives, ctxt, vars) return [] class MatchDirective(Directive): """Implementation of the ``py:match`` template directive. >>> from genshi.template import MarkupTemplate >>> tmpl = MarkupTemplate('''
... ... Hello ${select('@name')} ... ... ...
''') >>> print(tmpl.generate())
Hello Dude
""" __slots__ = ['path', 'namespaces', 'hints'] def __init__(self, value, template, hints=None, namespaces=None, lineno=-1, offset=-1): Directive.__init__(self, None, template, namespaces, lineno, offset) self.path = Path(value, template.filepath, lineno) self.namespaces = namespaces or {} self.hints = hints or () @classmethod def attach(cls, template, stream, value, namespaces, pos): hints = [] if type(value) is dict: if value.get('buffer', '').lower() == 'false': hints.append('not_buffered') if value.get('once', '').lower() == 'true': hints.append('match_once') if value.get('recursive', '').lower() == 'false': hints.append('not_recursive') value = value.get('path') return cls(value, template, frozenset(hints), namespaces, *pos[1:]), \ stream def __call__(self, stream, directives, ctxt, **vars): ctxt._match_templates.append((self.path.test(ignore_context=True), self.path, list(stream), self.hints, self.namespaces, directives)) return [] def __repr__(self): return '<%s "%s">' % (type(self).__name__, self.path.source) class ReplaceDirective(Directive): """Implementation of the ``py:replace`` template directive. This directive replaces the element with the result of evaluating the value of the ``py:replace`` attribute: >>> from genshi.template import MarkupTemplate >>> tmpl = MarkupTemplate('''
... Hello ...
''') >>> print(tmpl.generate(bar='Bye'))
Bye
This directive is equivalent to ``py:content`` combined with ``py:strip``, providing a less verbose way to achieve the same effect: >>> tmpl = MarkupTemplate('''
... Hello ...
''') >>> print(tmpl.generate(bar='Bye'))
Bye
""" __slots__ = [] @classmethod def attach(cls, template, stream, value, namespaces, pos): if type(value) is dict: value = value.get('value') if not value: raise TemplateSyntaxError('missing value for "replace" directive', template.filepath, *pos[1:]) expr = cls._parse_expr(value, template, *pos[1:]) return None, [(EXPR, expr, pos)] class StripDirective(Directive): """Implementation of the ``py:strip`` template directive. When the value of the ``py:strip`` attribute evaluates to ``True``, the element is stripped from the output >>> from genshi.template import MarkupTemplate >>> tmpl = MarkupTemplate('''
...
foo
...
''') >>> print(tmpl.generate())
foo
Leaving the attribute value empty is equivalent to a truth value. This directive is particulary interesting for named template functions or match templates that do not generate a top-level element: >>> tmpl = MarkupTemplate('''
...
... ${what} ...
... ${echo('foo')} ...
''') >>> print(tmpl.generate())
foo
""" __slots__ = [] def __call__(self, stream, directives, ctxt, **vars): def _generate(): if not self.expr or _eval_expr(self.expr, ctxt, vars): stream.next() # skip start tag previous = stream.next() for event in stream: yield previous previous = event else: for event in stream: yield event return _apply_directives(_generate(), directives, ctxt, vars) class ChooseDirective(Directive): """Implementation of the ``py:choose`` directive for conditionally selecting one of several body elements to display. If the ``py:choose`` expression is empty the expressions of nested ``py:when`` directives are tested for truth. The first true ``py:when`` body is output. If no ``py:when`` directive is matched then the fallback directive ``py:otherwise`` will be used. >>> from genshi.template import MarkupTemplate >>> tmpl = MarkupTemplate('''
... 0 ... 1 ... 2 ...
''') >>> print(tmpl.generate())
1
If the ``py:choose`` directive contains an expression, the nested ``py:when`` directives are tested for equality to the ``py:choose`` expression: >>> tmpl = MarkupTemplate('''
... 1 ... 2 ...
''') >>> print(tmpl.generate())
2
Behavior is undefined if a ``py:choose`` block contains content outside a ``py:when`` or ``py:otherwise`` block. Behavior is also undefined if a ``py:otherwise`` occurs before ``py:when`` blocks. """ __slots__ = ['matched', 'value'] @classmethod def attach(cls, template, stream, value, namespaces, pos): if type(value) is dict: value = value.get('test') return super(ChooseDirective, cls).attach(template, stream, value, namespaces, pos) def __call__(self, stream, directives, ctxt, **vars): info = [False, bool(self.expr), None] if self.expr: info[2] = _eval_expr(self.expr, ctxt, vars) ctxt._choice_stack.append(info) for event in _apply_directives(stream, directives, ctxt, vars): yield event ctxt._choice_stack.pop() class WhenDirective(Directive): """Implementation of the ``py:when`` directive for nesting in a parent with the ``py:choose`` directive. See the documentation of the `ChooseDirective` for usage. """ __slots__ = ['filename'] def __init__(self, value, template, namespaces=None, lineno=-1, offset=-1): Directive.__init__(self, value, template, namespaces, lineno, offset) self.filename = template.filepath @classmethod def attach(cls, template, stream, value, namespaces, pos): if type(value) is dict: value = value.get('test') return super(WhenDirective, cls).attach(template, stream, value, namespaces, pos) def __call__(self, stream, directives, ctxt, **vars): info = ctxt._choice_stack and ctxt._choice_stack[-1] if not info: raise TemplateRuntimeError('"when" directives can only be used ' 'inside a "choose" directive', self.filename, *(stream.next())[2][1:]) if info[0]: return [] if not self.expr and not info[1]: raise TemplateRuntimeError('either "choose" or "when" directive ' 'must have a test expression', self.filename, *(stream.next())[2][1:]) if info[1]: value = info[2] if self.expr: matched = value == _eval_expr(self.expr, ctxt, vars) else: matched = bool(value) else: matched = bool(_eval_expr(self.expr, ctxt, vars)) info[0] = matched if not matched: return [] return _apply_directives(stream, directives, ctxt, vars) class OtherwiseDirective(Directive): """Implementation of the ``py:otherwise`` directive for nesting in a parent with the ``py:choose`` directive. See the documentation of `ChooseDirective` for usage. """ __slots__ = ['filename'] def __init__(self, value, template, namespaces=None, lineno=-1, offset=-1): Directive.__init__(self, None, template, namespaces, lineno, offset) self.filename = template.filepath def __call__(self, stream, directives, ctxt, **vars): info = ctxt._choice_stack and ctxt._choice_stack[-1] if not info: raise TemplateRuntimeError('an "otherwise" directive can only be ' 'used inside a "choose" directive', self.filename, *(stream.next())[2][1:]) if info[0]: return [] info[0] = True return _apply_directives(stream, directives, ctxt, vars) class WithDirective(Directive): """Implementation of the ``py:with`` template directive, which allows shorthand access to variables and expressions. >>> from genshi.template import MarkupTemplate >>> tmpl = MarkupTemplate('''
... $x $y $z ...
''') >>> print(tmpl.generate(x=42))
42 7 52
""" __slots__ = ['vars'] def __init__(self, value, template, namespaces=None, lineno=-1, offset=-1): Directive.__init__(self, None, template, namespaces, lineno, offset) self.vars = [] value = value.strip() try: ast = _parse(value, 'exec') for node in ast.body: if not isinstance(node, _ast.Assign): raise TemplateSyntaxError('only assignment allowed in ' 'value of the "with" directive', template.filepath, lineno, offset) self.vars.append(([_assignment(n) for n in node.targets], Expression(node.value, template.filepath, lineno, lookup=template.lookup))) except SyntaxError, err: err.msg += ' in expression "%s" of "%s" directive' % (value, self.tagname) raise TemplateSyntaxError(err, template.filepath, lineno, offset + (err.offset or 0)) @classmethod def attach(cls, template, stream, value, namespaces, pos): if type(value) is dict: value = value.get('vars') return super(WithDirective, cls).attach(template, stream, value, namespaces, pos) def __call__(self, stream, directives, ctxt, **vars): frame = {} ctxt.push(frame) for targets, expr in self.vars: value = _eval_expr(expr, ctxt, vars) for assign in targets: assign(frame, value) for event in _apply_directives(stream, directives, ctxt, vars): yield event ctxt.pop() def __repr__(self): return '<%s>' % (type(self).__name__) Genshi-0.7/genshi/template/text.py0000664000175000017500000003025112101246064017516 0ustar simonsimon00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2006-2009 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which # you should have received as part of this distribution. The terms # are also available at http://genshi.edgewall.org/wiki/License. # # This software consists of voluntary contributions made by many # individuals. For the exact contribution history, see the revision # history and logs, available at http://genshi.edgewall.org/log/. """Plain text templating engine. This module implements two template language syntaxes, at least for a certain transitional period. `OldTextTemplate` (aliased to just `TextTemplate`) defines a syntax that was inspired by Cheetah/Velocity. `NewTextTemplate` on the other hand is inspired by the syntax of the Django template language, which has more explicit delimiting of directives, and is more flexible with regards to white space and line breaks. In a future release, `OldTextTemplate` will be phased out in favor of `NewTextTemplate`, as the names imply. Therefore the new syntax is strongly recommended for new projects, and existing projects may want to migrate to the new syntax to remain compatible with future Genshi releases. """ import re from genshi.core import TEXT from genshi.template.base import BadDirectiveError, Template, \ TemplateSyntaxError, EXEC, INCLUDE, SUB from genshi.template.eval import Suite from genshi.template.directives import * from genshi.template.directives import Directive from genshi.template.interpolation import interpolate __all__ = ['NewTextTemplate', 'OldTextTemplate', 'TextTemplate'] __docformat__ = 'restructuredtext en' class NewTextTemplate(Template): r"""Implementation of a simple text-based template engine. This class will replace `OldTextTemplate` in a future release. It uses a more explicit delimiting style for directives: instead of the old style which required putting directives on separate lines that were prefixed with a ``#`` sign, directives and commenbtsr are enclosed in delimiter pairs (by default ``{% ... %}`` and ``{# ... #}``, respectively). Variable substitution uses the same interpolation syntax as for markup languages: simple references are prefixed with a dollar sign, more complex expression enclosed in curly braces. >>> tmpl = NewTextTemplate('''Dear $name, ... ... {# This is a comment #} ... We have the following items for you: ... {% for item in items %} ... * ${'Item %d' % item} ... {% end %} ... ''') >>> print(tmpl.generate(name='Joe', items=[1, 2, 3]).render(encoding=None)) Dear Joe, We have the following items for you: * Item 1 * Item 2 * Item 3 By default, no spaces or line breaks are removed. If a line break should not be included in the output, prefix it with a backslash: >>> tmpl = NewTextTemplate('''Dear $name, ... ... {# This is a comment #}\ ... We have the following items for you: ... {% for item in items %}\ ... * $item ... {% end %}\ ... ''') >>> print(tmpl.generate(name='Joe', items=[1, 2, 3]).render(encoding=None)) Dear Joe, We have the following items for you: * 1 * 2 * 3 Backslashes are also used to escape the start delimiter of directives and comments: >>> tmpl = NewTextTemplate('''Dear $name, ... ... \{# This is a comment #} ... We have the following items for you: ... {% for item in items %}\ ... * $item ... {% end %}\ ... ''') >>> print(tmpl.generate(name='Joe', items=[1, 2, 3]).render(encoding=None)) Dear Joe, {# This is a comment #} We have the following items for you: * 1 * 2 * 3 :since: version 0.5 """ directives = [('def', DefDirective), ('when', WhenDirective), ('otherwise', OtherwiseDirective), ('for', ForDirective), ('if', IfDirective), ('choose', ChooseDirective), ('with', WithDirective)] serializer = 'text' _DIRECTIVE_RE = r'((? offset: text = _escape_sub(_escape_repl, source[offset:start]) for kind, data, pos in interpolate(text, self.filepath, lineno, lookup=self.lookup): stream.append((kind, data, pos)) lineno += len(text.splitlines()) lineno += len(source[start:end].splitlines()) command, value = mo.group(2, 3) if command == 'include': pos = (self.filename, lineno, 0) value = list(interpolate(value, self.filepath, lineno, 0, lookup=self.lookup)) if len(value) == 1 and value[0][0] is TEXT: value = value[0][1] stream.append((INCLUDE, (value, None, []), pos)) elif command == 'python': if not self.allow_exec: raise TemplateSyntaxError('Python code blocks not allowed', self.filepath, lineno) try: suite = Suite(value, self.filepath, lineno, lookup=self.lookup) except SyntaxError, err: raise TemplateSyntaxError(err, self.filepath, lineno + (err.lineno or 1) - 1) pos = (self.filename, lineno, 0) stream.append((EXEC, suite, pos)) elif command == 'end': depth -= 1 if depth in dirmap: directive, start_offset = dirmap.pop(depth) substream = stream[start_offset:] stream[start_offset:] = [(SUB, ([directive], substream), (self.filepath, lineno, 0))] elif command: cls = self.get_directive(command) if cls is None: raise BadDirectiveError(command) directive = 0, cls, value, None, (self.filepath, lineno, 0) dirmap[depth] = (directive, len(stream)) depth += 1 offset = end if offset < len(source): text = _escape_sub(_escape_repl, source[offset:]) for kind, data, pos in interpolate(text, self.filepath, lineno, lookup=self.lookup): stream.append((kind, data, pos)) return stream class OldTextTemplate(Template): """Legacy implementation of the old syntax text-based templates. This class is provided in a transition phase for backwards compatibility. New code should use the `NewTextTemplate` class and the improved syntax it provides. >>> tmpl = OldTextTemplate('''Dear $name, ... ... We have the following items for you: ... #for item in items ... * $item ... #end ... ... All the best, ... Foobar''') >>> print(tmpl.generate(name='Joe', items=[1, 2, 3]).render(encoding=None)) Dear Joe, We have the following items for you: * 1 * 2 * 3 All the best, Foobar """ directives = [('def', DefDirective), ('when', WhenDirective), ('otherwise', OtherwiseDirective), ('for', ForDirective), ('if', IfDirective), ('choose', ChooseDirective), ('with', WithDirective)] serializer = 'text' _DIRECTIVE_RE = re.compile(r'(?:^[ \t]*(? offset: text = source[offset:start] for kind, data, pos in interpolate(text, self.filepath, lineno, lookup=self.lookup): stream.append((kind, data, pos)) lineno += len(text.splitlines()) text = source[start:end].lstrip()[1:] lineno += len(text.splitlines()) directive = text.split(None, 1) if len(directive) > 1: command, value = directive else: command, value = directive[0], None if command == 'end': depth -= 1 if depth in dirmap: directive, start_offset = dirmap.pop(depth) substream = stream[start_offset:] stream[start_offset:] = [(SUB, ([directive], substream), (self.filepath, lineno, 0))] elif command == 'include': pos = (self.filename, lineno, 0) stream.append((INCLUDE, (value.strip(), None, []), pos)) elif command != '#': cls = self.get_directive(command) if cls is None: raise BadDirectiveError(command) directive = 0, cls, value, None, (self.filepath, lineno, 0) dirmap[depth] = (directive, len(stream)) depth += 1 offset = end if offset < len(source): text = source[offset:].replace('\\#', '#') for kind, data, pos in interpolate(text, self.filepath, lineno, lookup=self.lookup): stream.append((kind, data, pos)) return stream TextTemplate = OldTextTemplate Genshi-0.7/genshi/template/loader.py0000664000175000017500000003463512101246064020012 0ustar simonsimon00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2006-2010 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which # you should have received as part of this distribution. The terms # are also available at http://genshi.edgewall.org/wiki/License. # # This software consists of voluntary contributions made by many # individuals. For the exact contribution history, see the revision # history and logs, available at http://genshi.edgewall.org/log/. """Template loading and caching.""" import os try: import threading except ImportError: import dummy_threading as threading from genshi.template.base import TemplateError from genshi.util import LRUCache __all__ = ['TemplateLoader', 'TemplateNotFound', 'directory', 'package', 'prefixed'] __docformat__ = 'restructuredtext en' class TemplateNotFound(TemplateError): """Exception raised when a specific template file could not be found.""" def __init__(self, name, search_path): """Create the exception. :param name: the filename of the template :param search_path: the search path used to lookup the template """ TemplateError.__init__(self, 'Template "%s" not found' % name) self.search_path = search_path class TemplateLoader(object): """Responsible for loading templates from files on the specified search path. >>> import tempfile >>> fd, path = tempfile.mkstemp(suffix='.html', prefix='template') >>> os.write(fd, u'

$var

'.encode('utf-8')) 11 >>> os.close(fd) The template loader accepts a list of directory paths that are then used when searching for template files, in the given order: >>> loader = TemplateLoader([os.path.dirname(path)]) The `load()` method first checks the template cache whether the requested template has already been loaded. If not, it attempts to locate the template file, and returns the corresponding `Template` object: >>> from genshi.template import MarkupTemplate >>> template = loader.load(os.path.basename(path)) >>> isinstance(template, MarkupTemplate) True Template instances are cached: requesting a template with the same name results in the same instance being returned: >>> loader.load(os.path.basename(path)) is template True The `auto_reload` option can be used to control whether a template should be automatically reloaded when the file it was loaded from has been changed. Disable this automatic reloading to improve performance. >>> os.remove(path) """ def __init__(self, search_path=None, auto_reload=False, default_encoding=None, max_cache_size=25, default_class=None, variable_lookup='strict', allow_exec=True, callback=None): """Create the template laoder. :param search_path: a list of absolute path names that should be searched for template files, or a string containing a single absolute path; alternatively, any item on the list may be a ''load function'' that is passed a filename and returns a file-like object and some metadata :param auto_reload: whether to check the last modification time of template files, and reload them if they have changed :param default_encoding: the default encoding to assume when loading templates; defaults to UTF-8 :param max_cache_size: the maximum number of templates to keep in the cache :param default_class: the default `Template` subclass to use when instantiating templates :param variable_lookup: the variable lookup mechanism; either "strict" (the default), "lenient", or a custom lookup class :param allow_exec: whether to allow Python code blocks in templates :param callback: (optional) a callback function that is invoked after a template was initialized by this loader; the function is passed the template object as only argument. This callback can be used for example to add any desired filters to the template :see: `LenientLookup`, `StrictLookup` :note: Changed in 0.5: Added the `allow_exec` argument """ from genshi.template.markup import MarkupTemplate self.search_path = search_path if self.search_path is None: self.search_path = [] elif not isinstance(self.search_path, (list, tuple)): self.search_path = [self.search_path] self.auto_reload = auto_reload """Whether templates should be reloaded when the underlying file is changed""" self.default_encoding = default_encoding self.default_class = default_class or MarkupTemplate self.variable_lookup = variable_lookup self.allow_exec = allow_exec if callback is not None and not hasattr(callback, '__call__'): raise TypeError('The "callback" parameter needs to be callable') self.callback = callback self._cache = LRUCache(max_cache_size) self._uptodate = {} self._lock = threading.RLock() def __getstate__(self): state = self.__dict__.copy() state['_lock'] = None return state def __setstate__(self, state): self.__dict__ = state self._lock = threading.RLock() def load(self, filename, relative_to=None, cls=None, encoding=None): """Load the template with the given name. If the `filename` parameter is relative, this method searches the search path trying to locate a template matching the given name. If the file name is an absolute path, the search path is ignored. If the requested template is not found, a `TemplateNotFound` exception is raised. Otherwise, a `Template` object is returned that represents the parsed template. Template instances are cached to avoid having to parse the same template file more than once. Thus, subsequent calls of this method with the same template file name will return the same `Template` object (unless the ``auto_reload`` option is enabled and the file was changed since the last parse.) If the `relative_to` parameter is provided, the `filename` is interpreted as being relative to that path. :param filename: the relative path of the template file to load :param relative_to: the filename of the template from which the new template is being loaded, or ``None`` if the template is being loaded directly :param cls: the class of the template object to instantiate :param encoding: the encoding of the template to load; defaults to the ``default_encoding`` of the loader instance :return: the loaded `Template` instance :raises TemplateNotFound: if a template with the given name could not be found """ if cls is None: cls = self.default_class search_path = self.search_path # Make the filename relative to the template file its being loaded # from, but only if that file is specified as a relative path, or no # search path has been set up if relative_to and (not search_path or not os.path.isabs(relative_to)): filename = os.path.join(os.path.dirname(relative_to), filename) filename = os.path.normpath(filename) cachekey = filename self._lock.acquire() try: # First check the cache to avoid reparsing the same file try: tmpl = self._cache[cachekey] if not self.auto_reload: return tmpl uptodate = self._uptodate[cachekey] if uptodate is not None and uptodate(): return tmpl except (KeyError, OSError): pass isabs = False if os.path.isabs(filename): # Bypass the search path if the requested filename is absolute search_path = [os.path.dirname(filename)] isabs = True elif relative_to and os.path.isabs(relative_to): # Make sure that the directory containing the including # template is on the search path dirname = os.path.dirname(relative_to) if dirname not in search_path: search_path = list(search_path) + [dirname] isabs = True elif not search_path: # Uh oh, don't know where to look for the template raise TemplateError('Search path for templates not configured') for loadfunc in search_path: if isinstance(loadfunc, basestring): loadfunc = directory(loadfunc) try: filepath, filename, fileobj, uptodate = loadfunc(filename) except IOError: continue else: try: if isabs: # If the filename of either the included or the # including template is absolute, make sure the # included template gets an absolute path, too, # so that nested includes work properly without a # search path filename = filepath tmpl = self._instantiate(cls, fileobj, filepath, filename, encoding=encoding) if self.callback: self.callback(tmpl) self._cache[cachekey] = tmpl self._uptodate[cachekey] = uptodate finally: if hasattr(fileobj, 'close'): fileobj.close() return tmpl raise TemplateNotFound(filename, search_path) finally: self._lock.release() def _instantiate(self, cls, fileobj, filepath, filename, encoding=None): """Instantiate and return the `Template` object based on the given class and parameters. This function is intended for subclasses to override if they need to implement special template instantiation logic. Code that just uses the `TemplateLoader` should use the `load` method instead. :param cls: the class of the template object to instantiate :param fileobj: a readable file-like object containing the template source :param filepath: the absolute path to the template file :param filename: the path to the template file relative to the search path :param encoding: the encoding of the template to load; defaults to the ``default_encoding`` of the loader instance :return: the loaded `Template` instance :rtype: `Template` """ if encoding is None: encoding = self.default_encoding return cls(fileobj, filepath=filepath, filename=filename, loader=self, encoding=encoding, lookup=self.variable_lookup, allow_exec=self.allow_exec) @staticmethod def directory(path): """Loader factory for loading templates from a local directory. :param path: the path to the local directory containing the templates :return: the loader function to load templates from the given directory :rtype: ``function`` """ def _load_from_directory(filename): filepath = os.path.join(path, filename) fileobj = open(filepath, 'rb') mtime = os.path.getmtime(filepath) def _uptodate(): return mtime == os.path.getmtime(filepath) return filepath, filename, fileobj, _uptodate return _load_from_directory @staticmethod def package(name, path): """Loader factory for loading templates from egg package data. :param name: the name of the package containing the resources :param path: the path inside the package data :return: the loader function to load templates from the given package :rtype: ``function`` """ from pkg_resources import resource_stream def _load_from_package(filename): filepath = os.path.join(path, filename) return filepath, filename, resource_stream(name, filepath), None return _load_from_package @staticmethod def prefixed(**delegates): """Factory for a load function that delegates to other loaders depending on the prefix of the requested template path. The prefix is stripped from the filename when passing on the load request to the delegate. >>> load = prefixed( ... app1 = lambda filename: ('app1', filename, None, None), ... app2 = lambda filename: ('app2', filename, None, None) ... ) >>> print(load('app1/foo.html')) ('app1', 'app1/foo.html', None, None) >>> print(load('app2/bar.html')) ('app2', 'app2/bar.html', None, None) :param delegates: mapping of path prefixes to loader functions :return: the loader function :rtype: ``function`` """ def _dispatch_by_prefix(filename): for prefix, delegate in delegates.items(): if filename.startswith(prefix): if isinstance(delegate, basestring): delegate = directory(delegate) filepath, _, fileobj, uptodate = delegate( filename[len(prefix):].lstrip('/\\') ) return filepath, filename, fileobj, uptodate raise TemplateNotFound(filename, list(delegates.keys())) return _dispatch_by_prefix directory = TemplateLoader.directory package = TemplateLoader.package prefixed = TemplateLoader.prefixed Genshi-0.7/genshi/template/interpolation.py0000664000175000017500000001207612101246064021426 0ustar simonsimon00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2007-2009 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which # you should have received as part of this distribution. The terms # are also available at http://genshi.edgewall.org/wiki/License. # # This software consists of voluntary contributions made by many # individuals. For the exact contribution history, see the revision # history and logs, available at http://genshi.edgewall.org/log/. """String interpolation routines, i.e. the splitting up a given text into some parts that are literal strings, and others that are Python expressions. """ from itertools import chain import os import re from tokenize import PseudoToken from genshi.core import TEXT from genshi.template.base import TemplateSyntaxError, EXPR from genshi.template.eval import Expression __all__ = ['interpolate'] __docformat__ = 'restructuredtext en' NAMESTART = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ_' NAMECHARS = NAMESTART + '.0123456789' PREFIX = '$' token_re = re.compile('%s|%s(?s)' % ( r'[uU]?[rR]?("""|\'\'\')((?>> for kind, data, pos in interpolate("hey ${foo}bar"): ... print('%s %r' % (kind, data)) TEXT 'hey ' EXPR Expression('foo') TEXT 'bar' :param text: the text to parse :param filepath: absolute path to the file in which the text was found (optional) :param lineno: the line number at which the text was found (optional) :param offset: the column number at which the text starts in the source (optional) :param lookup: the variable lookup mechanism; either "lenient" (the default), "strict", or a custom lookup class :return: a list of `TEXT` and `EXPR` events :raise TemplateSyntaxError: when a syntax error in an expression is encountered """ pos = [filepath, lineno, offset] textbuf = [] textpos = None for is_expr, chunk in chain(lex(text, pos, filepath), [(True, '')]): if is_expr: if textbuf: yield TEXT, ''.join(textbuf), textpos del textbuf[:] textpos = None if chunk: try: expr = Expression(chunk.strip(), pos[0], pos[1], lookup=lookup) yield EXPR, expr, tuple(pos) except SyntaxError, err: raise TemplateSyntaxError(err, filepath, pos[1], pos[2] + (err.offset or 0)) else: textbuf.append(chunk) if textpos is None: textpos = tuple(pos) if '\n' in chunk: lines = chunk.splitlines() pos[1] += len(lines) - 1 pos[2] += len(lines[-1]) else: pos[2] += len(chunk) def lex(text, textpos, filepath): offset = pos = 0 end = len(text) escaped = False while 1: if escaped: offset = text.find(PREFIX, offset + 2) escaped = False else: offset = text.find(PREFIX, pos) if offset < 0 or offset == end - 1: break next = text[offset + 1] if next == '{': if offset > pos: yield False, text[pos:offset] pos = offset + 2 level = 1 while level: match = token_re.match(text, pos) if match is None or not match.group(): # if there isn't a match or the match is the empty # string, we're not going to match up braces ever raise TemplateSyntaxError('invalid syntax', filepath, *textpos[1:]) pos = match.end() tstart, tend = match.regs[3] token = text[tstart:tend] if token == '{': level += 1 elif token == '}': level -= 1 yield True, text[offset + 2:pos - 1] elif next in NAMESTART: if offset > pos: yield False, text[pos:offset] pos = offset pos += 1 while pos < end: char = text[pos] if char not in NAMECHARS: break pos += 1 yield True, text[offset + 1:pos].strip() elif not escaped and next == PREFIX: if offset > pos: yield False, text[pos:offset] escaped = True pos = offset + 1 else: yield False, text[pos:offset + 1] pos = offset + 1 if pos < end: yield False, text[pos:] Genshi-0.7/genshi/template/ast24.py0000664000175000017500000004241412101246064017473 0ustar simonsimon00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2008-2009 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which # you should have received as part of this distribution. The terms # are also available at http://genshi.edgewall.org/wiki/License. # # This software consists of voluntary contributions made by many # individuals. For the exact contribution history, see the revision # history and logs, available at http://genshi.edgewall.org/log/. """Emulation of the proper abstract syntax tree API for Python 2.4.""" import compiler import compiler.ast from genshi.template import _ast24 as _ast __all__ = ['_ast', 'parse'] __docformat__ = 'restructuredtext en' def _new(cls, *args, **kwargs): ret = cls() if ret._fields: for attr, value in zip(ret._fields, args): if attr in kwargs: raise ValueError('Field set both in args and kwargs') setattr(ret, attr, value) for attr in kwargs: if (getattr(ret, '_fields', None) and attr in ret._fields) \ or (getattr(ret, '_attributes', None) and attr in ret._attributes): setattr(ret, attr, kwargs[attr]) return ret class ASTUpgrader(object): """Transformer changing structure of Python 2.4 ASTs to Python 2.5 ones. Transforms ``compiler.ast`` Abstract Syntax Tree to builtin ``_ast``. It can use fake`` _ast`` classes and this way allow ``_ast`` emulation in Python 2.4. """ def __init__(self): self.out_flags = None self.lines = [-1] def _new(self, *args, **kwargs): return _new(lineno = self.lines[-1], *args, **kwargs) def visit(self, node): if node is None: return None if type(node) is tuple: return tuple([self.visit(n) for n in node]) lno = getattr(node, 'lineno', None) if lno is not None: self.lines.append(lno) visitor = getattr(self, 'visit_%s' % node.__class__.__name__, None) if visitor is None: raise Exception('Unhandled node type %r' % type(node)) retval = visitor(node) if lno is not None: self.lines.pop() return retval def visit_Module(self, node): body = self.visit(node.node) if node.doc: body = [self._new(_ast.Expr, self._new(_ast.Str, node.doc))] + body return self._new(_ast.Module, body) def visit_Expression(self, node): return self._new(_ast.Expression, self.visit(node.node)) def _extract_args(self, node): tab = node.argnames[:] if node.flags & compiler.ast.CO_VARKEYWORDS: kwarg = tab[-1] tab = tab[:-1] else: kwarg = None if node.flags & compiler.ast.CO_VARARGS: vararg = tab[-1] tab = tab[:-1] else: vararg = None def _tup(t): if isinstance(t, str): return self._new(_ast.Name, t, _ast.Store()) elif isinstance(t, tuple): elts = [_tup(x) for x in t] return self._new(_ast.Tuple, elts, _ast.Store()) else: raise NotImplemented args = [] for arg in tab: if isinstance(arg, str): args.append(self._new(_ast.Name, arg, _ast.Param())) elif isinstance(arg, tuple): args.append(_tup(arg)) else: assert False, node.__class__ defaults = [self.visit(d) for d in node.defaults] return self._new(_ast.arguments, args, vararg, kwarg, defaults) def visit_Function(self, node): if getattr(node, 'decorators', ()): decorators = [self.visit(d) for d in node.decorators.nodes] else: decorators = [] args = self._extract_args(node) body = self.visit(node.code) if node.doc: body = [self._new(_ast.Expr, self._new(_ast.Str, node.doc))] + body return self._new(_ast.FunctionDef, node.name, args, body, decorators) def visit_Class(self, node): #self.name_types.append(_ast.Load) bases = [self.visit(b) for b in node.bases] #self.name_types.pop() body = self.visit(node.code) if node.doc: body = [self._new(_ast.Expr, self._new(_ast.Str, node.doc))] + body return self._new(_ast.ClassDef, node.name, bases, body) def visit_Return(self, node): return self._new(_ast.Return, self.visit(node.value)) def visit_Assign(self, node): #self.name_types.append(_ast.Store) targets = [self.visit(t) for t in node.nodes] #self.name_types.pop() return self._new(_ast.Assign, targets, self.visit(node.expr)) aug_operators = { '+=': _ast.Add, '/=': _ast.Div, '//=': _ast.FloorDiv, '<<=': _ast.LShift, '%=': _ast.Mod, '*=': _ast.Mult, '**=': _ast.Pow, '>>=': _ast.RShift, '-=': _ast.Sub, } def visit_AugAssign(self, node): target = self.visit(node.node) # Because it's AugAssign target can't be list nor tuple # so we only have to change context of one node target.ctx = _ast.Store() op = self.aug_operators[node.op]() return self._new(_ast.AugAssign, target, op, self.visit(node.expr)) def _visit_Print(nl): def _visit(self, node): values = [self.visit(v) for v in node.nodes] return self._new(_ast.Print, self.visit(node.dest), values, nl) return _visit visit_Print = _visit_Print(False) visit_Printnl = _visit_Print(True) del _visit_Print def visit_For(self, node): return self._new(_ast.For, self.visit(node.assign), self.visit(node.list), self.visit(node.body), self.visit(node.else_)) def visit_While(self, node): return self._new(_ast.While, self.visit(node.test), self.visit(node.body), self.visit(node.else_)) def visit_If(self, node): def _level(tests, else_): test = self.visit(tests[0][0]) body = self.visit(tests[0][1]) if len(tests) == 1: orelse = self.visit(else_) else: orelse = [_level(tests[1:], else_)] return self._new(_ast.If, test, body, orelse) return _level(node.tests, node.else_) def visit_With(self, node): return self._new(_ast.With, self.visit(node.expr), self.visit(node.vars), self.visit(node.body)) def visit_Raise(self, node): return self._new(_ast.Raise, self.visit(node.expr1), self.visit(node.expr2), self.visit(node.expr3)) def visit_TryExcept(self, node): handlers = [] for type, name, body in node.handlers: handlers.append(self._new(_ast.excepthandler, self.visit(type), self.visit(name), self.visit(body))) return self._new(_ast.TryExcept, self.visit(node.body), handlers, self.visit(node.else_)) def visit_TryFinally(self, node): return self._new(_ast.TryFinally, self.visit(node.body), self.visit(node.final)) def visit_Assert(self, node): return self._new(_ast.Assert, self.visit(node.test), self.visit(node.fail)) def visit_Import(self, node): names = [self._new(_ast.alias, n[0], n[1]) for n in node.names] return self._new(_ast.Import, names) def visit_From(self, node): names = [self._new(_ast.alias, n[0], n[1]) for n in node.names] return self._new(_ast.ImportFrom, node.modname, names, 0) def visit_Exec(self, node): return self._new(_ast.Exec, self.visit(node.expr), self.visit(node.locals), self.visit(node.globals)) def visit_Global(self, node): return self._new(_ast.Global, node.names[:]) def visit_Discard(self, node): return self._new(_ast.Expr, self.visit(node.expr)) def _map_class(to): def _visit(self, node): return self._new(to) return _visit visit_Pass = _map_class(_ast.Pass) visit_Break = _map_class(_ast.Break) visit_Continue = _map_class(_ast.Continue) def _visit_BinOperator(opcls): def _visit(self, node): return self._new(_ast.BinOp, self.visit(node.left), opcls(), self.visit(node.right)) return _visit visit_Add = _visit_BinOperator(_ast.Add) visit_Div = _visit_BinOperator(_ast.Div) visit_FloorDiv = _visit_BinOperator(_ast.FloorDiv) visit_LeftShift = _visit_BinOperator(_ast.LShift) visit_Mod = _visit_BinOperator(_ast.Mod) visit_Mul = _visit_BinOperator(_ast.Mult) visit_Power = _visit_BinOperator(_ast.Pow) visit_RightShift = _visit_BinOperator(_ast.RShift) visit_Sub = _visit_BinOperator(_ast.Sub) del _visit_BinOperator def _visit_BitOperator(opcls): def _visit(self, node): def _make(nodes): if len(nodes) == 1: return self.visit(nodes[0]) left = _make(nodes[:-1]) right = self.visit(nodes[-1]) return self._new(_ast.BinOp, left, opcls(), right) return _make(node.nodes) return _visit visit_Bitand = _visit_BitOperator(_ast.BitAnd) visit_Bitor = _visit_BitOperator(_ast.BitOr) visit_Bitxor = _visit_BitOperator(_ast.BitXor) del _visit_BitOperator def _visit_UnaryOperator(opcls): def _visit(self, node): return self._new(_ast.UnaryOp, opcls(), self.visit(node.expr)) return _visit visit_Invert = _visit_UnaryOperator(_ast.Invert) visit_Not = _visit_UnaryOperator(_ast.Not) visit_UnaryAdd = _visit_UnaryOperator(_ast.UAdd) visit_UnarySub = _visit_UnaryOperator(_ast.USub) del _visit_UnaryOperator def _visit_BoolOperator(opcls): def _visit(self, node): values = [self.visit(n) for n in node.nodes] return self._new(_ast.BoolOp, opcls(), values) return _visit visit_And = _visit_BoolOperator(_ast.And) visit_Or = _visit_BoolOperator(_ast.Or) del _visit_BoolOperator cmp_operators = { '==': _ast.Eq, '!=': _ast.NotEq, '<': _ast.Lt, '<=': _ast.LtE, '>': _ast.Gt, '>=': _ast.GtE, 'is': _ast.Is, 'is not': _ast.IsNot, 'in': _ast.In, 'not in': _ast.NotIn, } def visit_Compare(self, node): left = self.visit(node.expr) ops = [] comparators = [] for optype, expr in node.ops: ops.append(self.cmp_operators[optype]()) comparators.append(self.visit(expr)) return self._new(_ast.Compare, left, ops, comparators) def visit_Lambda(self, node): args = self._extract_args(node) body = self.visit(node.code) return self._new(_ast.Lambda, args, body) def visit_IfExp(self, node): return self._new(_ast.IfExp, self.visit(node.test), self.visit(node.then), self.visit(node.else_)) def visit_Dict(self, node): keys = [self.visit(x[0]) for x in node.items] values = [self.visit(x[1]) for x in node.items] return self._new(_ast.Dict, keys, values) def visit_ListComp(self, node): generators = [self.visit(q) for q in node.quals] return self._new(_ast.ListComp, self.visit(node.expr), generators) def visit_GenExprInner(self, node): generators = [self.visit(q) for q in node.quals] return self._new(_ast.GeneratorExp, self.visit(node.expr), generators) def visit_GenExpr(self, node): return self.visit(node.code) def visit_GenExprFor(self, node): ifs = [self.visit(i) for i in node.ifs] return self._new(_ast.comprehension, self.visit(node.assign), self.visit(node.iter), ifs) def visit_ListCompFor(self, node): ifs = [self.visit(i) for i in node.ifs] return self._new(_ast.comprehension, self.visit(node.assign), self.visit(node.list), ifs) def visit_GenExprIf(self, node): return self.visit(node.test) visit_ListCompIf = visit_GenExprIf def visit_Yield(self, node): return self._new(_ast.Yield, self.visit(node.value)) def visit_CallFunc(self, node): args = [] keywords = [] for arg in node.args: if isinstance(arg, compiler.ast.Keyword): keywords.append(self._new(_ast.keyword, arg.name, self.visit(arg.expr))) else: args.append(self.visit(arg)) return self._new(_ast.Call, self.visit(node.node), args, keywords, self.visit(node.star_args), self.visit(node.dstar_args)) def visit_Backquote(self, node): return self._new(_ast.Repr, self.visit(node.expr)) def visit_Const(self, node): if node.value is None: # appears in slices return None elif isinstance(node.value, basestring): return self._new(_ast.Str, node.value) else: return self._new(_ast.Num, node.value) def visit_Name(self, node): return self._new(_ast.Name, node.name, _ast.Load()) def visit_Getattr(self, node): return self._new(_ast.Attribute, self.visit(node.expr), node.attrname, _ast.Load()) def visit_Tuple(self, node): nodes = [self.visit(n) for n in node.nodes] return self._new(_ast.Tuple, nodes, _ast.Load()) def visit_List(self, node): nodes = [self.visit(n) for n in node.nodes] return self._new(_ast.List, nodes, _ast.Load()) def get_ctx(self, flags): if flags == 'OP_DELETE': return _ast.Del() elif flags == 'OP_APPLY': return _ast.Load() elif flags == 'OP_ASSIGN': return _ast.Store() else: # FIXME Exception here assert False, repr(flags) def visit_AssName(self, node): self.out_flags = node.flags ctx = self.get_ctx(node.flags) return self._new(_ast.Name, node.name, ctx) def visit_AssAttr(self, node): self.out_flags = node.flags ctx = self.get_ctx(node.flags) return self._new(_ast.Attribute, self.visit(node.expr), node.attrname, ctx) def _visit_AssCollection(cls): def _visit(self, node): flags = None elts = [] for n in node.nodes: elts.append(self.visit(n)) if flags is None: flags = self.out_flags else: assert flags == self.out_flags self.out_flags = flags ctx = self.get_ctx(flags) return self._new(cls, elts, ctx) return _visit visit_AssList = _visit_AssCollection(_ast.List) visit_AssTuple = _visit_AssCollection(_ast.Tuple) del _visit_AssCollection def visit_Slice(self, node): lower = self.visit(node.lower) upper = self.visit(node.upper) ctx = self.get_ctx(node.flags) self.out_flags = node.flags return self._new(_ast.Subscript, self.visit(node.expr), self._new(_ast.Slice, lower, upper, None), ctx) def visit_Subscript(self, node): ctx = self.get_ctx(node.flags) subs = [self.visit(s) for s in node.subs] advanced = (_ast.Slice, _ast.Ellipsis) slices = [] nonindex = False for sub in subs: if isinstance(sub, advanced): nonindex = True slices.append(sub) else: slices.append(self._new(_ast.Index, sub)) if len(slices) == 1: slice = slices[0] elif nonindex: slice = self._new(_ast.ExtSlice, slices) else: slice = self._new(_ast.Tuple, slices, _ast.Load()) self.out_flags = node.flags return self._new(_ast.Subscript, self.visit(node.expr), slice, ctx) def visit_Sliceobj(self, node): a = [self.visit(n) for n in node.nodes + [None]*(3 - len(node.nodes))] return self._new(_ast.Slice, a[0], a[1], a[2]) def visit_Ellipsis(self, node): return self._new(_ast.Ellipsis) def visit_Stmt(self, node): def _check_del(n): # del x is just AssName('x', 'OP_DELETE') # we want to transform it to Delete([Name('x', Del())]) dcls = (_ast.Name, _ast.List, _ast.Subscript, _ast.Attribute) if isinstance(n, dcls) and isinstance(n.ctx, _ast.Del): return self._new(_ast.Delete, [n]) elif isinstance(n, _ast.Tuple) and isinstance(n.ctx, _ast.Del): # unpack last tuple to avoid making del (x, y, z,); # out of del x, y, z; (there's no difference between # this two in compiler.ast) return self._new(_ast.Delete, n.elts) else: return n def _keep(n): if isinstance(n, _ast.Expr) and n.value is None: return False else: return True return [s for s in [_check_del(self.visit(n)) for n in node.nodes] if _keep(s)] def parse(source, mode): node = compiler.parse(source, mode) return ASTUpgrader().visit(node) Genshi-0.7/genshi/template/tests/0000775000175000017500000000000012101253106017314 5ustar simonsimon00000000000000Genshi-0.7/genshi/template/tests/directives.py0000664000175000017500000012324112101246064022037 0ustar simonsimon00000000000000# -*- coding: utf-8 -*- # # Copyright (C) 2006-2010 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which # you should have received as part of this distribution. The terms # are also available at http://genshi.edgewall.org/wiki/License. # # This software consists of voluntary contributions made by many # individuals. For the exact contribution history, see the revision # history and logs, available at http://genshi.edgewall.org/log/. import doctest import re import sys import unittest from genshi.template import directives, MarkupTemplate, TextTemplate, \ TemplateRuntimeError, TemplateSyntaxError class AttrsDirectiveTestCase(unittest.TestCase): """Tests for the `py:attrs` template directive.""" def test_combined_with_loop(self): """ Verify that the directive has access to the loop variables. """ tmpl = MarkupTemplate(""" """) items = [{'id': 1}, {'id': 2}] self.assertEqual(""" """, tmpl.generate(items=items).render(encoding=None)) def test_update_existing_attr(self): """ Verify that an attribute value that evaluates to `None` removes an existing attribute of that name. """ tmpl = MarkupTemplate(""" """) self.assertEqual(""" """, tmpl.generate().render(encoding=None)) def test_remove_existing_attr(self): """ Verify that an attribute value that evaluates to `None` removes an existing attribute of that name. """ tmpl = MarkupTemplate(""" """) self.assertEqual(""" """, tmpl.generate().render(encoding=None)) class ChooseDirectiveTestCase(unittest.TestCase): """Tests for the `py:choose` template directive and the complementary directives `py:when` and `py:otherwise`.""" def test_multiple_true_whens(self): """ Verify that, if multiple `py:when` bodies match, only the first is output. """ tmpl = MarkupTemplate("""
1 2 3
""") self.assertEqual("""
1
""", tmpl.generate().render(encoding=None)) def test_otherwise(self): tmpl = MarkupTemplate("""
hidden hello
""") self.assertEqual("""
hello
""", tmpl.generate().render(encoding=None)) def test_nesting(self): """ Verify that `py:choose` blocks can be nested: """ tmpl = MarkupTemplate("""
2 3
""") self.assertEqual("""
3
""", tmpl.generate().render(encoding=None)) def test_complex_nesting(self): """ Verify more complex nesting. """ tmpl = MarkupTemplate("""
OK FAIL
""") self.assertEqual("""
OK
""", tmpl.generate().render(encoding=None)) def test_complex_nesting_otherwise(self): """ Verify more complex nesting using otherwise. """ tmpl = MarkupTemplate("""
FAIL OK
""") self.assertEqual("""
OK
""", tmpl.generate().render(encoding=None)) def test_when_with_strip(self): """ Verify that a when directive with a strip directive actually strips of the outer element. """ tmpl = MarkupTemplate("""
foo
""") self.assertEqual(""" foo """, tmpl.generate().render(encoding=None)) def test_when_outside_choose(self): """ Verify that a `when` directive outside of a `choose` directive is reported as an error. """ tmpl = MarkupTemplate("""
""") self.assertRaises(TemplateRuntimeError, str, tmpl.generate()) def test_otherwise_outside_choose(self): """ Verify that an `otherwise` directive outside of a `choose` directive is reported as an error. """ tmpl = MarkupTemplate("""
""") self.assertRaises(TemplateRuntimeError, str, tmpl.generate()) def test_when_without_test(self): """ Verify that an `when` directive that doesn't have a `test` attribute is reported as an error. """ tmpl = MarkupTemplate("""
foo
""") self.assertRaises(TemplateRuntimeError, str, tmpl.generate()) def test_when_without_test_but_with_choose_value(self): """ Verify that an `when` directive that doesn't have a `test` attribute works as expected as long as the parent `choose` directive has a test expression. """ tmpl = MarkupTemplate("""
foo
""") self.assertEqual(""" foo """, tmpl.generate(foo='Yeah').render(encoding=None)) def test_otherwise_without_test(self): """ Verify that an `otherwise` directive can be used without a `test` attribute. """ tmpl = MarkupTemplate("""
foo
""") self.assertEqual(""" foo """, tmpl.generate().render(encoding=None)) def test_as_element(self): """ Verify that the directive can also be used as an element. """ tmpl = MarkupTemplate(""" 1 2 3 """) self.assertEqual(""" 1 """, tmpl.generate().render(encoding=None)) def test_in_text_template(self): """ Verify that the directive works as expected in a text template. """ tmpl = TextTemplate("""#choose #when 1 == 1 1 #end #when 2 == 2 2 #end #when 3 == 3 3 #end #end""") self.assertEqual(""" 1\n""", tmpl.generate().render(encoding=None)) class DefDirectiveTestCase(unittest.TestCase): """Tests for the `py:def` template directive.""" def test_function_with_strip(self): """ Verify that a named template function with a strip directive actually strips of the outer element. """ tmpl = MarkupTemplate("""
${what}
${echo('foo')} """) self.assertEqual(""" foo """, tmpl.generate().render(encoding=None)) def test_exec_in_replace(self): tmpl = MarkupTemplate("""

${greeting}, ${name}!

""") self.assertEqual("""

hello, world!

""", tmpl.generate().render(encoding=None)) def test_as_element(self): """ Verify that the directive can also be used as an element. """ tmpl = MarkupTemplate(""" ${what} ${echo('foo')} """) self.assertEqual(""" foo """, tmpl.generate().render(encoding=None)) def test_nested_defs(self): """ Verify that a template function defined inside a conditional block can be called from outside that block. """ tmpl = MarkupTemplate(""" ${what} ${what} ${echo('foo')} """) self.assertEqual(""" foo """, tmpl.generate(semantic=True).render(encoding=None)) def test_function_with_default_arg(self): """ Verify that keyword arguments work with `py:def` directives. """ tmpl = MarkupTemplate(""" ${what} ${echo('foo')} """) self.assertEqual(""" foo """, tmpl.generate().render(encoding=None)) def test_invocation_in_attribute(self): tmpl = MarkupTemplate(""" ${what or 'something'}

bar

""") self.assertEqual("""

bar

""", tmpl.generate().render(encoding=None)) def test_invocation_in_attribute_none(self): tmpl = MarkupTemplate(""" ${None}

bar

""") self.assertEqual("""

bar

""", tmpl.generate().render(encoding=None)) def test_function_raising_typeerror(self): def badfunc(): raise TypeError tmpl = MarkupTemplate("""
${badfunc()}
""") self.assertRaises(TypeError, list, tmpl.generate(badfunc=badfunc)) def test_def_in_matched(self): tmpl = MarkupTemplate(""" ${select('*')} ${maketitle(True)} """) self.assertEqual(""" True """, tmpl.generate().render(encoding=None)) def test_in_text_template(self): """ Verify that the directive works as expected in a text template. """ tmpl = TextTemplate(""" #def echo(greeting, name='world') ${greeting}, ${name}! #end ${echo('Hi', name='you')} """) self.assertEqual(""" Hi, you! """, tmpl.generate().render(encoding=None)) def test_function_with_star_args(self): """ Verify that a named template function using "star arguments" works as expected. """ tmpl = MarkupTemplate("""
${repr(args)} ${repr(sorted(kwargs.items()))}
${f(1, 2, a=3, b=4)} """) self.assertEqual("""
[1, 2] [('a', 3), ('b', 4)]
""", tmpl.generate().render(encoding=None)) class ForDirectiveTestCase(unittest.TestCase): """Tests for the `py:for` template directive.""" def test_loop_with_strip(self): """ Verify that the combining the `py:for` directive with `py:strip` works correctly. """ tmpl = MarkupTemplate("""
${item}
""") self.assertEqual(""" 1 2 3 4 5 """, tmpl.generate(items=range(1, 6)).render(encoding=None)) def test_as_element(self): """ Verify that the directive can also be used as an element. """ tmpl = MarkupTemplate(""" ${item} """) self.assertEqual(""" 1 2 3 4 5 """, tmpl.generate(items=range(1, 6)).render(encoding=None)) def test_multi_assignment(self): """ Verify that assignment to tuples works correctly. """ tmpl = MarkupTemplate("""

key=$k, value=$v

 """) self.assertEqual("""

key=a, value=1

key=b, value=2

""", tmpl.generate(items=(('a', 1), ('b', 2))) .render(encoding=None)) def test_nested_assignment(self): """ Verify that assignment to nested tuples works correctly. """ tmpl = MarkupTemplate("""

$idx: key=$k, value=$v

 """) self.assertEqual("""

0: key=a, value=1

1: key=b, value=2

""", tmpl.generate(items=enumerate([('a', 1), ('b', 2)])) .render(encoding=None)) def test_not_iterable(self): """ Verify that assignment to nested tuples works correctly. """ tmpl = MarkupTemplate(""" $item """, filename='test.html') try: list(tmpl.generate(foo=12)) self.fail('Expected TemplateRuntimeError') except TypeError, e: assert (str(e) == "iteration over non-sequence" or str(e) == "'int' object is not iterable") exc_type, exc_value, exc_traceback = sys.exc_info() frame = exc_traceback.tb_next frames = [] while frame.tb_next: frame = frame.tb_next frames.append(frame) self.assertEqual("", frames[-1].tb_frame.f_code.co_name) self.assertEqual('test.html', frames[-1].tb_frame.f_code.co_filename) self.assertEqual(2, frames[-1].tb_lineno) def test_for_with_empty_value(self): """ Verify an empty 'for' value is an error """ try: MarkupTemplate(""" empty """, filename='test.html').generate() self.fail('ExpectedTemplateSyntaxError') except TemplateSyntaxError, e: self.assertEqual('test.html', e.filename) if sys.version_info[:2] > (2,4): self.assertEqual(2, e.lineno) class IfDirectiveTestCase(unittest.TestCase): """Tests for the `py:if` template directive.""" def test_loop_with_strip(self): """ Verify that the combining the `py:if` directive with `py:strip` works correctly. """ tmpl = MarkupTemplate(""" ${bar} """) self.assertEqual(""" Hello """, tmpl.generate(foo=True, bar='Hello').render(encoding=None)) def test_as_element(self): """ Verify that the directive can also be used as an element. """ tmpl = MarkupTemplate(""" ${bar} """) self.assertEqual(""" Hello """, tmpl.generate(foo=True, bar='Hello').render(encoding=None)) class MatchDirectiveTestCase(unittest.TestCase): """Tests for the `py:match` template directive.""" def test_with_strip(self): """ Verify that a match template can produce the same kind of element that it matched without entering an infinite recursion. """ tmpl = MarkupTemplate("""
${select('text()')}
 Hey Joe """) self.assertEqual("""
Hey Joe
""", tmpl.generate().render(encoding=None)) def test_without_strip(self): """ Verify that a match template can produce the same kind of element that it matched without entering an infinite recursion. """ tmpl = MarkupTemplate("""
${select('text()')}
 Hey Joe """) self.assertEqual("""
Hey Joe
 """, tmpl.generate().render(encoding=None)) def test_as_element(self): """ Verify that the directive can also be used as an element. """ tmpl = MarkupTemplate("""
${select('text()')}
 Hey Joe """) self.assertEqual("""
Hey Joe
""", tmpl.generate().render(encoding=None)) def test_recursive_match_1(self): """ Match directives are applied recursively, meaning that they are also applied to any content they may have produced themselves: """ tmpl = MarkupTemplate("""
${select('*')}
""") self.assertEqual("""
""", tmpl.generate().render(encoding=None)) def test_recursive_match_2(self): """ When two or more match templates match the same element and also themselves output the element they match, avoiding recursion is even more complex, but should work. """ tmpl = MarkupTemplate("""