zope.app.generations-3.6.1/ 0000775 0001771 0002004 00000000000 11707310574 016725 5 ustar menesis menesis 0000000 0000000 zope.app.generations-3.6.1/setup.py 0000664 0001771 0002004 00000006015 11707310564 020440 0 ustar menesis menesis 0000000 0000000 ##############################################################################
#
# Copyright (c) 2006 Zope Foundation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
##############################################################################
# This package is developed by the Zope Toolkit project, documented here:
# http://docs.zope.org/zopetoolkit
# When developing and releasing this package, please follow the documented
# Zope Toolkit policies as described by this documentation.
##############################################################################
"""Setup for zope.app.generations package
$Id: setup.py 124147 2012-01-23 16:56:34Z menesis $
"""
import os
from setuptools import setup, find_packages
def read(*rnames):
return open(os.path.join(os.path.dirname(__file__), *rnames)).read()
setup(name='zope.app.generations',
version='3.6.1',
author='Zope Corporation and Contributors',
author_email='zope-dev@zope.org',
description='Zope Application Schema Generations',
long_description=(
read('README.txt')
+ '\n\n.. contents::\n\n' +
'======================\n'
'Detailed Documentation\n'
'======================\n'
+ '\n\n' +
read('src', 'zope', 'app', 'generations', 'README.txt')
+ '\n\n' +
read('CHANGES.txt')
),
keywords = "zope3 zodb schema generation",
classifiers = [
'Development Status :: 5 - Production/Stable',
'Environment :: Web Environment',
'Intended Audience :: Developers',
'License :: OSI Approved :: Zope Public License',
'Programming Language :: Python',
'Natural Language :: English',
'Operating System :: OS Independent',
'Topic :: Internet :: WWW/HTTP',
'Framework :: Zope3'],
url='http://pypi.python.org/pypi/zope.app.generations',
license='ZPL 2.1',
packages=find_packages('src'),
package_dir = {'': 'src'},
namespace_packages=['zope', 'zope.app'],
extras_require = dict(test=[
'zope.app.testing',
'zope.app.zcmlfiles',
'zope.login',
'zope.password',
'zope.publisher >= 3.12',
'zope.securitypolicy',
]),
install_requires=['setuptools',
'zope.app.renderer',
'zope.interface',
'zope.app.publication',
'ZODB3',
'zope.processlifetime',
'zope.applicationcontrol',
],
include_package_data = True,
zip_safe = False,
)
zope.app.generations-3.6.1/CHANGES.txt 0000664 0001771 0002004 00000003222 11707310564 020534 0 ustar menesis menesis 0000000 0000000 =======
CHANGES
=======
3.6.1 (2012-01-23)
------------------
- Replaced an undeclared test dependency on ``zope.app.authentication`` with
``zope.password``.
3.6.0 (2010-09-17)
------------------
- ``zope.app.generations`` depended on ``zope.app.applicationcontrol`` but
did not declare it. Modernized dependecy to ``zope.applicationcontrol`` as
the needed interface has been moved there.
- Using python's ``doctest`` module instead of deprecated
``zope.testing.doctest[unit]``.
- Replaced a testing dependency on ``zope.app.securitypolicy`` with one on
``zope.securitypolicy``.
3.5.1 (2010-01-08)
------------------
- Depend on new ``zope.processlifetime`` interfaces instead of using
BBB imports from ``zope.app.appsetup``.
- Fix ftesting.zcml due to ``zope.securitypolicy`` update.
- Fix tests using a newer zope.publisher that requires zope.login.
3.5.0 (2009-04-05)
------------------
- Moved ``getRootFolder`` utility method from
``zope.app.zopeappgenerations`` to ``zope.app.generations.utility``.
- Removed not necessary install dependency on ``zope.app.testing``.
3.4.2 (2009-01-27)
------------------
- Provide more logging output for the various stages and actions of evolving a
database.
- Fixed bug: A failing last generation would allow starting an app server
without having evolved to the minimum generation.
- Substitute zope.app.zapi by direct calls to its wrapped apis. See
bug 219302.
- Corrected author email and home page address.
3.4.1 (2007-10-31)
------------------
- Resolve ``ZopeSecurityPolicy`` deprecation warning.
3.4.0 (2007-10-24)
------------------
- Initial release independent of the main Zope tree.
zope.app.generations-3.6.1/PKG-INFO 0000664 0001771 0002004 00000046161 11707310574 020032 0 ustar menesis menesis 0000000 0000000 Metadata-Version: 1.1
Name: zope.app.generations
Version: 3.6.1
Summary: Zope Application Schema Generations
Home-page: http://pypi.python.org/pypi/zope.app.generations
Author: Zope Corporation and Contributors
Author-email: zope-dev@zope.org
License: ZPL 2.1
Description: Generations are a way of updating objects in the database when the application
schema changes. An application schema is essentially the structure of data,
the structure of classes in the case of ZODB or the table descriptions in the
case of a relational database.
.. contents::
======================
Detailed Documentation
======================
Generations are a way of updating objects in the database when the application
schema changes. An application schema is essentially the structure of data,
the structure of classes in the case of ZODB or the table descriptions in the
case of a relational database.
When you change your application's data structures, for example,
you change the semantic meaning of an existing field in a class, you will
have a problem with databases that were created before your change. For a
more thorough discussion and possible solutions, see
http://wiki.zope.org/zope3/DatabaseGenerations
We will be using the component architecture, and we will need a database and a
connection:
>>> import cgi
>>> from pprint import pprint
>>> from zope.interface import implements
>>> from zope.app.testing import ztapi
>>> from ZODB.tests.util import DB
>>> db = DB()
>>> conn = db.open()
>>> root = conn.root()
Imagine that our application is an oracle: you can teach it to react to
phrases. Let's keep it simple and store the data in a dict:
>>> root['answers'] = {'Hello': 'Hi & how do you do?',
... 'Meaning of life?': '42',
... 'four < ?': 'four < five'}
>>> import transaction
>>> transaction.commit()
Initial setup
-------------
Here's some generations-specific code. We will create and register a
SchemaManager. SchemaManagers are responsible for the actual updates of the
database. This one will be just a dummy. The point here is to make the
generations module aware that our application supports generations.
The default implementation of SchemaManager is not suitable for this test
because it uses Python modules to manage generations. For now, it
will be just fine, since we don't want it to do anything just yet.
>>> from zope.app.generations.interfaces import ISchemaManager
>>> from zope.app.generations.generations import SchemaManager
>>> dummy_manager = SchemaManager(minimum_generation=0, generation=0)
>>> ztapi.provideUtility(ISchemaManager, dummy_manager, name='some.app')
'some.app' is a unique identifier. You should use a URI or the dotted name
of your package.
When you start Zope and a database is opened, an event
IDatabaseOpenedWithRoot is sent. Zope registers
evolveMinimumSubscriber by default as a handler for this event. Let's
simulate this:
>>> class DatabaseOpenedEventStub(object):
... def __init__(self, database):
... self.database = database
>>> event = DatabaseOpenedEventStub(db)
>>> from zope.app.generations.generations import evolveMinimumSubscriber
>>> evolveMinimumSubscriber(event)
The consequence of this action is that now the database contains the fact
that our current schema number is 0. When we update the schema, Zope3 will
have an idea of what the starting point was. Here, see?
>>> from zope.app.generations.generations import generations_key
>>> root[generations_key]['some.app']
0
In real life you should never have to bother with this key directly,
but you should be aware that it exists.
Upgrade scenario
----------------
Back to the story. Some time passes and one of our clients gets hacked because
we forgot to escape HTML special characters! The horror! We must fix this
problem ASAP without losing any data. We decide to use generations to impress
our peers.
Let's update the schema manager (drop the old one and install a new custom
one):
>>> ztapi.unprovideUtility(ISchemaManager, name='some.app')
>>> class MySchemaManager(object):
... implements(ISchemaManager)
...
... minimum_generation = 1
... generation = 2
...
... def evolve(self, context, generation):
... root = context.connection.root()
... answers = root['answers']
... if generation == 1:
... for question, answer in answers.items():
... answers[question] = cgi.escape(answer)
... elif generation == 2:
... for question, answer in answers.items():
... del answers[question]
... answers[cgi.escape(question)] = answer
... else:
... raise ValueError("Bummer")
... root['answers'] = answers # ping persistence
... transaction.commit()
>>> manager = MySchemaManager()
>>> ztapi.provideUtility(ISchemaManager, manager, name='some.app')
We have set `minimum_generation` to 1. That means that our application
will refuse to run with a database older than generation 1. The `generation`
attribute is set to 2, which means that the latest generation that this
SchemaManager knows about is 2.
evolve() is the workhorse here. Its job is to get the database from
`generation`-1 to `generation`. It gets a context which has the attribute
'connection', which is a connection to the ZODB. You can use that to change
objects like in this example.
In this particular implementation generation 1 escapes the answers (say,
critical, because they can be entered by anyone!), generation 2 escapes the
questions (say, less important, because these can be entered by authorized
personell only).
In fact, you don't really need a custom implementation of ISchemaManager. One
is available, we have used it for a dummy previously. It uses Python modules
for organization of evolver functions. See its docstring for more information.
In real life you will have much more complex object structures than the one
here. To make your life easier, there are two very useful functions available
in zope.app.generations.utility: findObjectsMatching() and
findObjectsProviding(). They will dig through containers recursively to help
you seek out old objects that you wish to update, by interface or by some other
criteria. They are easy to understand, check their docstrings.
Generations in action
---------------------
So, our furious client downloads our latest code and restarts Zope. The event
is automatically sent again:
>>> event = DatabaseOpenedEventStub(db)
>>> evolveMinimumSubscriber(event)
Shazam! The client is happy again!
>>> pprint(root['answers'])
{'Hello': 'Hi & how do you do?',
'Meaning of life?': '42',
'four < ?': 'four < five'}
Because evolveMinimumSubscriber is very lazy, it only updates the database just
enough so that your application can use it (to the `minimum_generation`, that
is). Indeed, the marker indicates that the database generation has been bumped
to 1:
>>> root[generations_key]['some.app']
1
We see that generations are working, so we decide to take the next step
and evolve to generation 2. Let's see how this can be done manually:
>>> from zope.app.generations.generations import evolve
>>> evolve(db)
>>> pprint(root['answers'])
{'Hello': 'Hi & how do you do?',
'Meaning of life?': '42',
'four < ?': 'four < five'}
>>> root[generations_key]['some.app']
2
Default behaviour of `evolve` upgrades to the latest generation provided by
the SchemaManager. You can use the `how` argument to evolve() when you want
just to check if you need to update or if you want to be lazy like the
subscriber which we have called previously.
Ordering of schema managers
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Frequently subsystems used to compose an application rely on other
subsystems to operate properly. If both subsystems provide schema
managers, it is often helpful to know the order in which the evolvers
will be invoked. This allows a framework and it's clients to be able
to evolve in concert, and the clients can know that the framework will
be evolved before or after itself.
This can be accomplished by controlling the names of the schema
manager utilities. The schema managers are run in the order
determined by sorting their names.
>>> manager1 = SchemaManager(minimum_generation=0, generation=0)
>>> manager2 = SchemaManager(minimum_generation=0, generation=0)
>>> ztapi.provideUtility(
... ISchemaManager, manager1, name='another.app')
>>> ztapi.provideUtility(
... ISchemaManager, manager2, name='another.app-extension')
Notice how the name of the first package is used to create a namespace
for dependent packages. This is not a requirement of the framework,
but a convenient pattern for this usage.
Let's evolve the database to establish these generations:
>>> event = DatabaseOpenedEventStub(db)
>>> evolveMinimumSubscriber(event)
>>> root[generations_key]['another.app']
0
>>> root[generations_key]['another.app-extension']
0
Let's assume that for some reason each of these subsystems needs to
add a generation, and that generation 1 of 'another.app-extension'
depends on generation 1 of 'another.app'. We'll need to provide
schema managers for each that record that they've been run so we can
verify the result:
>>> ztapi.unprovideUtility(ISchemaManager, name='another.app')
>>> ztapi.unprovideUtility(ISchemaManager, name='another.app-extension')
>>> class FoundationSchemaManager(object):
... implements(ISchemaManager)
...
... minimum_generation = 1
... generation = 1
...
... def evolve(self, context, generation):
... root = context.connection.root()
... ordering = root.get('ordering', [])
... if generation == 1:
... ordering.append('foundation 1')
... print 'foundation generation 1'
... else:
... raise ValueError("Bummer")
... root['ordering'] = ordering # ping persistence
... transaction.commit()
>>> class DependentSchemaManager(object):
... implements(ISchemaManager)
...
... minimum_generation = 1
... generation = 1
...
... def evolve(self, context, generation):
... root = context.connection.root()
... ordering = root.get('ordering', [])
... if generation == 1:
... ordering.append('dependent 1')
... print 'dependent generation 1'
... else:
... raise ValueError("Bummer")
... root['ordering'] = ordering # ping persistence
... transaction.commit()
>>> manager1 = FoundationSchemaManager()
>>> manager2 = DependentSchemaManager()
>>> ztapi.provideUtility(
... ISchemaManager, manager1, name='another.app')
>>> ztapi.provideUtility(
... ISchemaManager, manager2, name='another.app-extension')
Evolving the database now will always run the 'another.app' evolver
before the 'another.app-extension' evolver:
>>> event = DatabaseOpenedEventStub(db)
>>> evolveMinimumSubscriber(event)
foundation generation 1
dependent generation 1
>>> root['ordering']
['foundation 1', 'dependent 1']
Installation
------------
In the the example above, we manually initialized the answers. We
shouldn't have to do that manually. The application should be able to
do that automatically.
IInstallableSchemaManager extends ISchemaManager, providing an install
method for performing an intial installation of an application. This
is a better alternative than registering database-opened subscribers.
Let's define a new schema manager that includes installation:
>>> ztapi.unprovideUtility(ISchemaManager, name='some.app')
>>> from zope.app.generations.interfaces import IInstallableSchemaManager
>>> class MySchemaManager(object):
... implements(IInstallableSchemaManager)
...
... minimum_generation = 1
... generation = 2
...
... def install(self, context):
... root = context.connection.root()
... root['answers'] = {'Hello': 'Hi & how do you do?',
... 'Meaning of life?': '42',
... 'four < ?': 'four < five'}
... transaction.commit()
...
... def evolve(self, context, generation):
... root = context.connection.root()
... answers = root['answers']
... if generation == 1:
... for question, answer in answers.items():
... answers[question] = cgi.escape(answer)
... elif generation == 2:
... for question, answer in answers.items():
... del answers[question]
... answers[cgi.escape(question)] = answer
... else:
... raise ValueError("Bummer")
... root['answers'] = answers # ping persistence
... transaction.commit()
>>> manager = MySchemaManager()
>>> ztapi.provideUtility(ISchemaManager, manager, name='some.app')
Now, lets open a new database:
>>> db.close()
>>> db = DB()
>>> conn = db.open()
>>> 'answers' in conn.root()
False
>>> event = DatabaseOpenedEventStub(db)
>>> evolveMinimumSubscriber(event)
>>> conn.sync()
>>> root = conn.root()
>>> pprint(root['answers'])
{'Hello': 'Hi & how do you do?',
'Meaning of life?': '42',
'four < ?': 'four < five'}
>>> root[generations_key]['some.app']
2
=======
CHANGES
=======
3.6.1 (2012-01-23)
------------------
- Replaced an undeclared test dependency on ``zope.app.authentication`` with
``zope.password``.
3.6.0 (2010-09-17)
------------------
- ``zope.app.generations`` depended on ``zope.app.applicationcontrol`` but
did not declare it. Modernized dependecy to ``zope.applicationcontrol`` as
the needed interface has been moved there.
- Using python's ``doctest`` module instead of deprecated
``zope.testing.doctest[unit]``.
- Replaced a testing dependency on ``zope.app.securitypolicy`` with one on
``zope.securitypolicy``.
3.5.1 (2010-01-08)
------------------
- Depend on new ``zope.processlifetime`` interfaces instead of using
BBB imports from ``zope.app.appsetup``.
- Fix ftesting.zcml due to ``zope.securitypolicy`` update.
- Fix tests using a newer zope.publisher that requires zope.login.
3.5.0 (2009-04-05)
------------------
- Moved ``getRootFolder`` utility method from
``zope.app.zopeappgenerations`` to ``zope.app.generations.utility``.
- Removed not necessary install dependency on ``zope.app.testing``.
3.4.2 (2009-01-27)
------------------
- Provide more logging output for the various stages and actions of evolving a
database.
- Fixed bug: A failing last generation would allow starting an app server
without having evolved to the minimum generation.
- Substitute zope.app.zapi by direct calls to its wrapped apis. See
bug 219302.
- Corrected author email and home page address.
3.4.1 (2007-10-31)
------------------
- Resolve ``ZopeSecurityPolicy`` deprecation warning.
3.4.0 (2007-10-24)
------------------
- Initial release independent of the main Zope tree.
Keywords: zope3 zodb schema generation
Platform: UNKNOWN
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Web Environment
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Zope Public License
Classifier: Programming Language :: Python
Classifier: Natural Language :: English
Classifier: Operating System :: OS Independent
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Framework :: Zope3
zope.app.generations-3.6.1/bootstrap.py 0000664 0001771 0002004 00000003373 11707310564 021321 0 ustar menesis menesis 0000000 0000000 ##############################################################################
#
# Copyright (c) 2006 Zope Foundation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
##############################################################################
"""Bootstrap a buildout-based project
Simply run this script in a directory containing a buildout.cfg.
The script accepts buildout command-line options, so you can
use the -c option to specify an alternate configuration file.
$Id: bootstrap.py 124134 2012-01-23 15:20:44Z menesis $
"""
import os, shutil, sys, tempfile, urllib2
tmpeggs = tempfile.mkdtemp()
ez = {}
exec urllib2.urlopen('http://peak.telecommunity.com/dist/ez_setup.py'
).read() in ez
ez['use_setuptools'](to_dir=tmpeggs, download_delay=0)
import pkg_resources
cmd = 'from setuptools.command.easy_install import main; main()'
if sys.platform == 'win32':
cmd = '"%s"' % cmd # work around spawn lamosity on windows
ws = pkg_resources.working_set
assert os.spawnle(
os.P_WAIT, sys.executable, sys.executable,
'-c', cmd, '-mqNxd', tmpeggs, 'zc.buildout',
dict(os.environ,
PYTHONPATH=
ws.find(pkg_resources.Requirement.parse('setuptools')).location
),
) == 0
ws.add_entry(tmpeggs)
ws.require('zc.buildout')
import zc.buildout.buildout
zc.buildout.buildout.main(sys.argv[1:] + ['bootstrap'])
shutil.rmtree(tmpeggs)
zope.app.generations-3.6.1/COPYRIGHT.txt 0000664 0001771 0002004 00000000040 11707310564 021027 0 ustar menesis menesis 0000000 0000000 Zope Foundation and Contributors zope.app.generations-3.6.1/src/ 0000775 0001771 0002004 00000000000 11707310574 017514 5 ustar menesis menesis 0000000 0000000 zope.app.generations-3.6.1/src/zope.app.generations.egg-info/ 0000775 0001771 0002004 00000000000 11707310574 025257 5 ustar menesis menesis 0000000 0000000 zope.app.generations-3.6.1/src/zope.app.generations.egg-info/PKG-INFO 0000664 0001771 0002004 00000046161 11707310567 026366 0 ustar menesis menesis 0000000 0000000 Metadata-Version: 1.1
Name: zope.app.generations
Version: 3.6.1
Summary: Zope Application Schema Generations
Home-page: http://pypi.python.org/pypi/zope.app.generations
Author: Zope Corporation and Contributors
Author-email: zope-dev@zope.org
License: ZPL 2.1
Description: Generations are a way of updating objects in the database when the application
schema changes. An application schema is essentially the structure of data,
the structure of classes in the case of ZODB or the table descriptions in the
case of a relational database.
.. contents::
======================
Detailed Documentation
======================
Generations are a way of updating objects in the database when the application
schema changes. An application schema is essentially the structure of data,
the structure of classes in the case of ZODB or the table descriptions in the
case of a relational database.
When you change your application's data structures, for example,
you change the semantic meaning of an existing field in a class, you will
have a problem with databases that were created before your change. For a
more thorough discussion and possible solutions, see
http://wiki.zope.org/zope3/DatabaseGenerations
We will be using the component architecture, and we will need a database and a
connection:
>>> import cgi
>>> from pprint import pprint
>>> from zope.interface import implements
>>> from zope.app.testing import ztapi
>>> from ZODB.tests.util import DB
>>> db = DB()
>>> conn = db.open()
>>> root = conn.root()
Imagine that our application is an oracle: you can teach it to react to
phrases. Let's keep it simple and store the data in a dict:
>>> root['answers'] = {'Hello': 'Hi & how do you do?',
... 'Meaning of life?': '42',
... 'four < ?': 'four < five'}
>>> import transaction
>>> transaction.commit()
Initial setup
-------------
Here's some generations-specific code. We will create and register a
SchemaManager. SchemaManagers are responsible for the actual updates of the
database. This one will be just a dummy. The point here is to make the
generations module aware that our application supports generations.
The default implementation of SchemaManager is not suitable for this test
because it uses Python modules to manage generations. For now, it
will be just fine, since we don't want it to do anything just yet.
>>> from zope.app.generations.interfaces import ISchemaManager
>>> from zope.app.generations.generations import SchemaManager
>>> dummy_manager = SchemaManager(minimum_generation=0, generation=0)
>>> ztapi.provideUtility(ISchemaManager, dummy_manager, name='some.app')
'some.app' is a unique identifier. You should use a URI or the dotted name
of your package.
When you start Zope and a database is opened, an event
IDatabaseOpenedWithRoot is sent. Zope registers
evolveMinimumSubscriber by default as a handler for this event. Let's
simulate this:
>>> class DatabaseOpenedEventStub(object):
... def __init__(self, database):
... self.database = database
>>> event = DatabaseOpenedEventStub(db)
>>> from zope.app.generations.generations import evolveMinimumSubscriber
>>> evolveMinimumSubscriber(event)
The consequence of this action is that now the database contains the fact
that our current schema number is 0. When we update the schema, Zope3 will
have an idea of what the starting point was. Here, see?
>>> from zope.app.generations.generations import generations_key
>>> root[generations_key]['some.app']
0
In real life you should never have to bother with this key directly,
but you should be aware that it exists.
Upgrade scenario
----------------
Back to the story. Some time passes and one of our clients gets hacked because
we forgot to escape HTML special characters! The horror! We must fix this
problem ASAP without losing any data. We decide to use generations to impress
our peers.
Let's update the schema manager (drop the old one and install a new custom
one):
>>> ztapi.unprovideUtility(ISchemaManager, name='some.app')
>>> class MySchemaManager(object):
... implements(ISchemaManager)
...
... minimum_generation = 1
... generation = 2
...
... def evolve(self, context, generation):
... root = context.connection.root()
... answers = root['answers']
... if generation == 1:
... for question, answer in answers.items():
... answers[question] = cgi.escape(answer)
... elif generation == 2:
... for question, answer in answers.items():
... del answers[question]
... answers[cgi.escape(question)] = answer
... else:
... raise ValueError("Bummer")
... root['answers'] = answers # ping persistence
... transaction.commit()
>>> manager = MySchemaManager()
>>> ztapi.provideUtility(ISchemaManager, manager, name='some.app')
We have set `minimum_generation` to 1. That means that our application
will refuse to run with a database older than generation 1. The `generation`
attribute is set to 2, which means that the latest generation that this
SchemaManager knows about is 2.
evolve() is the workhorse here. Its job is to get the database from
`generation`-1 to `generation`. It gets a context which has the attribute
'connection', which is a connection to the ZODB. You can use that to change
objects like in this example.
In this particular implementation generation 1 escapes the answers (say,
critical, because they can be entered by anyone!), generation 2 escapes the
questions (say, less important, because these can be entered by authorized
personell only).
In fact, you don't really need a custom implementation of ISchemaManager. One
is available, we have used it for a dummy previously. It uses Python modules
for organization of evolver functions. See its docstring for more information.
In real life you will have much more complex object structures than the one
here. To make your life easier, there are two very useful functions available
in zope.app.generations.utility: findObjectsMatching() and
findObjectsProviding(). They will dig through containers recursively to help
you seek out old objects that you wish to update, by interface or by some other
criteria. They are easy to understand, check their docstrings.
Generations in action
---------------------
So, our furious client downloads our latest code and restarts Zope. The event
is automatically sent again:
>>> event = DatabaseOpenedEventStub(db)
>>> evolveMinimumSubscriber(event)
Shazam! The client is happy again!
>>> pprint(root['answers'])
{'Hello': 'Hi & how do you do?',
'Meaning of life?': '42',
'four < ?': 'four < five'}
Because evolveMinimumSubscriber is very lazy, it only updates the database just
enough so that your application can use it (to the `minimum_generation`, that
is). Indeed, the marker indicates that the database generation has been bumped
to 1:
>>> root[generations_key]['some.app']
1
We see that generations are working, so we decide to take the next step
and evolve to generation 2. Let's see how this can be done manually:
>>> from zope.app.generations.generations import evolve
>>> evolve(db)
>>> pprint(root['answers'])
{'Hello': 'Hi & how do you do?',
'Meaning of life?': '42',
'four < ?': 'four < five'}
>>> root[generations_key]['some.app']
2
Default behaviour of `evolve` upgrades to the latest generation provided by
the SchemaManager. You can use the `how` argument to evolve() when you want
just to check if you need to update or if you want to be lazy like the
subscriber which we have called previously.
Ordering of schema managers
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Frequently subsystems used to compose an application rely on other
subsystems to operate properly. If both subsystems provide schema
managers, it is often helpful to know the order in which the evolvers
will be invoked. This allows a framework and it's clients to be able
to evolve in concert, and the clients can know that the framework will
be evolved before or after itself.
This can be accomplished by controlling the names of the schema
manager utilities. The schema managers are run in the order
determined by sorting their names.
>>> manager1 = SchemaManager(minimum_generation=0, generation=0)
>>> manager2 = SchemaManager(minimum_generation=0, generation=0)
>>> ztapi.provideUtility(
... ISchemaManager, manager1, name='another.app')
>>> ztapi.provideUtility(
... ISchemaManager, manager2, name='another.app-extension')
Notice how the name of the first package is used to create a namespace
for dependent packages. This is not a requirement of the framework,
but a convenient pattern for this usage.
Let's evolve the database to establish these generations:
>>> event = DatabaseOpenedEventStub(db)
>>> evolveMinimumSubscriber(event)
>>> root[generations_key]['another.app']
0
>>> root[generations_key]['another.app-extension']
0
Let's assume that for some reason each of these subsystems needs to
add a generation, and that generation 1 of 'another.app-extension'
depends on generation 1 of 'another.app'. We'll need to provide
schema managers for each that record that they've been run so we can
verify the result:
>>> ztapi.unprovideUtility(ISchemaManager, name='another.app')
>>> ztapi.unprovideUtility(ISchemaManager, name='another.app-extension')
>>> class FoundationSchemaManager(object):
... implements(ISchemaManager)
...
... minimum_generation = 1
... generation = 1
...
... def evolve(self, context, generation):
... root = context.connection.root()
... ordering = root.get('ordering', [])
... if generation == 1:
... ordering.append('foundation 1')
... print 'foundation generation 1'
... else:
... raise ValueError("Bummer")
... root['ordering'] = ordering # ping persistence
... transaction.commit()
>>> class DependentSchemaManager(object):
... implements(ISchemaManager)
...
... minimum_generation = 1
... generation = 1
...
... def evolve(self, context, generation):
... root = context.connection.root()
... ordering = root.get('ordering', [])
... if generation == 1:
... ordering.append('dependent 1')
... print 'dependent generation 1'
... else:
... raise ValueError("Bummer")
... root['ordering'] = ordering # ping persistence
... transaction.commit()
>>> manager1 = FoundationSchemaManager()
>>> manager2 = DependentSchemaManager()
>>> ztapi.provideUtility(
... ISchemaManager, manager1, name='another.app')
>>> ztapi.provideUtility(
... ISchemaManager, manager2, name='another.app-extension')
Evolving the database now will always run the 'another.app' evolver
before the 'another.app-extension' evolver:
>>> event = DatabaseOpenedEventStub(db)
>>> evolveMinimumSubscriber(event)
foundation generation 1
dependent generation 1
>>> root['ordering']
['foundation 1', 'dependent 1']
Installation
------------
In the the example above, we manually initialized the answers. We
shouldn't have to do that manually. The application should be able to
do that automatically.
IInstallableSchemaManager extends ISchemaManager, providing an install
method for performing an intial installation of an application. This
is a better alternative than registering database-opened subscribers.
Let's define a new schema manager that includes installation:
>>> ztapi.unprovideUtility(ISchemaManager, name='some.app')
>>> from zope.app.generations.interfaces import IInstallableSchemaManager
>>> class MySchemaManager(object):
... implements(IInstallableSchemaManager)
...
... minimum_generation = 1
... generation = 2
...
... def install(self, context):
... root = context.connection.root()
... root['answers'] = {'Hello': 'Hi & how do you do?',
... 'Meaning of life?': '42',
... 'four < ?': 'four < five'}
... transaction.commit()
...
... def evolve(self, context, generation):
... root = context.connection.root()
... answers = root['answers']
... if generation == 1:
... for question, answer in answers.items():
... answers[question] = cgi.escape(answer)
... elif generation == 2:
... for question, answer in answers.items():
... del answers[question]
... answers[cgi.escape(question)] = answer
... else:
... raise ValueError("Bummer")
... root['answers'] = answers # ping persistence
... transaction.commit()
>>> manager = MySchemaManager()
>>> ztapi.provideUtility(ISchemaManager, manager, name='some.app')
Now, lets open a new database:
>>> db.close()
>>> db = DB()
>>> conn = db.open()
>>> 'answers' in conn.root()
False
>>> event = DatabaseOpenedEventStub(db)
>>> evolveMinimumSubscriber(event)
>>> conn.sync()
>>> root = conn.root()
>>> pprint(root['answers'])
{'Hello': 'Hi & how do you do?',
'Meaning of life?': '42',
'four < ?': 'four < five'}
>>> root[generations_key]['some.app']
2
=======
CHANGES
=======
3.6.1 (2012-01-23)
------------------
- Replaced an undeclared test dependency on ``zope.app.authentication`` with
``zope.password``.
3.6.0 (2010-09-17)
------------------
- ``zope.app.generations`` depended on ``zope.app.applicationcontrol`` but
did not declare it. Modernized dependecy to ``zope.applicationcontrol`` as
the needed interface has been moved there.
- Using python's ``doctest`` module instead of deprecated
``zope.testing.doctest[unit]``.
- Replaced a testing dependency on ``zope.app.securitypolicy`` with one on
``zope.securitypolicy``.
3.5.1 (2010-01-08)
------------------
- Depend on new ``zope.processlifetime`` interfaces instead of using
BBB imports from ``zope.app.appsetup``.
- Fix ftesting.zcml due to ``zope.securitypolicy`` update.
- Fix tests using a newer zope.publisher that requires zope.login.
3.5.0 (2009-04-05)
------------------
- Moved ``getRootFolder`` utility method from
``zope.app.zopeappgenerations`` to ``zope.app.generations.utility``.
- Removed not necessary install dependency on ``zope.app.testing``.
3.4.2 (2009-01-27)
------------------
- Provide more logging output for the various stages and actions of evolving a
database.
- Fixed bug: A failing last generation would allow starting an app server
without having evolved to the minimum generation.
- Substitute zope.app.zapi by direct calls to its wrapped apis. See
bug 219302.
- Corrected author email and home page address.
3.4.1 (2007-10-31)
------------------
- Resolve ``ZopeSecurityPolicy`` deprecation warning.
3.4.0 (2007-10-24)
------------------
- Initial release independent of the main Zope tree.
Keywords: zope3 zodb schema generation
Platform: UNKNOWN
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Web Environment
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Zope Public License
Classifier: Programming Language :: Python
Classifier: Natural Language :: English
Classifier: Operating System :: OS Independent
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Framework :: Zope3
zope.app.generations-3.6.1/src/zope.app.generations.egg-info/dependency_links.txt 0000664 0001771 0002004 00000000001 11707310567 031327 0 ustar menesis menesis 0000000 0000000
zope.app.generations-3.6.1/src/zope.app.generations.egg-info/namespace_packages.txt 0000664 0001771 0002004 00000000016 11707310567 031611 0 ustar menesis menesis 0000000 0000000 zope
zope.app
zope.app.generations-3.6.1/src/zope.app.generations.egg-info/not-zip-safe 0000664 0001771 0002004 00000000001 11707310565 027505 0 ustar menesis menesis 0000000 0000000
zope.app.generations-3.6.1/src/zope.app.generations.egg-info/top_level.txt 0000664 0001771 0002004 00000000005 11707310567 030006 0 ustar menesis menesis 0000000 0000000 zope
zope.app.generations-3.6.1/src/zope.app.generations.egg-info/requires.txt 0000664 0001771 0002004 00000000343 11707310567 027661 0 ustar menesis menesis 0000000 0000000 setuptools
zope.app.renderer
zope.interface
zope.app.publication
ZODB3
zope.processlifetime
zope.applicationcontrol
[test]
zope.app.testing
zope.app.zcmlfiles
zope.login
zope.password
zope.publisher >= 3.12
zope.securitypolicy zope.app.generations-3.6.1/src/zope.app.generations.egg-info/SOURCES.txt 0000664 0001771 0002004 00000002747 11707310567 027157 0 ustar menesis menesis 0000000 0000000 CHANGES.txt
COPYRIGHT.txt
LICENSE.txt
README.txt
bootstrap.py
buildout.cfg
setup.py
src/zope/__init__.py
src/zope.app.generations.egg-info/PKG-INFO
src/zope.app.generations.egg-info/SOURCES.txt
src/zope.app.generations.egg-info/dependency_links.txt
src/zope.app.generations.egg-info/namespace_packages.txt
src/zope.app.generations.egg-info/not-zip-safe
src/zope.app.generations.egg-info/requires.txt
src/zope.app.generations.egg-info/top_level.txt
src/zope/app/__init__.py
src/zope/app/generations/README.txt
src/zope/app/generations/__init__.py
src/zope/app/generations/configure.zcml
src/zope/app/generations/ftesting.zcml
src/zope/app/generations/generations.py
src/zope/app/generations/interfaces.py
src/zope/app/generations/subscriber.zcml
src/zope/app/generations/testing.py
src/zope/app/generations/tests.py
src/zope/app/generations/utility.py
src/zope/app/generations/browser/__init__.py
src/zope/app/generations/browser/configure.zcml
src/zope/app/generations/browser/managerdetails.pt
src/zope/app/generations/browser/managerdetails.py
src/zope/app/generations/browser/managers.pt
src/zope/app/generations/browser/managers.py
src/zope/app/generations/browser/tests.py
src/zope/app/generations/demo/__init__.py
src/zope/app/generations/demo/evolve1.py
src/zope/app/generations/demo/evolve2.py
src/zope/app/generations/demo/evolve3.py
src/zope/app/generations/demo/install.py
src/zope/app/generations/demo2/__init__.py
src/zope/app/generations/demo3/__init__.py
src/zope/app/generations/demo3/install.py zope.app.generations-3.6.1/src/zope/ 0000775 0001771 0002004 00000000000 11707310574 020471 5 ustar menesis menesis 0000000 0000000 zope.app.generations-3.6.1/src/zope/__init__.py 0000664 0001771 0002004 00000000310 11707310564 022573 0 ustar menesis menesis 0000000 0000000 # this is a namespace package
try:
import pkg_resources
pkg_resources.declare_namespace(__name__)
except ImportError:
import pkgutil
__path__ = pkgutil.extend_path(__path__, __name__)
zope.app.generations-3.6.1/src/zope/app/ 0000775 0001771 0002004 00000000000 11707310574 021251 5 ustar menesis menesis 0000000 0000000 zope.app.generations-3.6.1/src/zope/app/generations/ 0000775 0001771 0002004 00000000000 11707310574 023567 5 ustar menesis menesis 0000000 0000000 zope.app.generations-3.6.1/src/zope/app/generations/configure.zcml 0000664 0001771 0002004 00000001564 11707310564 026444 0 ustar menesis menesis 0000000 0000000
Evolver 1
\n'), ('to', 1)] [('from', 1), ('info', u'Evolver 2
\n'), ('to', 2)] [('from', 2), ('info', ''), ('to', 3)] We'd better clean up: >>> tearDown() """ id = property(lambda self: self.request['id']) def getEvolvers(self): id = self.id manager = zope.component.getUtility(ISchemaManager, id) evolvers = [] for gen in range(manager.minimum_generation, manager.generation): info = manager.getInfo(gen+1) if info is None: info = '' else: # XXX: the renderer *expects* unicode as input encoding (ajung) renderer = ReStructuredTextToHTMLRenderer( unicode(info), self.request) info = renderer.render() evolvers.append({'from': gen, 'to': gen+1, 'info': info}) return evolvers zope.app.generations-3.6.1/src/zope/app/generations/generations.py 0000664 0001771 0002004 00000040734 11707310564 026466 0 ustar menesis menesis 0000000 0000000 ############################################################################## # # Copyright (c) 2004 Zope Foundation and Contributors. # All Rights Reserved. # # This software is subject to the provisions of the Zope Public License, # Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution. # THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED # WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED # WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS # FOR A PARTICULAR PURPOSE. # ############################################################################## """Experimental support for application database generations $Id: generations.py 124134 2012-01-23 15:20:44Z menesis $ """ __docformat__ = 'restructuredtext' import logging import transaction import zope.component import zope.interface from interfaces import GenerationTooHigh, GenerationTooLow, UnableToEvolve from interfaces import ISchemaManager, IInstallableSchemaManager logger = logging.getLogger('zope.app.generations') generations_key = 'zope.app.generations' class SchemaManager(object): """Schema manager Schema managers implement `IInstallableSchemaManager` using scripts provided as module methods. You create a schema manager by providing mimumum and maximum generations and a package providing modules named ``evolveN``, where ``N`` is a generation number. Each module provides a function, `evolve` that evolves a database from the previous generation. For the sake of the example, we'll use the demo package defined in here. See the modules there for simple examples of evolution scripts. So, if we'll create a SchemaManager: >>> manager = SchemaManager(1, 3, 'zope.app.generations.demo') and we'll create a test database and context: >>> from ZODB.tests.util import DB >>> db = DB() >>> context = Context() >>> context.connection = db.open() Then we'll evolve the database from generation 1 to 3: >>> manager.evolve(context, 2) >>> manager.evolve(context, 3) >>> transaction.commit() The demo evolvers simply record their data in a root key: >>> from zope.app.generations.demo import key >>> conn = db.open() >>> conn.root()[key] (2, 3) You can get the information for each evolver by specifying the destination generation of the evolver as argument to the `getInfo()` method: >>> manager.getInfo(1) 'Evolver 1' >>> manager.getInfo(2) 'Evolver 2' >>> manager.getInfo(3) is None True If a package provides an install script, then it will be called when the manager's intall method is called: >>> conn.sync() >>> del conn.root()[key] >>> transaction.commit() >>> conn.root().get(key) >>> manager.install(context) >>> transaction.commit() >>> conn.sync() >>> conn.root()[key] ('installed',) If there is not install script, the manager will do nothing on an install: >>> manager = SchemaManager(1, 3, 'zope.app.generations.demo2') >>> manager.install(context) We handle ImportErrors within the script specially, so they get promoted: >>> manager = SchemaManager(1, 3, 'zope.app.generations.demo3') >>> manager.install(context) Traceback (most recent call last): ImportError: No module named nonexistingmodule We'd better clean up: >>> context.connection.close() >>> conn.close() >>> db.close() """ zope.interface.implements(IInstallableSchemaManager) def __init__(self, minimum_generation=0, generation=0, package_name=None): if generation < minimum_generation: raise ValueError("generation is less than minimum_generation", generation, minimum_generation) if minimum_generation < 0: raise ValueError("generations must be non-negative", minimum_generation) if generation and not package_name: raise ValueError("A package name must be supplied if the" " generation is non-zero") self.minimum_generation = minimum_generation self.generation = generation self.package_name = package_name def evolve(self, context, generation): """Evolve a database to reflect software/schema changes """ name = "%s.evolve%d" % (self.package_name, generation) evolver = __import__(name, {}, {}, ['*']) evolver.evolve(context) def install(self, context): """Evolve a database to reflect software/schema changes """ name = "%s.install" % self.package_name try: evolver = __import__(name, {}, {}, ['*']) except ImportError, m: if str(m) not in ('No module named %s' % name, 'No module named install'): # This was an import error *within* the module, so we re-raise. raise else: evolver.evolve(context) def getInfo(self, generation): """Get the information from the evolver function's doc string.""" evolver = __import__( "%s.evolve%d" % (self.package_name, generation), {}, {}, ['*']) return evolver.evolve.__doc__ class Context(object): pass def findManagers(): # Hook to let Chris use this for Zope 2 return zope.component.getUtilitiesFor(ISchemaManager) def PersistentDict(): # Another hook to let Chris use this for Zope 2 import persistent.dict return persistent.dict.PersistentDict() EVOLVE, EVOLVENOT, EVOLVEMINIMUM = 'EVOLVE', 'EVOLVENOT', 'EVOLVEMINIMUM' def evolve(db, how=EVOLVE): """Evolve a database We evolve a database using registered application schema managers. Here's an example (silly) schema manager: >>> from zope.app.generations.interfaces import ISchemaManager >>> class FauxApp(object): ... zope.interface.implements(ISchemaManager) ... ... erron = None # Raise an error is asked to evolve to this ... ... def __init__(self, name, minimum_generation, generation): ... self.name, self.generation = name, generation ... self.minimum_generation = minimum_generation ... ... def evolve(self, context, generation): ... if generation == self.erron: ... raise ValueError(generation) ... ... context.connection.root()[self.name] = generation Evolving a database will cause log messages to be written, so we need a logging handler: >>> from zope.testing import loggingsupport >>> loghandler = loggingsupport.InstalledHandler('zope.app.generations') >>> def print_log(): ... print loghandler ... loghandler.clear() We also need to set up the component system, since we'll be registering utilities: >>> from zope.app.testing.placelesssetup import setUp, tearDown >>> setUp() Now, we'll create and register some handlers: >>> from zope.app.testing import ztapi >>> app1 = FauxApp('app1', 0, 1) >>> ztapi.provideUtility(ISchemaManager, app1, name='app1') >>> app2 = FauxApp('app2', 5, 11) >>> ztapi.provideUtility(ISchemaManager, app2, name='app2') If we create a new database, and evolve it, we'll simply update the generation data: >>> from ZODB.tests.util import DB >>> db = DB(database_name='testdb') >>> conn = db.open() >>> root = conn.root() >>> evolve(db) >>> conn.sync() >>> root[generations_key]['app1'] 1 >>> root[generations_key]['app2'] 11 >>> print_log() zope.app.generations INFO testdb: evolving in mode EVOLVE But nothing will have been done to the database: >>> root.get('app1') >>> root.get('app2') Now if we increase the generation of one of the apps: >>> app1.generation += 1 >>> evolve(db) >>> print_log() zope.app.generations INFO testdb: evolving in mode EVOLVE zope.app.generations INFO testdb/app1: currently at generation 1, targetting generation 2 zope.app.generations DEBUG testdb/app1: evolving to generation 2 zope.app.generations DEBUG testdb/app2: up-to-date at generation 11 We'll see that the generation data has updated: >>> conn.sync() >>> root[generations_key]['app1'] 2 >>> root[generations_key]['app2'] 11 And that the database was updated for that application: >>> root.get('app1') 2 >>> root.get('app2') If there is an error updating a particular generation, but the generation is greater than the minimum generation, then we won't get an error from evolve, but we will get a log message. >>> app1.erron = 4 >>> app1.generation = 7 >>> evolve(db) >>> print_log() zope.app.generations INFO testdb: evolving in mode EVOLVE zope.app.generations INFO testdb/app1: currently at generation 2, targetting generation 7 zope.app.generations DEBUG testdb/app1: evolving to generation 3 zope.app.generations DEBUG testdb/app1: evolving to generation 4 zope.app.generations ERROR testdb/app1: failed to evolve to generation 4 zope.app.generations DEBUG testdb/app2: up-to-date at generation 11 The database will have been updated for previous generations: >>> conn.sync() >>> root[generations_key]['app1'] 3 >>> root.get('app1') 3 If we set the minimum generation for app1 to something greater than 3: >>> app1.minimum_generation = 4 Then we'll get an error if we try to evolve, since we can't get past 3 and 3 is less than 4: >>> evolve(db) Traceback (most recent call last): ... UnableToEvolve: (4, u'app1', 7) We'll also get a log entry: >>> print_log() zope.app.generations INFO testdb: evolving in mode EVOLVE zope.app.generations INFO testdb/app1: currently at generation 3, targetting generation 7 zope.app.generations DEBUG testdb/app1: evolving to generation 4 zope.app.generations ERROR testdb/app1: failed to evolve to generation 4 So far, we've used evolve in its default policy, in which we evolve as far as we can up to the current generation. There are two other policies: EVOLVENOT -- Don't evolve, but make sure that the application is at the minimum generation EVOLVEMINIMUM -- Evolve only to the minimum generation Let's change unset erron for app1 so we don't get an error when we try to evolve. >>> app1.erron = None Now, we'll call evolve with EVOLVENOT: >>> evolve(db, EVOLVENOT) Traceback (most recent call last): ... GenerationTooLow: (3, u'app1', 4) >>> print_log() zope.app.generations INFO testdb: evolving in mode EVOLVENOT zope.app.generations ERROR testdb/app1: current generation too low (3 < 4) but mode is EVOLVENOT We got an error because we aren't at the minimum generation for app1. The database generation for app1 is still 3 because we didn't do any evolution: >>> conn.sync() >>> root[generations_key]['app1'] 3 >>> root.get('app1') 3 Now, if we use EVOLVEMINIMUM instead, we'll evolve to the minimum generation: >>> evolve(db, EVOLVEMINIMUM) >>> conn.sync() >>> root[generations_key]['app1'] 4 >>> root.get('app1') 4 >>> print_log() zope.app.generations INFO testdb: evolving in mode EVOLVEMINIMUM zope.app.generations INFO testdb/app1: currently at generation 3, targetting generation 4 zope.app.generations DEBUG testdb/app1: evolving to generation 4 zope.app.generations DEBUG testdb/app2: up-to-date at generation 11 If we happen to install an app that has a generation that is less than the database generation, we'll get an error, because there is no way to get the database to a generation that the app understands: >>> app1.generation = 2 >>> app1.minimum_generation = 0 >>> evolve(db) Traceback (most recent call last): ... GenerationTooHigh: (4, u'app1', 2) >>> print_log() zope.app.generations INFO testdb: evolving in mode EVOLVE zope.app.generations ERROR testdb/app1: current generation too high (4 > 2) We'd better clean up: >>> loghandler.uninstall() >>> conn.close() >>> db.close() >>> tearDown() """ db_name = db.database_name or 'main db' logger.info('%s: evolving in mode %s' % (db_name, how)) conn = db.open() try: context = Context() context.connection = conn root = conn.root() generations = root.get(generations_key) if generations is None: generations = root[generations_key] = PersistentDict() transaction.commit() for key, manager in sorted(findManagers()): generation = generations.get(key) if generation == manager.generation: logger.debug('%s/%s: up-to-date at generation %s' % (db_name, key, generation)) continue if generation is None: # This is a new database, so no old data if IInstallableSchemaManager.providedBy(manager): try: logger.info("%s/%s: running install generation", db_name, key) manager.install(context) except: transaction.abort() logger.exception("%s/%s: failed to run install", db_name, key) raise generations[key] = manager.generation transaction.commit() continue if generation > manager.generation: logger.error('%s/%s: current generation too high (%d > %d)', db_name, key, generation, manager.generation) raise GenerationTooHigh(generation, key, manager.generation) if generation < manager.minimum_generation: if how == EVOLVENOT: logger.error('%s/%s: current generation too low ' '(%d < %d) but mode is %s', db_name, key, generation, manager.minimum_generation, how) raise GenerationTooLow( generation, key, manager.minimum_generation) else: if how != EVOLVE: continue if how == EVOLVEMINIMUM: target = manager.minimum_generation else: target = manager.generation logger.info('%s/%s: currently at generation %d, targetting generation %d', db_name, key, generation, target) while generation < target: generation += 1 try: transaction.begin() logger.debug('%s/%s: evolving to generation %d', db_name, key, generation) manager.evolve(context, generation) generations[key] = generation transaction.commit() except: # An unguarded handler is intended here transaction.abort() logger.exception( "%s/%s: failed to evolve to generation %d", db_name, key, generation) if generation <= manager.minimum_generation: raise UnableToEvolve(generation, key, manager.generation) break finally: conn.close() def evolveSubscriber(event): evolve(event.database, EVOLVE) def evolveNotSubscriber(event): evolve(event.database, EVOLVENOT) def evolveMinimumSubscriber(event): evolve(event.database, EVOLVEMINIMUM) zope.app.generations-3.6.1/src/zope/app/generations/README.txt 0000664 0001771 0002004 00000031754 11707310564 025276 0 ustar menesis menesis 0000000 0000000 Generations are a way of updating objects in the database when the application schema changes. An application schema is essentially the structure of data, the structure of classes in the case of ZODB or the table descriptions in the case of a relational database. When you change your application's data structures, for example, you change the semantic meaning of an existing field in a class, you will have a problem with databases that were created before your change. For a more thorough discussion and possible solutions, see http://wiki.zope.org/zope3/DatabaseGenerations We will be using the component architecture, and we will need a database and a connection: >>> import cgi >>> from pprint import pprint >>> from zope.interface import implements >>> from zope.app.testing import ztapi >>> from ZODB.tests.util import DB >>> db = DB() >>> conn = db.open() >>> root = conn.root() Imagine that our application is an oracle: you can teach it to react to phrases. Let's keep it simple and store the data in a dict: >>> root['answers'] = {'Hello': 'Hi & how do you do?', ... 'Meaning of life?': '42', ... 'four < ?': 'four < five'} >>> import transaction >>> transaction.commit() Initial setup ------------- Here's some generations-specific code. We will create and register a SchemaManager. SchemaManagers are responsible for the actual updates of the database. This one will be just a dummy. The point here is to make the generations module aware that our application supports generations. The default implementation of SchemaManager is not suitable for this test because it uses Python modules to manage generations. For now, it will be just fine, since we don't want it to do anything just yet. >>> from zope.app.generations.interfaces import ISchemaManager >>> from zope.app.generations.generations import SchemaManager >>> dummy_manager = SchemaManager(minimum_generation=0, generation=0) >>> ztapi.provideUtility(ISchemaManager, dummy_manager, name='some.app') 'some.app' is a unique identifier. You should use a URI or the dotted name of your package. When you start Zope and a database is opened, an event IDatabaseOpenedWithRoot is sent. Zope registers evolveMinimumSubscriber by default as a handler for this event. Let's simulate this: >>> class DatabaseOpenedEventStub(object): ... def __init__(self, database): ... self.database = database >>> event = DatabaseOpenedEventStub(db) >>> from zope.app.generations.generations import evolveMinimumSubscriber >>> evolveMinimumSubscriber(event) The consequence of this action is that now the database contains the fact that our current schema number is 0. When we update the schema, Zope3 will have an idea of what the starting point was. Here, see? >>> from zope.app.generations.generations import generations_key >>> root[generations_key]['some.app'] 0 In real life you should never have to bother with this key directly, but you should be aware that it exists. Upgrade scenario ---------------- Back to the story. Some time passes and one of our clients gets hacked because we forgot to escape HTML special characters! The horror! We must fix this problem ASAP without losing any data. We decide to use generations to impress our peers. Let's update the schema manager (drop the old one and install a new custom one): >>> ztapi.unprovideUtility(ISchemaManager, name='some.app') >>> class MySchemaManager(object): ... implements(ISchemaManager) ... ... minimum_generation = 1 ... generation = 2 ... ... def evolve(self, context, generation): ... root = context.connection.root() ... answers = root['answers'] ... if generation == 1: ... for question, answer in answers.items(): ... answers[question] = cgi.escape(answer) ... elif generation == 2: ... for question, answer in answers.items(): ... del answers[question] ... answers[cgi.escape(question)] = answer ... else: ... raise ValueError("Bummer") ... root['answers'] = answers # ping persistence ... transaction.commit() >>> manager = MySchemaManager() >>> ztapi.provideUtility(ISchemaManager, manager, name='some.app') We have set `minimum_generation` to 1. That means that our application will refuse to run with a database older than generation 1. The `generation` attribute is set to 2, which means that the latest generation that this SchemaManager knows about is 2. evolve() is the workhorse here. Its job is to get the database from `generation`-1 to `generation`. It gets a context which has the attribute 'connection', which is a connection to the ZODB. You can use that to change objects like in this example. In this particular implementation generation 1 escapes the answers (say, critical, because they can be entered by anyone!), generation 2 escapes the questions (say, less important, because these can be entered by authorized personell only). In fact, you don't really need a custom implementation of ISchemaManager. One is available, we have used it for a dummy previously. It uses Python modules for organization of evolver functions. See its docstring for more information. In real life you will have much more complex object structures than the one here. To make your life easier, there are two very useful functions available in zope.app.generations.utility: findObjectsMatching() and findObjectsProviding(). They will dig through containers recursively to help you seek out old objects that you wish to update, by interface or by some other criteria. They are easy to understand, check their docstrings. Generations in action --------------------- So, our furious client downloads our latest code and restarts Zope. The event is automatically sent again: >>> event = DatabaseOpenedEventStub(db) >>> evolveMinimumSubscriber(event) Shazam! The client is happy again! >>> pprint(root['answers']) {'Hello': 'Hi & how do you do?', 'Meaning of life?': '42', 'four < ?': 'four < five'} Because evolveMinimumSubscriber is very lazy, it only updates the database just enough so that your application can use it (to the `minimum_generation`, that is). Indeed, the marker indicates that the database generation has been bumped to 1: >>> root[generations_key]['some.app'] 1 We see that generations are working, so we decide to take the next step and evolve to generation 2. Let's see how this can be done manually: >>> from zope.app.generations.generations import evolve >>> evolve(db) >>> pprint(root['answers']) {'Hello': 'Hi & how do you do?', 'Meaning of life?': '42', 'four < ?': 'four < five'} >>> root[generations_key]['some.app'] 2 Default behaviour of `evolve` upgrades to the latest generation provided by the SchemaManager. You can use the `how` argument to evolve() when you want just to check if you need to update or if you want to be lazy like the subscriber which we have called previously. Ordering of schema managers ~~~~~~~~~~~~~~~~~~~~~~~~~~~ Frequently subsystems used to compose an application rely on other subsystems to operate properly. If both subsystems provide schema managers, it is often helpful to know the order in which the evolvers will be invoked. This allows a framework and it's clients to be able to evolve in concert, and the clients can know that the framework will be evolved before or after itself. This can be accomplished by controlling the names of the schema manager utilities. The schema managers are run in the order determined by sorting their names. >>> manager1 = SchemaManager(minimum_generation=0, generation=0) >>> manager2 = SchemaManager(minimum_generation=0, generation=0) >>> ztapi.provideUtility( ... ISchemaManager, manager1, name='another.app') >>> ztapi.provideUtility( ... ISchemaManager, manager2, name='another.app-extension') Notice how the name of the first package is used to create a namespace for dependent packages. This is not a requirement of the framework, but a convenient pattern for this usage. Let's evolve the database to establish these generations: >>> event = DatabaseOpenedEventStub(db) >>> evolveMinimumSubscriber(event) >>> root[generations_key]['another.app'] 0 >>> root[generations_key]['another.app-extension'] 0 Let's assume that for some reason each of these subsystems needs to add a generation, and that generation 1 of 'another.app-extension' depends on generation 1 of 'another.app'. We'll need to provide schema managers for each that record that they've been run so we can verify the result: >>> ztapi.unprovideUtility(ISchemaManager, name='another.app') >>> ztapi.unprovideUtility(ISchemaManager, name='another.app-extension') >>> class FoundationSchemaManager(object): ... implements(ISchemaManager) ... ... minimum_generation = 1 ... generation = 1 ... ... def evolve(self, context, generation): ... root = context.connection.root() ... ordering = root.get('ordering', []) ... if generation == 1: ... ordering.append('foundation 1') ... print 'foundation generation 1' ... else: ... raise ValueError("Bummer") ... root['ordering'] = ordering # ping persistence ... transaction.commit() >>> class DependentSchemaManager(object): ... implements(ISchemaManager) ... ... minimum_generation = 1 ... generation = 1 ... ... def evolve(self, context, generation): ... root = context.connection.root() ... ordering = root.get('ordering', []) ... if generation == 1: ... ordering.append('dependent 1') ... print 'dependent generation 1' ... else: ... raise ValueError("Bummer") ... root['ordering'] = ordering # ping persistence ... transaction.commit() >>> manager1 = FoundationSchemaManager() >>> manager2 = DependentSchemaManager() >>> ztapi.provideUtility( ... ISchemaManager, manager1, name='another.app') >>> ztapi.provideUtility( ... ISchemaManager, manager2, name='another.app-extension') Evolving the database now will always run the 'another.app' evolver before the 'another.app-extension' evolver: >>> event = DatabaseOpenedEventStub(db) >>> evolveMinimumSubscriber(event) foundation generation 1 dependent generation 1 >>> root['ordering'] ['foundation 1', 'dependent 1'] Installation ------------ In the the example above, we manually initialized the answers. We shouldn't have to do that manually. The application should be able to do that automatically. IInstallableSchemaManager extends ISchemaManager, providing an install method for performing an intial installation of an application. This is a better alternative than registering database-opened subscribers. Let's define a new schema manager that includes installation: >>> ztapi.unprovideUtility(ISchemaManager, name='some.app') >>> from zope.app.generations.interfaces import IInstallableSchemaManager >>> class MySchemaManager(object): ... implements(IInstallableSchemaManager) ... ... minimum_generation = 1 ... generation = 2 ... ... def install(self, context): ... root = context.connection.root() ... root['answers'] = {'Hello': 'Hi & how do you do?', ... 'Meaning of life?': '42', ... 'four < ?': 'four < five'} ... transaction.commit() ... ... def evolve(self, context, generation): ... root = context.connection.root() ... answers = root['answers'] ... if generation == 1: ... for question, answer in answers.items(): ... answers[question] = cgi.escape(answer) ... elif generation == 2: ... for question, answer in answers.items(): ... del answers[question] ... answers[cgi.escape(question)] = answer ... else: ... raise ValueError("Bummer") ... root['answers'] = answers # ping persistence ... transaction.commit() >>> manager = MySchemaManager() >>> ztapi.provideUtility(ISchemaManager, manager, name='some.app') Now, lets open a new database: >>> db.close() >>> db = DB() >>> conn = db.open() >>> 'answers' in conn.root() False >>> event = DatabaseOpenedEventStub(db) >>> evolveMinimumSubscriber(event) >>> conn.sync() >>> root = conn.root() >>> pprint(root['answers']) {'Hello': 'Hi & how do you do?', 'Meaning of life?': '42', 'four < ?': 'four < five'} >>> root[generations_key]['some.app'] 2 zope.app.generations-3.6.1/src/zope/app/generations/__init__.py 0000664 0001771 0002004 00000001365 11707310564 025704 0 ustar menesis menesis 0000000 0000000 ############################################################################## # # Copyright (c) 2004 Zope Foundation and Contributors. # All Rights Reserved. # # This software is subject to the provisions of the Zope Public License, # Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution. # THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED # WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED # WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS # FOR A PARTICULAR PURPOSE. # ############################################################################## """Experimental support for application database generations $Id: __init__.py 124134 2012-01-23 15:20:44Z menesis $ """ zope.app.generations-3.6.1/src/zope/app/generations/utility.py 0000664 0001771 0002004 00000011267 11707310564 025652 0 ustar menesis menesis 0000000 0000000 ############################################################################## # # Copyright (c) 2004 Zope Foundation and Contributors. # All Rights Reserved. # # This software is subject to the provisions of the Zope Public License, # Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution. # THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED # WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED # WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS # FOR A PARTICULAR PURPOSE. # ############################################################################## """Utility functions for evolving database generations. $Id: utility.py 124134 2012-01-23 15:20:44Z menesis $ """ __docformat__ = "reStructuredText" import zope.app.publication.zopepublication def findObjectsMatching(root, condition): """Find all objects in the root that match the condition. The condition is a callable Python object that takes an object as an argument and must return `True` or `False`. All sub-objects of the root will also be searched recursively. All mapping objects providing `values()` are supported. Example: >>> class A(dict): ... def __init__(self, name): ... self.name = name >>> class B(dict): ... def __init__(self, name): ... self.name = name >>> class C(dict): ... def __init__(self, name): ... self.name = name >>> tree = A('a1') >>> tree['b1'] = B('b1') >>> tree['c1'] = C('c1') >>> tree['b1']['a2'] = A('a2') >>> tree['b1']['b2'] = B('b2') >>> tree['b1']['b2']['c2'] = C('c2') >>> tree['b1']['b2']['a3'] = A('a3') # Find all instances of class A >>> matches = findObjectsMatching(tree, lambda x: isinstance(x, A)) >>> names = [x.name for x in matches] >>> names.sort() >>> names ['a1', 'a2', 'a3'] # Find all objects having a '2' in the name >>> matches = findObjectsMatching(tree, lambda x: '2' in x.name) >>> names = [x.name for x in matches] >>> names.sort() >>> names ['a2', 'b2', 'c2'] """ if condition(root): yield root if hasattr(root, 'values'): for subobj in root.values(): for match in findObjectsMatching(subobj, condition): yield match def findObjectsProviding(root, interface): """Find all objects in the root that provide the specified interface. All sub-objects of the root will also be searched recursively. Example: >>> from zope.interface import Interface, implements >>> class IA(Interface): ... pass >>> class IB(Interface): ... pass >>> class IC(IA): ... pass >>> class A(dict): ... implements(IA) ... def __init__(self, name): ... self.name = name >>> class B(dict): ... implements(IB) ... def __init__(self, name): ... self.name = name >>> class C(dict): ... implements(IC) ... def __init__(self, name): ... self.name = name >>> tree = A('a1') >>> tree['b1'] = B('b1') >>> tree['c1'] = C('c1') >>> tree['b1']['a2'] = A('a2') >>> tree['b1']['b2'] = B('b2') >>> tree['b1']['b2']['c2'] = C('c2') >>> tree['b1']['b2']['a3'] = A('a3') # Find all objects that provide IB >>> matches = findObjectsProviding(tree, IB) >>> names = [x.name for x in matches] >>> names.sort() >>> names ['b1', 'b2'] # Find all objects that provide IA >>> matches = findObjectsProviding(tree, IA) >>> names = [x.name for x in matches] >>> names.sort() >>> names ['a1', 'a2', 'a3', 'c1', 'c2'] """ for match in findObjectsMatching(root, interface.providedBy): yield match def getRootFolder(context): """Get the root folder of the ZODB. We need some set up. Create a database: >>> from ZODB.tests.util import DB >>> from zope.app.generations.generations import Context >>> db = DB() >>> context = Context() >>> context.connection = db.open() >>> root = context.connection.root() Add a root folder: >>> from zope.site.folder import rootFolder >>> from zope.app.publication.zopepublication import ZopePublication >>> import transaction >>> root[ZopePublication.root_name] = rootFolder() >>> transaction.commit() Now we can get the root folder using the function: >>> getRootFolder(context) # doctest: +ELLIPSIS