testresources-0.2.7/0000755000175000017500000000000012076756602015567 5ustar robertcrobertc00000000000000testresources-0.2.7/NEWS0000644000175000017500000001362112076756342016272 0ustar robertcrobertc00000000000000--------------------------- testresources release notes --------------------------- IN DEVELOPMENT -------------- 0.2.7 ----- IMPROVEMENTS ~~~~~~~~~~~~ * FixtureResource was not triggering cleanups or resets between uses, this is fixed (but doing so cleanly involved a new extension point - ``_reset`` on ``TestResourceManager``. This is called from ``reset`` which should no longer be overridden. (Though overridden versions will still behave correctly - the change is backwards compatible). Lastly two new TestResult methods were added to track reset (as opposed to make and clean). (Robert Collins, James Westby, #1023423) * TestResourceManager.reset() was not taking dependency dirtiness into consideration. (Brian Sutherland, #783488) 0.2.6 ----- IMPROVEMENTS ~~~~~~~~~~~~ * NEWS made clearer. (Martin Pool) * Python3.2+ compatible. (Robert Collins) 0.2.5 ----- IMPROVEMENTS ~~~~~~~~~~~~ * Added ``testresources.FixtureResource`` to wrap ``fixtures.Fixture`` instances. (Robert Collins) * super() is now called from ResourcedTestCase fixing a long standing issue with using it as a mix-in in Python 2.4 and above. (Tim Cole, #771505) * Typo in NEWS fixed. (Thommi Richards) 0.2.4 ----- CHANGES ~~~~~~~ * Relicenced to BSD / Apache2.0 with full agreement of contributors. IMPROVEMENTS ~~~~~~~~~~~~ * Substantially improved documentation in README. (mbp) * Rename TestResource to TestResourceManager leaving TestResource as an alias. Fixes bug #520769. * Test sorting no longer performs N! work on tests that can never benefit from order optimisations (when resources are not shared an arbitrary order is sufficient). Partial fix for bug #522338. * Test sorting now uses a heuristic on each partition to get a sort that is no worse than twice the optimal number of setup/teardown operations and is fast to calculate. Fixes bug #522338 0.2.3 ----- CHANGES ~~~~~~~ IMPROVEMENTS ~~~~~~~~~~~~ * Distribute doc/*.py in the source tarball. * New helper testresources.GenericResource which should remove the need for much boilerplate when using testresources with existing test fixtures. BUG FIXES ~~~~~~~~~ API CHANGES ~~~~~~~~~~~ * New public functions testresources.setUpResources and testresources.tearDownResources for folk that cannot easily use ResourcedTestCase. Fixes bug #504146. INTERNALS ~~~~~~~~~ 0.2.2 ----- BUG FIXES ~~~~~~~~~ * OptimisingTestSuite.addTest was not unpacking OptimisingTestSuite instances. When using testresources.TestLoader to load tests this was generating an unoptimised layout, with each test in its own little suite, and no resource sharing happening. 0.2.1 ------ A small bugfix release for compatibility with newer testtools. INTERNALS ~~~~~~~~~ * One test case wasn't upcalling setUp properly, causing test failures with more recent testtools that check for this mistake. 0.2 --- CHANGES: * testresources needs testtools to run the testresources test suite. You can still use testresources without using testtools. (Jonathan Lange) IMPROVEMENTS: * Many more docstrings. (Jonathan Lange) * Expanded README. (Jonathan Lange) * Expanded TODO. (Jonathan Lange) * Resources can now be reset by overriding TestResource.reset, which for some resources is significantly cheaper. If checking for dirtiness is expensive, isDirty can also be overridden. (James Henstridge, Robert Collins) * Started keeping a NEWS file! (Jonathan Lange) * Resource creation and destruction are traced by calling methods on the TestResult object that tests are being run with. (Robert Collins, #284125) BUG FIXES: * Calling getResource on a dirty resource now triggers a clean and re-make of that resource. (Jonathan Lange) * All resources are dropped when a test with no declared resources is run. (James Henstridge) * A dirty or changed dependency of a resource makes the resource dirty too. (Robert Collins, #324202) API CHANGES: * adsorbSuite is now deprecated in favour of addTest. addTest now flattens standard library TestSuites and distributes custom TestSuite across their member tests. (Jonathan Lange) * ResourcedTestCase.setUpResources and tearDownResources are now instance methods, not static methods. (Jonathan Lange) * All methods on TestResource are now instance methods, and thus tests should use instances of TestResource subclasses, not the classes themselves. (Jonathan Lange) * Now imports from testtools rather than pyunit3k. (Jonathan Lange) * ResourcedTestCase will now look for resources in the 'resources' attribute, rather than the '_resources' attribute. (Jonathan Lange) * ResourcedTestCase.setUpResources and tearDownResources are now instance methods, not static methods. (Jonathan Lange) * SampleTestResource has been removed. (Jonathan Lange) * TestResource.make has had an API change: it must now accept a dependency_resources parameter which is a dictionary listing the dependencies that will be provided to the resource. This parameter is provided so the resource can access its dependencies during setUp, if needed. (Robert Collins) * TestResource subclasses should override 'make' and 'clean' where they previously overrode '_makeResource' and '_cleanResource'. (Jonathan Lange) * TestResource.setResource has been renamed to _setResource. (Jonathan Lange) INTERNALS: * A lot of the tests have been split up into smaller tests. Others have been refactored. (Jonathan Lange) * If calling finishedWith on a TestResource reduces its usage count to zero, then the TestResource considers itself clean, i.e. _dirty is set to True. (Jonathan Lange) * OptimisingTestSuite has been refactored internally so that the way we switch active resources and determine the cost of switching is more obvious. (Jonathan Lange) testresources-0.2.7/lib/0000755000175000017500000000000012076756602016335 5ustar robertcrobertc00000000000000testresources-0.2.7/lib/testresources.egg-info/0000755000175000017500000000000012076756602022741 5ustar robertcrobertc00000000000000testresources-0.2.7/lib/testresources.egg-info/PKG-INFO0000644000175000017500000003072012076756602024040 0ustar robertcrobertc00000000000000Metadata-Version: 1.1 Name: testresources Version: 0.2.7 Summary: Testresources, a pyunit extension for managing expensive test resources Home-page: https://launchpad.net/testresources Author: Testresources developers Author-email: https://launchpad.net/~testresources-developers License: UNKNOWN Description: testresources: extensions to python unittest to allow declarative use of resources by test cases. Copyright (C) 2005-2013 Robert Collins Licensed under either the Apache License, Version 2.0 or the BSD 3-clause license at the users choice. A copy of both licenses are available in the project source as Apache-2.0 and BSD. You may not use this file except in compliance with one of these two licences. Unless required by applicable law or agreed to in writing, software distributed under these licenses is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the license you chose for the specific language governing permissions and limitations under that license. See the COPYING file for full details on the licensing of Testresources. Testresources +++++++++++++ testresources extends unittest with a clean and simple api to provide test optimisation where expensive common resources are needed for test cases - for example sample working trees for VCS systems, reference databases for enterprise applications, or web servers ... let imagination run wild. Dependencies to build/selftest ============================== * Python 2.4+ (or 3.2+) * testtools (http://pypi.python.org/pypi/testtools/) * fixtures (http://pypi.python.org/pypi/fixtures) Dependencies to use testresources ================================= * Python 2.4+ (or 3.2+) How testresources Works ======================= The basic idea of testresources is: * Tests declare the resources they need in a ``resources`` attribute. * When the test is run, the required resource objects are allocated (either newly constructed, or reused), and assigned to attributes of the TestCase. testresources distinguishes a 'resource manager' (a subclass of ``TestResourceManager``) which acts as a kind of factory, and a 'resource' which can be any kind of object returned from the manager class's ``getResource`` method. Resources are either clean or dirty. Being clean means they have same state in all important ways as a newly constructed instance and they can therefore be safely reused. Main Classes ============ testresources.ResourcedTestCase ------------------------------- By extending or mixing-in this class, tests can have necessary resources automatically allocated and disposed or recycled. ResourceTestCase can be used as a base class for tests, and when that is done tests will have their ``resources`` attribute automatically checked for resources by both OptimisingTestSuite and their own setUp() and tearDown() methods. (This allows tests to remain functional without needing this specific TestSuite as a container). Alternatively, you can call setUpResources(self, resources, test_result) and tearDownResources(self, resources, test_result) from your own classes setUp and tearDown and the same behaviour will be activated. To declare the use of a resource, set the ``resources`` attribute to a list of tuples of ``(attribute_name, resource_manager)``. During setUp, for each declared requirement, the test gains an attribute pointing to an allocated resource, which is the result of calling ``resource_manager.getResource()``. ``finishedWith`` will be called on each resource during tearDown(). For example:: class TestLog(testresources.ResourcedTestCase): resources = [('branch', BzrPopulatedBranch())] def test_log(self): show_log(self.branch, ...) testresources.TestResourceManager --------------------------------- A TestResourceManager is an object that tests can use to create resources. It can be overridden to manage different types of resources. Normally test code doesn't need to call any methods on it, as this will be arranged by the testresources machinery. When implementing a new ``TestResourceManager`` subclass you should consider overriding these methods: ``make`` Must be overridden in every concrete subclass. Returns a new instance of the resource object (the actual resource, not the TestResourceManager). Doesn't need to worry about reuse, which is taken care of separately. This method is only called when a new resource is definitely needed. ``make`` is called by ``getResource``; you should not normally need to override the latter. ``clean`` Cleans up an existing resource instance, eg by deleting a directory or closing a network connection. By default this does nothing, which may be appropriate for resources that are automatically garbage collected. ``_reset`` Reset a no-longer-used dirty resource to a clean state. By default this just discards it and creates a new one, but for some resources there may be a faster way to reset them. ``isDirty`` Check whether an existing resource is dirty. By default this just reports whether ``TestResourceManager.dirtied`` has been called or any of the dependency resources are dirty. For instance:: class TemporaryDirectoryResource(TestResourceManager): def clean(self, resource): shutil.rmtree(resource) def make(self): return tempfile.mkdtemp() def isDirty(self, resource): # Can't detect when the directory is written to, so assume it # can never be reused. We could list the directory, but that might # not catch it being open as a cwd etc. return True The ``resources`` list on the TestResourceManager object is used to declare dependencies. For instance, a DataBaseResource that needs a TemporaryDirectory might be declared with a resources list:: class DataBaseResource(TestResourceManager): resources = [("scratchdir", TemporaryDirectoryResource())] Most importantly, two getResources to the same TestResourceManager with no finishedWith call in the middle, will return the same object as long as it is not dirty. When a Test has a dependency and that dependency successfully completes but returns None, the framework does *not* consider this an error: be sure to always return a valid resource, or raise an error. Error handling hasn't been heavily exercised, but any bugs in this area will be promptly dealt with. A sample TestResourceManager can be found in the doc/ folder. See pydoc testresources.TestResourceManager for details. testresources.GenericResource ----------------------------- Glue to adapt testresources to an existing resource-like class. testresources.FixtureResource ----------------------------- Glue to adapt testresources to the simpler fixtures.Fixture API. Long term testresources is likely to consolidate on that simpler API as the recommended method of writing resources. testresources.OptimisingTestSuite --------------------------------- This TestSuite will introspect all the test cases it holds directly and if they declare needed resources, will run the tests in an order that attempts to minimise the number of setup and tear downs required. It attempts to achieve this by callling getResource() and finishedWith() around the sequence of tests that use a specific resource. Tests are added to an OptimisingTestSuite as normal. Any standard library TestSuite objects will be flattened, while any custom TestSuite subclasses will be distributed across their member tests. This means that any custom logic in test suites should be preserved, at the price of some level of optimisation. Because the test suite does the optimisation, you can control the amount of optimising that takes place by adding more or fewer tests to a single OptimisingTestSuite. You could add everything to a single OptimisingTestSuite, getting global optimisation or you could use several smaller OptimisingTestSuites. testresources.TestLoader ------------------------ This is a trivial TestLoader that creates OptimisingTestSuites by default. unittest.TestResult ------------------- testresources will log activity about resource creation and destruction to the result object tests are run with. 6 extension methods are looked for: ``startCleanResource``, ``stopCleanResource``, ``startMakeResource``, ``stopMakeResource``, ``startResetResource`` and finally ``stopResetResource``. ``testresources.tests.ResultWithResourceExtensions`` is an example of a ``TestResult`` with these methods present. Controlling Resource Reuse ========================== When or how do I mark the resource dirtied? The simplest approach is to have ``TestResourceManager.make`` call ``self.dirtied``: the resource is always immediately dirty and will never be reused without first being reset. This is appropriate when the underlying resource is cheap to reset or recreate, or when it's hard to detect whether it's been dirtied or to trap operations that change it. Alternatively, override ``TestResourceManager.isDirty`` and inspect the resource to see if it is safe to reuse. Finally, you can arrange for the returned resource to always call back to ``TestResourceManager.dirtied`` on the first operation that mutates it. FAQ === * Can I dynamically request resources inside a test method? Generally, no, you shouldn't do this. The idea is that the resources are declared statically, so that testresources can "smooth" resource usage across several tests. But, you may be able to find some object that is statically declared and reusable to act as the resource, which can then provide methods to generate sub-elements of itself during a test. * If the resource is held inside the TestResourceManager object, and the TestResourceManager is typically constructed inline in the test case ``resources`` attribute, how can they be shared across different test classes? Good question. I guess you should arrange for a single instance to be held in an appropriate module scope, then referenced by the test classes that want to share it. Keywords: unittest testing fixtures Platform: UNKNOWN Classifier: Development Status :: 6 - Mature Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: BSD License Classifier: License :: OSI Approved :: Apache Software License Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 3 Classifier: Topic :: Software Development :: Quality Assurance Classifier: Topic :: Software Development :: Testing testresources-0.2.7/lib/testresources.egg-info/top_level.txt0000644000175000017500000000001612076756602025470 0ustar robertcrobertc00000000000000testresources testresources-0.2.7/lib/testresources.egg-info/SOURCES.txt0000644000175000017500000000105412076756602024625 0ustar robertcrobertc00000000000000MANIFEST.in NEWS README setup.py doc/example.py lib/testresources/__init__.py lib/testresources.egg-info/PKG-INFO lib/testresources.egg-info/SOURCES.txt lib/testresources.egg-info/dependency_links.txt lib/testresources.egg-info/top_level.txt lib/testresources/tests/TestUtil.py lib/testresources/tests/__init__.py lib/testresources/tests/test_optimising_test_suite.py lib/testresources/tests/test_resource_graph.py lib/testresources/tests/test_resourced_test_case.py lib/testresources/tests/test_test_loader.py lib/testresources/tests/test_test_resource.pytestresources-0.2.7/lib/testresources.egg-info/dependency_links.txt0000644000175000017500000000000112076756602027007 0ustar robertcrobertc00000000000000 testresources-0.2.7/lib/testresources/0000755000175000017500000000000012076756602021247 5ustar robertcrobertc00000000000000testresources-0.2.7/lib/testresources/tests/0000755000175000017500000000000012076756602022411 5ustar robertcrobertc00000000000000testresources-0.2.7/lib/testresources/tests/test_resource_graph.py0000644000175000017500000001143212076614173027027 0ustar robertcrobertc00000000000000# # testresources: extensions to python unittest to allow declaritive use # of resources by test cases. # # Copyright (c) 2005-2010 Testresources Contributors # # Licensed under either the Apache License, Version 2.0 or the BSD 3-clause # license at the users choice. A copy of both licenses are available in the # project source as Apache-2.0 and BSD. You may not use this file except in # compliance with one of these two licences. # # Unless required by applicable law or agreed to in writing, software distributed # under these licenses is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR # CONDITIONS OF ANY KIND, either express or implied. See the license you chose # for the specific language governing permissions and limitations under that # license. # """Test _resource_graph(resource_sets).""" import testtools import testresources from testresources import split_by_resources, _resource_graph from testresources.tests import ResultWithResourceExtensions import unittest def test_suite(): from testresources.tests import TestUtil loader = TestUtil.TestLoader() result = loader.loadTestsFromName(__name__) return result class TestResourceGraph(testtools.TestCase): def test_empty(self): no_resources = frozenset() resource_sets = [no_resources] self.assertEqual({no_resources:set([])}, _resource_graph(resource_sets)) def test_discrete(self): resset1 = frozenset([testresources.TestResourceManager()]) resset2 = frozenset([testresources.TestResourceManager()]) resource_sets = [resset1, resset2] result = _resource_graph(resource_sets) self.assertEqual({resset1:set([]), resset2:set([])}, result) def test_overlapping(self): res1 = testresources.TestResourceManager() res2 = testresources.TestResourceManager() resset1 = frozenset([res1]) resset2 = frozenset([res2]) resset3 = frozenset([res1, res2]) resource_sets = [resset1, resset2, resset3] result = _resource_graph(resource_sets) self.assertEqual( {resset1:set([resset3]), resset2:set([resset3]), resset3:set([resset1, resset2])}, result) class TestDigraphToGraph(testtools.TestCase): def test_wikipedia_example(self): """Converting a digraph mirrors it in the XZ axis (matrix view). See http://en.wikipedia.org/wiki/Travelling_salesman_problem \ #Solving_by_conversion_to_Symmetric_TSP """ # A B C # A 1 2 # B 6 3 # C 5 4 A = "A" Ap = "A'" B = "B" Bp = "B'" C = "C" Cp = "C'" digraph = {A:{ B:1, C:2}, B:{A:6, C:3}, C:{A:5, B:4 }} # and the output # A B C A' B' C' # A 0 6 5 # B 1 0 4 # C 2 3 0 # A' 0 1 2 # B' 6 0 3 # C' 5 4 0 expected = { A :{ Ap:0, Bp:6, Cp:5}, B :{ Ap:1, Bp:0, Cp:4}, C :{ Ap:2, Bp:3, Cp:0}, Ap:{A:0, B:1, C:2 }, Bp:{A:6, B:0, C:3 }, Cp:{A:5, B:4, C:0 }} self.assertEqual(expected, testresources._digraph_to_graph(digraph, {A:Ap, B:Bp, C:Cp})) class TestKruskalsMST(testtools.TestCase): def test_wikipedia_example(self): """Performing KruskalsMST on a graph returns a spanning tree. See http://en.wikipedia.org/wiki/Kruskal%27s_algorithm. """ A = "A" B = "B" C = "C" D = "D" E = "E" F = "F" G = "G" graph = { A:{ B:7, D:5}, B:{A:7, C:8, D:9, E:7}, C:{ B:8, E:5}, D:{A:5, B:9, E:15, F:6}, E:{ B:7, C:5, D:15, F:8, G:9}, F:{ D:6, E:8, G:11}, G:{ E:9, F:11}} expected = { A:{ B:7, D:5}, B:{A:7, E:7}, C:{ E:5}, D:{A:5, F:6}, E:{ B:7, C:5, G:9}, F:{ D:6}, G:{ E:9}} result = testresources._kruskals_graph_MST(graph) e_weight = sum(sum(row.values()) for row in expected.values()) r_weight = sum(sum(row.values()) for row in result.values()) self.assertEqual(e_weight, r_weight) self.assertEqual(expected, testresources._kruskals_graph_MST(graph)) testresources-0.2.7/lib/testresources/tests/test_optimising_test_suite.py0000644000175000017500000006733711560345664030473 0ustar robertcrobertc00000000000000# testresources: extensions to python unittest to allow declaritive use # of resources by test cases. # # Copyright (c) 2005-2010 Testresources Contributors # # Licensed under either the Apache License, Version 2.0 or the BSD 3-clause # license at the users choice. A copy of both licenses are available in the # project source as Apache-2.0 and BSD. You may not use this file except in # compliance with one of these two licences. # # Unless required by applicable law or agreed to in writing, software distributed # under these licenses is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR # CONDITIONS OF ANY KIND, either express or implied. See the license you chose # for the specific language governing permissions and limitations under that # license. # import testtools import random import testresources from testresources import split_by_resources from testresources.tests import ResultWithResourceExtensions import unittest def test_suite(): from testresources.tests import TestUtil loader = TestUtil.TestLoader() result = loader.loadTestsFromName(__name__) return result class CustomSuite(unittest.TestSuite): """Custom TestSuite that's comparable using == and !=.""" def __eq__(self, other): return (self.__class__ == other.__class__ and self._tests == other._tests) def __ne__(self, other): return not self.__eq__(other) class MakeCounter(testresources.TestResource): """Test resource that counts makes and cleans.""" def __init__(self): testresources.TestResource.__init__(self) self.cleans = 0 self.makes = 0 self.calls = [] def clean(self, resource): self.cleans += 1 self.calls.append(('clean', resource)) def make(self, dependency_resources): self.makes += 1 resource = "boo %d" % self.makes self.calls.append(('make', resource)) return resource class TestOptimisingTestSuite(testtools.TestCase): def makeTestCase(self, test_running_hook=None): """Make a normal TestCase.""" class TestCaseForTesting(unittest.TestCase): def runTest(self): if test_running_hook: test_running_hook(self) return TestCaseForTesting('runTest') def makeResourcedTestCase(self, resource_manager, test_running_hook): """Make a ResourcedTestCase.""" class ResourcedTestCaseForTesting(testresources.ResourcedTestCase): def runTest(self): test_running_hook(self) test_case = ResourcedTestCaseForTesting('runTest') test_case.resources = [('_default', resource_manager)] return test_case def setUp(self): super(TestOptimisingTestSuite, self).setUp() self.optimising_suite = testresources.OptimisingTestSuite() def testAddTest(self): # Adding a single test case is the same as adding one using the # standard addTest. case = self.makeTestCase() self.optimising_suite.addTest(case) self.assertEqual([case], self.optimising_suite._tests) def testAddTestSuite(self): # Adding a standard test suite is the same as adding all the tests in # that suite. case = self.makeTestCase() suite = unittest.TestSuite([case]) self.optimising_suite.addTest(suite) self.assertEqual([case], self.optimising_suite._tests) def testAddTestOptimisingTestSuite(self): # when adding an optimising test suite, it should be unpacked. case = self.makeTestCase() suite1 = testresources.OptimisingTestSuite([case]) suite2 = testresources.OptimisingTestSuite([case]) self.optimising_suite.addTest(suite1) self.optimising_suite.addTest(suite2) self.assertEqual([case, case], self.optimising_suite._tests) def testAddFlattensStandardSuiteStructure(self): # addTest will get rid of all unittest.TestSuite structure when adding # a test, no matter how much nesting is going on. case1 = self.makeTestCase() case2 = self.makeTestCase() case3 = self.makeTestCase() suite = unittest.TestSuite( [unittest.TestSuite([case1, unittest.TestSuite([case2])]), case3]) self.optimising_suite.addTest(suite) self.assertEqual([case1, case2, case3], self.optimising_suite._tests) def testAddDistributesNonStandardSuiteStructure(self): # addTest distributes all non-standard TestSuites across their # members. case1 = self.makeTestCase() case2 = self.makeTestCase() inner_suite = unittest.TestSuite([case2]) suite = CustomSuite([case1, inner_suite]) self.optimising_suite.addTest(suite) self.assertEqual( [CustomSuite([case1]), CustomSuite([inner_suite])], self.optimising_suite._tests) def testAddPullsNonStandardSuitesUp(self): # addTest flattens standard TestSuites, even those that contain custom # suites. When it reaches the custom suites, it distributes them # across their members. case1 = self.makeTestCase() case2 = self.makeTestCase() inner_suite = CustomSuite([case1, case2]) self.optimising_suite.addTest( unittest.TestSuite([unittest.TestSuite([inner_suite])])) self.assertEqual( [CustomSuite([case1]), CustomSuite([case2])], self.optimising_suite._tests) def testSingleCaseResourceAcquisition(self): sample_resource = MakeCounter() def getResourceCount(test): self.assertEqual(sample_resource._uses, 2) case = self.makeResourcedTestCase(sample_resource, getResourceCount) self.optimising_suite.addTest(case) result = unittest.TestResult() self.optimising_suite.run(result) self.assertEqual(result.testsRun, 1) self.assertEqual(result.wasSuccessful(), True) self.assertEqual(sample_resource._uses, 0) def testResourceReuse(self): make_counter = MakeCounter() def getResourceCount(test): self.assertEqual(make_counter._uses, 2) case = self.makeResourcedTestCase(make_counter, getResourceCount) case2 = self.makeResourcedTestCase(make_counter, getResourceCount) self.optimising_suite.addTest(case) self.optimising_suite.addTest(case2) result = unittest.TestResult() self.optimising_suite.run(result) self.assertEqual(result.testsRun, 2) self.assertEqual(result.wasSuccessful(), True) self.assertEqual(make_counter._uses, 0) self.assertEqual(make_counter.makes, 1) self.assertEqual(make_counter.cleans, 1) def testResultPassedToResources(self): resource_manager = MakeCounter() test_case = self.makeTestCase(lambda x:None) test_case.resources = [('_default', resource_manager)] self.optimising_suite.addTest(test_case) result = ResultWithResourceExtensions() self.optimising_suite.run(result) # We should see the resource made and cleaned once. As its not a # resource aware test, it won't make any calls itself. self.assertEqual(4, len(result._calls)) def testOptimisedRunNonResourcedTestCase(self): case = self.makeTestCase() self.optimising_suite.addTest(case) result = unittest.TestResult() self.optimising_suite.run(result) self.assertEqual(result.testsRun, 1) self.assertEqual(result.wasSuccessful(), True) def testSortTestsCalled(self): # OptimisingTestSuite.run() calls sortTests on the suite. class MockOptimisingTestSuite(testresources.OptimisingTestSuite): def sortTests(self): self.sorted = True suite = MockOptimisingTestSuite() suite.sorted = False suite.run(None) self.assertEqual(suite.sorted, True) def testResourcesDroppedForNonResourcedTestCase(self): sample_resource = MakeCounter() def resourced_case_hook(test): self.assertTrue(sample_resource._uses > 0) self.optimising_suite.addTest(self.makeResourcedTestCase( sample_resource, resourced_case_hook)) def normal_case_hook(test): # The resource should not be acquired when the normal test # runs. self.assertEqual(sample_resource._uses, 0) self.optimising_suite.addTest(self.makeTestCase(normal_case_hook)) result = unittest.TestResult() self.optimising_suite.run(result) self.assertEqual(result.testsRun, 2) self.assertEqual([], result.failures) self.assertEqual([], result.errors) self.assertEqual(result.wasSuccessful(), True) def testDirtiedResourceNotRecreated(self): make_counter = MakeCounter() def dirtyResource(test): make_counter.dirtied(test._default) case = self.makeResourcedTestCase(make_counter, dirtyResource) self.optimising_suite.addTest(case) result = unittest.TestResult() self.optimising_suite.run(result) self.assertEqual(result.testsRun, 1) self.assertEqual(result.wasSuccessful(), True) # The resource should only have been made once. self.assertEqual(make_counter.makes, 1) def testDirtiedResourceCleanedUp(self): make_counter = MakeCounter() def testOne(test): make_counter.calls.append('test one') make_counter.dirtied(test._default) def testTwo(test): make_counter.calls.append('test two') case1 = self.makeResourcedTestCase(make_counter, testOne) case2 = self.makeResourcedTestCase(make_counter, testTwo) self.optimising_suite.addTest(case1) self.optimising_suite.addTest(case2) result = unittest.TestResult() self.optimising_suite.run(result) self.assertEqual(result.testsRun, 2) self.assertEqual(result.wasSuccessful(), True) # Two resources should have been created and cleaned up self.assertEqual(make_counter.calls, [('make', 'boo 1'), 'test one', ('clean', 'boo 1'), ('make', 'boo 2'), 'test two', ('clean', 'boo 2')]) class TestSplitByResources(testtools.TestCase): """Tests for split_by_resources.""" def makeTestCase(self): return unittest.TestCase('run') def makeResourcedTestCase(self, has_resource=True): case = testresources.ResourcedTestCase('run') if has_resource: case.resources = [('resource', testresources.TestResource())] return case def testNoTests(self): self.assertEqual({frozenset(): []}, split_by_resources([])) def testJustNormalCases(self): normal_case = self.makeTestCase() resource_set_tests = split_by_resources([normal_case]) self.assertEqual({frozenset(): [normal_case]}, resource_set_tests) def testJustResourcedCases(self): resourced_case = self.makeResourcedTestCase() resource = resourced_case.resources[0][1] resource_set_tests = split_by_resources([resourced_case]) self.assertEqual({frozenset(): [], frozenset([resource]): [resourced_case]}, resource_set_tests) def testMultipleResources(self): resource1 = testresources.TestResource() resource2 = testresources.TestResource() resourced_case = self.makeResourcedTestCase(has_resource=False) resourced_case.resources = [('resource1', resource1), ('resource2', resource2)] resource_set_tests = split_by_resources([resourced_case]) self.assertEqual({frozenset(): [], frozenset([resource1, resource2]): [resourced_case]}, resource_set_tests) def testDependentResources(self): resource1 = testresources.TestResource() resource2 = testresources.TestResource() resource1.resources = [('foo', resource2)] resourced_case = self.makeResourcedTestCase(has_resource=False) resourced_case.resources = [('resource1', resource1)] resource_set_tests = split_by_resources([resourced_case]) self.assertEqual({frozenset(): [], frozenset([resource1, resource2]): [resourced_case]}, resource_set_tests) def testResourcedCaseWithNoResources(self): resourced_case = self.makeResourcedTestCase(has_resource=False) resource_set_tests = split_by_resources([resourced_case]) self.assertEqual({frozenset(): [resourced_case]}, resource_set_tests) def testMixThemUp(self): normal_cases = [self.makeTestCase() for i in range(3)] normal_cases.extend([ self.makeResourcedTestCase(has_resource=False) for i in range(3)]) resourced_cases = [self.makeResourcedTestCase() for i in range(3)] all_cases = normal_cases + resourced_cases # XXX: Maybe I shouldn't be using random here. random.shuffle(all_cases) resource_set_tests = split_by_resources(all_cases) self.assertEqual(set(normal_cases), set(resource_set_tests[frozenset()])) for case in resourced_cases: resource = case.resources[0][1] self.assertEqual([case], resource_set_tests[frozenset([resource])]) class TestCostOfSwitching(testtools.TestCase): """Tests for cost_of_switching.""" def setUp(self): super(TestCostOfSwitching, self).setUp() self.suite = testresources.OptimisingTestSuite() def makeResource(self, setUpCost=1, tearDownCost=1): resource = testresources.TestResource() resource.setUpCost = setUpCost resource.tearDownCost = tearDownCost return resource def testNoResources(self): # The cost of switching from no resources to no resources is 0. self.assertEqual(0, self.suite.cost_of_switching(set(), set())) def testSameResources(self): # The cost of switching to the same set of resources is also 0. a = self.makeResource() b = self.makeResource() self.assertEqual(0, self.suite.cost_of_switching(set([a]), set([a]))) self.assertEqual( 0, self.suite.cost_of_switching(set([a, b]), set([a, b]))) # XXX: The next few tests demonstrate the current behaviour of the system. # We'll change them later. def testNewResources(self): a = self.makeResource() b = self.makeResource() self.assertEqual(1, self.suite.cost_of_switching(set(), set([a]))) self.assertEqual( 1, self.suite.cost_of_switching(set([a]), set([a, b]))) self.assertEqual(2, self.suite.cost_of_switching(set(), set([a, b]))) def testOldResources(self): a = self.makeResource() b = self.makeResource() self.assertEqual(1, self.suite.cost_of_switching(set([a]), set())) self.assertEqual( 1, self.suite.cost_of_switching(set([a, b]), set([a]))) self.assertEqual(2, self.suite.cost_of_switching(set([a, b]), set())) def testCombo(self): a = self.makeResource() b = self.makeResource() c = self.makeResource() self.assertEqual(2, self.suite.cost_of_switching(set([a]), set([b]))) self.assertEqual( 2, self.suite.cost_of_switching(set([a, c]), set([b, c]))) class TestCostGraph(testtools.TestCase): """Tests for calculating the cost graph of resourced test cases.""" def makeResource(self, setUpCost=1, tearDownCost=1): resource = testresources.TestResource() resource.setUpCost = setUpCost resource.tearDownCost = tearDownCost return resource def testEmptyGraph(self): suite = testresources.OptimisingTestSuite() graph = suite._getGraph([]) self.assertEqual({}, graph) def testSingletonGraph(self): resource = self.makeResource() suite = testresources.OptimisingTestSuite() graph = suite._getGraph([frozenset()]) self.assertEqual({frozenset(): {}}, graph) def testTwoCasesInGraph(self): res1 = self.makeResource() res2 = self.makeResource() set1 = frozenset([res1, res2]) set2 = frozenset([res2]) no_resources = frozenset() suite = testresources.OptimisingTestSuite() graph = suite._getGraph([no_resources, set1, set2]) self.assertEqual({no_resources: {set1: 2, set2: 1}, set1: {no_resources: 2, set2: 1}, set2: {no_resources: 1, set1: 1 }}, graph) class TestGraphStuff(testtools.TestCase): def setUp(self): super(TestGraphStuff, self).setUp() class MockTest(unittest.TestCase): def __repr__(self): """The representation is the tests name. This makes it easier to debug sorting failures. """ return self.id().split('.')[-1] def test_one(self): pass def test_two(self): pass def test_three(self): pass def test_four(self): pass self.case1 = MockTest("test_one") self.case2 = MockTest("test_two") self.case3 = MockTest("test_three") self.case4 = MockTest("test_four") self.cases = [] self.cases.append(self.case1) self.cases.append(self.case2) self.cases.append(self.case3) self.cases.append(self.case4) def sortTests(self, tests): suite = testresources.OptimisingTestSuite() suite.addTests(tests) suite.sortTests() return suite._tests def _permute_four(self, cases): case1, case2, case3, case4 = cases permutations = [] permutations.append([case1, case2, case3, case4]) permutations.append([case1, case2, case4, case3]) permutations.append([case1, case3, case2, case4]) permutations.append([case1, case3, case4, case2]) permutations.append([case1, case4, case2, case3]) permutations.append([case1, case4, case3, case2]) permutations.append([case2, case1, case3, case4]) permutations.append([case2, case1, case4, case3]) permutations.append([case2, case3, case1, case4]) permutations.append([case2, case3, case4, case1]) permutations.append([case2, case4, case1, case3]) permutations.append([case2, case4, case3, case1]) permutations.append([case3, case2, case1, case4]) permutations.append([case3, case2, case4, case1]) permutations.append([case3, case1, case2, case4]) permutations.append([case3, case1, case4, case2]) permutations.append([case3, case4, case2, case1]) permutations.append([case3, case4, case1, case2]) permutations.append([case4, case2, case3, case1]) permutations.append([case4, case2, case1, case3]) permutations.append([case4, case3, case2, case1]) permutations.append([case4, case3, case1, case2]) permutations.append([case4, case1, case2, case3]) permutations.append([case4, case1, case3, case2]) return permutations def testBasicSortTests(self): # Test every permutation of inputs, with legacy tests. # Cannot use equal costs because of the use of # a 2*optimal heuristic for sorting: with equal # costs the wrong sort order is < twice the optimal # weight, and thus can be selected. resource_one = testresources.TestResource() resource_two = testresources.TestResource() resource_two.setUpCost = 5 resource_two.tearDownCost = 5 resource_three = testresources.TestResource() self.case1.resources = [ ("_one", resource_one), ("_two", resource_two)] self.case2.resources = [ ("_two", resource_two), ("_three", resource_three)] self.case3.resources = [("_three", resource_three)] # acceptable sorted orders are: # 1, 2, 3, 4 # 3, 2, 1, 4 for permutation in self._permute_four(self.cases): self.assertIn( self.sortTests(permutation), [ [self.case1, self.case2, self.case3, self.case4], [self.case3, self.case2, self.case1, self.case4]]) def testGlobalMinimum(self): # When a local minimum leads to a global non-minum, the global # non-minimum is still reached. We construct this by having a resource # that appears very cheap (it has a low setup cost) but is very # expensive to tear down. Then we have it be used twice: the global # minimum depends on only tearing it down once. To prevent it # accidentally being chosen twice, we make one use of it be # on its own, and another with a resource to boost its cost, # finally we put a resource which is more expensive to setup # than the expensive teardown is to teardown, but less expensive # than it + the small booster to setup. # valid results are - the expensive setup, then both expensive # teardowns, and the legacy fourth, or # both expensive teardowns and then the expensive setup (and the legacy # fourth) # case1 has expensive setup (one) # case2 has expensive teardown (two) # case3 has expensive teardown + boost (three) resource_one = testresources.TestResource() resource_one.setUpCost = 20 resource_two = testresources.TestResource() resource_two.tearDownCost = 50 resource_three = testresources.TestResource() resource_three.setUpCost = 72 # node costs: # ->1 = r1.up = 20 # ->2 = r2.up = 1 # ->3 = r2.up + r3.up = 122 # 1->2 = r1.down + r2.up = 2 # 1->3 = r1.down + r2.up + r3.up = 93 # 2->1 = r2.down + r1.up = 70 # 2->3 = r3.up = 72 # 3->1 = r1.up + r2.down + r3.down= 71 # 3->2 = r3.down = 1 # 1-> = r1.down = 1 # 2-> = r2.down = 50 # 3-> = r3.down + r3.down = 51 # naive path = 2, 1, 3 = 1 + 70 + 93 + 51 = 215 # better = 2, 3, 1 = 1 + 72 + 71 + 1 = 145 acceptable_orders = [ [self.case1, self.case2, self.case3, self.case4], [self.case1, self.case3, self.case2, self.case4], [self.case2, self.case3, self.case1, self.case4], [self.case3, self.case2, self.case1, self.case4], ] self.case1.resources = [ ("_one", resource_one)] self.case2.resources = [ ("_two", resource_two)] self.case3.resources = [("_two", resource_two), ("_three", resource_three)] for permutation in self._permute_four(self.cases): self.assertIn(self.sortTests(permutation), acceptable_orders) def testSortIsStableWithinGroups(self): """Tests with the same resources maintain their relative order.""" resource_one = testresources.TestResource() resource_two = testresources.TestResource() self.case1.resources = [("_one", resource_one)] self.case2.resources = [("_one", resource_one)] self.case3.resources = [("_one", resource_one), ("_two", resource_two)] self.case4.resources = [("_one", resource_one), ("_two", resource_two)] for permutation in self._permute_four(self.cases): sorted = self.sortTests(permutation) self.assertEqual( permutation.index(self.case1) < permutation.index(self.case2), sorted.index(self.case1) < sorted.index(self.case2)) self.assertEqual( permutation.index(self.case3) < permutation.index(self.case4), sorted.index(self.case3) < sorted.index(self.case4)) def testSortingTwelveIndependentIsFast(self): # Given twelve independent resource sets, my patience is not exhausted. managers = [] for pos in range(12): managers.append(testresources.TestResourceManager()) # Add more sample tests cases = [self.case1, self.case2, self.case3, self.case4] for pos in range(5,13): cases.append( testtools.clone_test_with_new_id(cases[0], 'case%d' % pos)) # We care that this is fast in this test, so we don't need to have # overlapping resource usage for case, manager in zip(cases, managers): case.resources = [('_resource', manager)] # Any sort is ok, as long as its the right length :) result = self.sortTests(cases) self.assertEqual(12, len(result)) def testSortingTwelveOverlappingIsFast(self): # Given twelve connected resource sets, my patience is not exhausted. managers = [] for pos in range(12): managers.append(testresources.TestResourceManager()) # Add more sample tests cases = [self.case1, self.case2, self.case3, self.case4] for pos in range(5,13): cases.append( testtools.clone_test_with_new_id(cases[0], 'case%d' % pos)) tempdir = testresources.TestResourceManager() # give all tests a tempdir, enough to provoke a single partition in # the current code. for case, manager in zip(cases, managers): case.resources = [('_resource', manager), ('tempdir', tempdir)] # Any sort is ok, as long as its the right length :) result = self.sortTests(cases) self.assertEqual(12, len(result)) def testSortConsidersDependencies(self): """Tests with different dependencies are sorted together.""" # We test this by having two resources (one and two) that share a very # expensive dependency (dep). So one and two have to sort together. By # using a cheap resource directly from several tests we can force the # optimise to choose between keeping the cheap resource together or # keeping the expensive dependency together. # Test1, res_one, res_common_one # Test2, res_two, res_common_two # Test3, res_common_one, res_common_two # In a dependency naive sort, we will have test3 between test1 and # test2 always. In a dependency aware sort, test1 and two will # always group. resource_one = testresources.TestResource() resource_two = testresources.TestResource() resource_one_common = testresources.TestResource() # make it cheaper to keep a _common resource than to switch both # resources (when dependencies are ignored) resource_one_common.setUpCost = 2 resource_one_common.tearDownCost = 2 resource_two_common = testresources.TestResource() resource_two_common.setUpCost = 2 resource_two_common.tearDownCost = 2 dep = testresources.TestResource() dep.setUpCost = 20 dep.tearDownCost = 20 resource_one.resources.append(("dep1", dep)) resource_two.resources.append(("dep2", dep)) self.case1.resources = [("withdep", resource_one), ("common", resource_one_common)] self.case2.resources = [("withdep", resource_two), ("common", resource_two_common)] self.case3.resources = [("_one", resource_one_common), ("_two", resource_two_common)] self.case4.resources = [] acceptable_orders = [ [self.case1, self.case2, self.case3, self.case4], [self.case2, self.case1, self.case3, self.case4], [self.case3, self.case1, self.case2, self.case4], [self.case3, self.case2, self.case1, self.case4], ] for permutation in self._permute_four(self.cases): self.assertIn(self.sortTests(permutation), acceptable_orders) testresources-0.2.7/lib/testresources/tests/test_test_resource.py0000644000175000017500000005031612076756016026714 0ustar robertcrobertc00000000000000# testresources: extensions to python unittest to allow declaritive use # of resources by test cases. # # Copyright (c) 2005-2010 Testresources Contributors # # Licensed under either the Apache License, Version 2.0 or the BSD 3-clause # license at the users choice. A copy of both licenses are available in the # project source as Apache-2.0 and BSD. You may not use this file except in # compliance with one of these two licences. # # Unless required by applicable law or agreed to in writing, software distributed # under these licenses is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR # CONDITIONS OF ANY KIND, either express or implied. See the license you chose # for the specific language governing permissions and limitations under that # license. # from fixtures.tests.helpers import LoggingFixture import testtools import testresources from testresources.tests import ( ResultWithResourceExtensions, ResultWithoutResourceExtensions, ) def test_suite(): loader = testresources.tests.TestUtil.TestLoader() result = loader.loadTestsFromName(__name__) return result class MockResourceInstance(object): def __init__(self, name): self._name = name def __eq__(self, other): return self.__dict__ == other.__dict__ def __cmp__(self, other): return cmp(self.__dict__, other.__dict__) def __repr__(self): return self._name class MockResource(testresources.TestResourceManager): """Mock resource that logs the number of make and clean calls.""" def __init__(self): super(MockResource, self).__init__() self.makes = 0 self.cleans = 0 def clean(self, resource): self.cleans += 1 def make(self, dependency_resources): self.makes += 1 return MockResourceInstance("Boo!") class MockResettableResource(MockResource): """Mock resource that logs the number of reset calls too.""" def __init__(self): super(MockResettableResource, self).__init__() self.resets = 0 def _reset(self, resource, dependency_resources): self.resets += 1 resource._name += "!" self._dirty = False return resource class TestTestResource(testtools.TestCase): def testUnimplementedGetResource(self): # By default, TestResource raises NotImplementedError on getResource # because make is not defined initially. resource_manager = testresources.TestResource() self.assertRaises(NotImplementedError, resource_manager.getResource) def testInitiallyNotDirty(self): resource_manager = testresources.TestResource() self.assertEqual(False, resource_manager._dirty) def testInitiallyUnused(self): resource_manager = testresources.TestResource() self.assertEqual(0, resource_manager._uses) def testInitiallyNoCurrentResource(self): resource_manager = testresources.TestResource() self.assertEqual(None, resource_manager._currentResource) def testneededResourcesDefault(self): # Calling neededResources on a default TestResource returns the # resource. resource = testresources.TestResource() self.assertEqual([resource], resource.neededResources()) def testneededResourcesDependenciesFirst(self): # Calling neededResources on a TestResource with dependencies puts the # dependencies first. resource = testresources.TestResource() dep1 = testresources.TestResource() dep2 = testresources.TestResource() resource.resources.append(("dep1", dep1)) resource.resources.append(("dep2", dep2)) self.assertEqual([dep1, dep2, resource], resource.neededResources()) def testneededResourcesClosure(self): # Calling neededResources on a TestResource with dependencies includes # the needed resources of the needed resources. resource = testresources.TestResource() dep1 = testresources.TestResource() dep2 = testresources.TestResource() resource.resources.append(("dep1", dep1)) dep1.resources.append(("dep2", dep2)) self.assertEqual([dep2, dep1, resource], resource.neededResources()) def testDefaultCosts(self): # The base TestResource costs 1 to set up and to tear down. resource_manager = testresources.TestResource() self.assertEqual(resource_manager.setUpCost, 1) self.assertEqual(resource_manager.tearDownCost, 1) def testGetResourceReturnsMakeResource(self): resource_manager = MockResource() resource = resource_manager.getResource() self.assertEqual(resource_manager.make({}), resource) def testGetResourceIncrementsUses(self): resource_manager = MockResource() resource_manager.getResource() self.assertEqual(1, resource_manager._uses) resource_manager.getResource() self.assertEqual(2, resource_manager._uses) def testGetResourceDoesntDirty(self): resource_manager = MockResource() resource_manager.getResource() self.assertEqual(resource_manager._dirty, False) def testGetResourceSetsCurrentResource(self): resource_manager = MockResource() resource = resource_manager.getResource() self.assertIs(resource_manager._currentResource, resource) def testGetResourceTwiceReturnsIdenticalResource(self): resource_manager = MockResource() resource1 = resource_manager.getResource() resource2 = resource_manager.getResource() self.assertIs(resource1, resource2) def testGetResourceCallsMakeResource(self): resource_manager = MockResource() resource_manager.getResource() self.assertEqual(1, resource_manager.makes) def testIsDirty(self): resource_manager = MockResource() r = resource_manager.getResource() resource_manager.dirtied(r) self.assertTrue(resource_manager.isDirty()) resource_manager.finishedWith(r) def testIsDirtyIsTrueIfDependenciesChanged(self): resource_manager = MockResource() dep1 = MockResource() dep2 = MockResource() dep3 = MockResource() resource_manager.resources.append(("dep1", dep1)) resource_manager.resources.append(("dep2", dep2)) resource_manager.resources.append(("dep3", dep3)) r = resource_manager.getResource() dep2.dirtied(r.dep2) r2 =dep2.getResource() self.assertTrue(resource_manager.isDirty()) resource_manager.finishedWith(r) dep2.finishedWith(r2) def testIsDirtyIsTrueIfDependenciesAreDirty(self): resource_manager = MockResource() dep1 = MockResource() dep2 = MockResource() dep3 = MockResource() resource_manager.resources.append(("dep1", dep1)) resource_manager.resources.append(("dep2", dep2)) resource_manager.resources.append(("dep3", dep3)) r = resource_manager.getResource() dep2.dirtied(r.dep2) self.assertTrue(resource_manager.isDirty()) resource_manager.finishedWith(r) def testRepeatedGetResourceCallsMakeResourceOnceOnly(self): resource_manager = MockResource() resource_manager.getResource() resource_manager.getResource() self.assertEqual(1, resource_manager.makes) def testGetResourceResetsUsedResource(self): resource_manager = MockResettableResource() resource_manager.getResource() resource = resource_manager.getResource() self.assertEqual(1, resource_manager.makes) resource_manager.dirtied(resource) resource_manager.getResource() self.assertEqual(1, resource_manager.makes) self.assertEqual(1, resource_manager.resets) resource_manager.finishedWith(resource) def testIsResetIfDependenciesAreDirty(self): resource_manager = MockResource() dep1 = MockResettableResource() resource_manager.resources.append(("dep1", dep1)) r = resource_manager.getResource() dep1.dirtied(r.dep1) # if we get the resource again, it should be cleaned. r = resource_manager.getResource() self.assertFalse(resource_manager.isDirty()) self.assertFalse(dep1.isDirty()) resource_manager.finishedWith(r) resource_manager.finishedWith(r) def testUsedResourceResetBetweenUses(self): resource_manager = MockResettableResource() # take two refs; like happens with OptimisingTestSuite. resource_manager.getResource() resource = resource_manager.getResource() resource_manager.dirtied(resource) resource_manager.finishedWith(resource) # Get again, but its been dirtied. resource = resource_manager.getResource() resource_manager.finishedWith(resource) resource_manager.finishedWith(resource) # The resource is made once, reset once and cleaned once. self.assertEqual(1, resource_manager.makes) self.assertEqual(1, resource_manager.resets) self.assertEqual(1, resource_manager.cleans) def testFinishedWithDecrementsUses(self): resource_manager = MockResource() resource = resource_manager.getResource() resource = resource_manager.getResource() self.assertEqual(2, resource_manager._uses) resource_manager.finishedWith(resource) self.assertEqual(1, resource_manager._uses) resource_manager.finishedWith(resource) self.assertEqual(0, resource_manager._uses) def testFinishedWithResetsCurrentResource(self): resource_manager = MockResource() resource = resource_manager.getResource() resource_manager.finishedWith(resource) self.assertIs(None, resource_manager._currentResource) def testFinishedWithCallsCleanResource(self): resource_manager = MockResource() resource = resource_manager.getResource() resource_manager.finishedWith(resource) self.assertEqual(1, resource_manager.cleans) def testUsingTwiceMakesAndCleansTwice(self): resource_manager = MockResource() resource = resource_manager.getResource() resource_manager.finishedWith(resource) resource = resource_manager.getResource() resource_manager.finishedWith(resource) self.assertEqual(2, resource_manager.makes) self.assertEqual(2, resource_manager.cleans) def testFinishedWithCallsCleanResourceOnceOnly(self): resource_manager = MockResource() resource = resource_manager.getResource() resource = resource_manager.getResource() resource_manager.finishedWith(resource) self.assertEqual(0, resource_manager.cleans) resource_manager.finishedWith(resource) self.assertEqual(1, resource_manager.cleans) def testFinishedWithMarksNonDirty(self): resource_manager = MockResource() resource = resource_manager.getResource() resource_manager.dirtied(resource) resource_manager.finishedWith(resource) self.assertEqual(False, resource_manager._dirty) def testResourceAvailableBetweenFinishedWithCalls(self): resource_manager = MockResource() resource = resource_manager.getResource() resource = resource_manager.getResource() resource_manager.finishedWith(resource) self.assertIs(resource, resource_manager._currentResource) resource_manager.finishedWith(resource) def testDirtiedSetsDirty(self): resource_manager = MockResource() resource = resource_manager.getResource() self.assertEqual(False, resource_manager._dirty) resource_manager.dirtied(resource) self.assertEqual(True, resource_manager._dirty) def testDirtyingResourceTriggersCleanOnGet(self): resource_manager = MockResource() resource1 = resource_manager.getResource() resource2 = resource_manager.getResource() resource_manager.dirtied(resource2) resource_manager.finishedWith(resource2) self.assertEqual(0, resource_manager.cleans) resource3 = resource_manager.getResource() self.assertEqual(1, resource_manager.cleans) resource_manager.finishedWith(resource3) resource_manager.finishedWith(resource1) self.assertEqual(2, resource_manager.cleans) def testDefaultResetMethodPreservesCleanResource(self): resource_manager = MockResource() resource = resource_manager.getResource() self.assertEqual(1, resource_manager.makes) self.assertEqual(False, resource_manager._dirty) resource_manager.reset(resource) self.assertEqual(1, resource_manager.makes) self.assertEqual(0, resource_manager.cleans) def testDefaultResetMethodRecreatesDirtyResource(self): resource_manager = MockResource() resource = resource_manager.getResource() self.assertEqual(1, resource_manager.makes) resource_manager.dirtied(resource) resource_manager.reset(resource) self.assertEqual(2, resource_manager.makes) self.assertEqual(1, resource_manager.cleans) def testDefaultResetResetsDependencies(self): resource_manager = MockResettableResource() dep1 = MockResettableResource() dep2 = MockResettableResource() resource_manager.resources.append(("dep1", dep1)) resource_manager.resources.append(("dep2", dep2)) # A typical OptimisingTestSuite workflow r_outer = resource_manager.getResource() # test 1 r_inner = resource_manager.getResource() dep2.dirtied(r_inner.dep2) resource_manager.finishedWith(r_inner) # test 2 r_inner = resource_manager.getResource() dep2.dirtied(r_inner.dep2) resource_manager.finishedWith(r_inner) resource_manager.finishedWith(r_outer) # Dep 1 was clean, doesn't do a reset, and should only have one # make+clean. self.assertEqual(1, dep1.makes) self.assertEqual(1, dep1.cleans) self.assertEqual(0, dep1.resets) # Dep 2 was dirty, so _reset happens, and likewise only one make and # clean. self.assertEqual(1, dep2.makes) self.assertEqual(1, dep2.cleans) self.assertEqual(1, dep2.resets) # The top layer should have had a reset happen, and only one make and # clean. self.assertEqual(1, resource_manager.makes) self.assertEqual(1, resource_manager.cleans) self.assertEqual(1, resource_manager.resets) def testDirtyingWhenUnused(self): resource_manager = MockResource() resource = resource_manager.getResource() resource_manager.finishedWith(resource) resource_manager.dirtied(resource) self.assertEqual(1, resource_manager.makes) resource = resource_manager.getResource() self.assertEqual(2, resource_manager.makes) def testFinishedActivityForResourceWithoutExtensions(self): result = ResultWithoutResourceExtensions() resource_manager = MockResource() r = resource_manager.getResource() resource_manager.finishedWith(r, result) def testFinishedActivityForResourceWithExtensions(self): result = ResultWithResourceExtensions() resource_manager = MockResource() r = resource_manager.getResource() expected = [("clean", "start", resource_manager), ("clean", "stop", resource_manager)] resource_manager.finishedWith(r, result) self.assertEqual(expected, result._calls) def testGetActivityForResourceWithoutExtensions(self): result = ResultWithoutResourceExtensions() resource_manager = MockResource() r = resource_manager.getResource(result) resource_manager.finishedWith(r) def testGetActivityForResourceWithExtensions(self): result = ResultWithResourceExtensions() resource_manager = MockResource() r = resource_manager.getResource(result) expected = [("make", "start", resource_manager), ("make", "stop", resource_manager)] resource_manager.finishedWith(r) self.assertEqual(expected, result._calls) def testResetActivityForResourceWithoutExtensions(self): result = ResultWithoutResourceExtensions() resource_manager = MockResource() resource_manager.getResource() r = resource_manager.getResource() resource_manager.dirtied(r) resource_manager.finishedWith(r) r = resource_manager.getResource(result) resource_manager.dirtied(r) resource_manager.finishedWith(r) resource_manager.finishedWith(resource_manager._currentResource) def testResetActivityForResourceWithExtensions(self): result = ResultWithResourceExtensions() resource_manager = MockResource() expected = [("reset", "start", resource_manager), ("reset", "stop", resource_manager), ] resource_manager.getResource() r = resource_manager.getResource() resource_manager.dirtied(r) resource_manager.finishedWith(r) r = resource_manager.getResource(result) resource_manager.dirtied(r) resource_manager.finishedWith(r) resource_manager.finishedWith(resource_manager._currentResource) self.assertEqual(expected, result._calls) class TestGenericResource(testtools.TestCase): def test_default_uses_setUp_tearDown(self): calls = [] class Wrapped: def setUp(self): calls.append('setUp') def tearDown(self): calls.append('tearDown') mgr = testresources.GenericResource(Wrapped) resource = mgr.getResource() self.assertEqual(['setUp'], calls) mgr.finishedWith(resource) self.assertEqual(['setUp', 'tearDown'], calls) self.assertIsInstance(resource, Wrapped) def test_dependencies_passed_to_factory(self): calls = [] class Wrapped: def __init__(self, **args): calls.append(args) def setUp(self):pass def tearDown(self):pass class Trivial(testresources.TestResource): def __init__(self, thing): testresources.TestResource.__init__(self) self.thing = thing def make(self, dependency_resources):return self.thing def clean(self, resource):pass mgr = testresources.GenericResource(Wrapped) mgr.resources = [('foo', Trivial('foo')), ('bar', Trivial('bar'))] resource = mgr.getResource() self.assertEqual([{'foo':'foo', 'bar':'bar'}], calls) mgr.finishedWith(resource) def test_setup_teardown_controllable(self): calls = [] class Wrapped: def start(self): calls.append('setUp') def stop(self): calls.append('tearDown') mgr = testresources.GenericResource(Wrapped, setup_method_name='start', teardown_method_name='stop') resource = mgr.getResource() self.assertEqual(['setUp'], calls) mgr.finishedWith(resource) self.assertEqual(['setUp', 'tearDown'], calls) self.assertIsInstance(resource, Wrapped) def test_always_dirty(self): class Wrapped: def setUp(self):pass def tearDown(self):pass mgr = testresources.GenericResource(Wrapped) resource = mgr.getResource() self.assertTrue(mgr.isDirty()) mgr.finishedWith(resource) class TestFixtureResource(testtools.TestCase): def test_uses_setUp_cleanUp(self): fixture = LoggingFixture() mgr = testresources.FixtureResource(fixture) resource = mgr.getResource() self.assertEqual(fixture, resource) self.assertEqual(['setUp'], fixture.calls) mgr.finishedWith(resource) self.assertEqual(['setUp', 'cleanUp'], fixture.calls) def test_always_dirty(self): fixture = LoggingFixture() mgr = testresources.FixtureResource(fixture) resource = mgr.getResource() self.assertTrue(mgr.isDirty()) mgr.finishedWith(resource) def test_reset_called(self): fixture = LoggingFixture() mgr = testresources.FixtureResource(fixture) resource = mgr.getResource() mgr.reset(resource) mgr.finishedWith(resource) self.assertEqual( ['setUp', 'reset', 'cleanUp'], fixture.calls) testresources-0.2.7/lib/testresources/tests/TestUtil.py0000644000175000017500000000550212076613406024534 0ustar robertcrobertc00000000000000# Copyright (c) 2004 Canonical Limited # Author: Robert Collins # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA # import sys import logging import unittest class LogCollector(logging.Handler): def __init__(self): logging.Handler.__init__(self) self.records=[] def emit(self, record): self.records.append(record.getMessage()) def makeCollectingLogger(): """I make a logger instance that collects its logs for programmatic analysis -> (logger, collector)""" logger=logging.Logger("collector") handler=LogCollector() handler.setFormatter(logging.Formatter("%(levelname)s: %(message)s")) logger.addHandler(handler) return logger, handler def visitTests(suite, visitor): """A foreign method for visiting the tests in a test suite.""" if isinstance(suite, unittest.TestCase): visitor.visitCase(suite) return for test in suite._tests: #Abusing types to avoid monkey patching unittest.TestCase. # Maybe that would be better? try: test.visit(visitor) except AttributeError: if isinstance(test, unittest.TestCase): visitor.visitCase(test) elif isinstance(test, unittest.TestSuite): visitor.visitSuite(test) visitTests(test, visitor) else: print("unvisitable non-unittest.TestCase element %r (%r)" % (test, test.__class__)) class TestSuite(unittest.TestSuite): """I am an extended TestSuite with a visitor interface. This is primarily to allow filtering of tests - and suites or more in the future. An iterator of just tests wouldn't scale...""" def visit(self, visitor): """visit the composite. Visiting is depth-first. current callbacks are visitSuite and visitCase.""" visitor.visitSuite(self) visitTests(self, visitor) class TestLoader(unittest.TestLoader): """Custome TestLoader to set the right TestSuite class.""" suiteClass = TestSuite class TestVisitor(object): """A visitor for Tests""" def visitSuite(self, aTestSuite): pass def visitCase(self, aTestCase): pass testresources-0.2.7/lib/testresources/tests/__init__.py0000644000175000017500000000470012076755623024525 0ustar robertcrobertc00000000000000# testresources: extensions to python unittest to allow declaritive use # of resources by test cases. # # Copyright (c) 2005-2010 Testresources Contributors # # Licensed under either the Apache License, Version 2.0 or the BSD 3-clause # license at the users choice. A copy of both licenses are available in the # project source as Apache-2.0 and BSD. You may not use this file except in # compliance with one of these two licences. # # Unless required by applicable law or agreed to in writing, software distributed # under these licenses is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR # CONDITIONS OF ANY KIND, either express or implied. See the license you chose # for the specific language governing permissions and limitations under that # license. # from unittest import TestResult import testresources from testresources.tests import TestUtil def test_suite(): import testresources.tests.test_optimising_test_suite import testresources.tests.test_resourced_test_case import testresources.tests.test_test_loader import testresources.tests.test_test_resource import testresources.tests.test_resource_graph result = TestUtil.TestSuite() result.addTest(testresources.tests.test_test_loader.test_suite()) result.addTest(testresources.tests.test_test_resource.test_suite()) result.addTest(testresources.tests.test_resourced_test_case.test_suite()) result.addTest(testresources.tests.test_resource_graph.test_suite()) result.addTest( testresources.tests.test_optimising_test_suite.test_suite()) return result class ResultWithoutResourceExtensions(object): """A test fake which does not have resource extensions.""" class ResultWithResourceExtensions(TestResult): """A test fake which has resource extensions.""" def __init__(self): TestResult.__init__(self) self._calls = [] def startCleanResource(self, resource): self._calls.append(("clean", "start", resource)) def stopCleanResource(self, resource): self._calls.append(("clean", "stop", resource)) def startMakeResource(self, resource): self._calls.append(("make", "start", resource)) def stopMakeResource(self, resource): self._calls.append(("make", "stop", resource)) def startResetResource(self, resource): self._calls.append(("reset", "start", resource)) def stopResetResource(self, resource): self._calls.append(("reset", "stop", resource)) testresources-0.2.7/lib/testresources/tests/test_test_loader.py0000644000175000017500000000245411342050734026320 0ustar robertcrobertc00000000000000# testresources: extensions to python unittest to allow declaritive use # of resources by test cases. # # Copyright (c) 2005-2010 Testresources Contributors # # Licensed under either the Apache License, Version 2.0 or the BSD 3-clause # license at the users choice. A copy of both licenses are available in the # project source as Apache-2.0 and BSD. You may not use this file except in # compliance with one of these two licences. # # Unless required by applicable law or agreed to in writing, software distributed # under these licenses is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR # CONDITIONS OF ANY KIND, either express or implied. See the license you chose # for the specific language governing permissions and limitations under that # license. # import testtools from testresources import TestLoader, OptimisingTestSuite from testresources.tests import TestUtil def test_suite(): loader = TestUtil.TestLoader() result = loader.loadTestsFromName(__name__) return result class TestTestLoader(testtools.TestCase): def testSuiteType(self): # The testresources TestLoader loads tests into an # OptimisingTestSuite. loader = TestLoader() suite = loader.loadTestsFromName(__name__) self.assertIsInstance(suite, OptimisingTestSuite) testresources-0.2.7/lib/testresources/tests/test_resourced_test_case.py0000644000175000017500000001473211560346757030061 0ustar robertcrobertc00000000000000# testresources: extensions to python unittest to allow declaritive use # of resources by test cases. # # Copyright (c) 2005-2010 Testresources Contributors # # Licensed under either the Apache License, Version 2.0 or the BSD 3-clause # license at the users choice. A copy of both licenses are available in the # project source as Apache-2.0 and BSD. You may not use this file except in # compliance with one of these two licences. # # Unless required by applicable law or agreed to in writing, software distributed # under these licenses is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR # CONDITIONS OF ANY KIND, either express or implied. See the license you chose # for the specific language governing permissions and limitations under that # license. # import unittest import testtools import testresources from testresources.tests import ResultWithResourceExtensions def test_suite(): loader = testresources.tests.TestUtil.TestLoader() result = loader.loadTestsFromName(__name__) return result class MockResource(testresources.TestResource): """Resource used for testing ResourcedTestCase.""" def __init__(self, resource): testresources.TestResource.__init__(self) self._resource = resource def make(self, dependency_resources): return self._resource class MockResourceInstance(object): """A resource instance.""" class TestResourcedTestCase(testtools.TestCase): def setUp(self): super(TestResourcedTestCase, self).setUp() class Example(testresources.ResourcedTestCase): def test_example(self): pass self.resourced_case = Example('test_example') self.resource = self.getUniqueString() self.resource_manager = MockResource(self.resource) def testSetUpUsesSuper(self): class OtherBaseCase(unittest.TestCase): setUpCalled = False def setUp(self): self.setUpCalled = True super(OtherBaseCase, self).setUp() class OurCase(testresources.ResourcedTestCase, OtherBaseCase): def runTest(self): pass ourCase = OurCase() ourCase.setUp() self.assertTrue(ourCase.setUpCalled) def testTearDownUsesSuper(self): class OtherBaseCase(unittest.TestCase): tearDownCalled = False def tearDown(self): self.tearDownCalled = True super(OtherBaseCase, self).setUp() class OurCase(testresources.ResourcedTestCase, OtherBaseCase): def runTest(self): pass ourCase = OurCase() ourCase.setUp() ourCase.tearDown() self.assertTrue(ourCase.tearDownCalled) def testDefaults(self): self.assertEqual(self.resourced_case.resources, []) def testResultPassedToResources(self): result = ResultWithResourceExtensions() self.resourced_case.resources = [("foo", self.resource_manager)] self.resourced_case.run(result) self.assertEqual(4, len(result._calls)) def testSetUpResourcesSingle(self): # setUpResources installs the resources listed in ResourcedTestCase. self.resourced_case.resources = [("foo", self.resource_manager)] testresources.setUpResources(self.resourced_case, self.resourced_case.resources, None) self.assertEqual(self.resource, self.resourced_case.foo) def testSetUpResourcesMultiple(self): # setUpResources installs the resources listed in ResourcedTestCase. self.resourced_case.resources = [ ('foo', self.resource_manager), ('bar', MockResource('bar_resource'))] testresources.setUpResources(self.resourced_case, self.resourced_case.resources, None) self.assertEqual(self.resource, self.resourced_case.foo) self.assertEqual('bar_resource', self.resourced_case.bar) def testSetUpResourcesSetsUpDependences(self): resource = MockResourceInstance() self.resource_manager = MockResource(resource) self.resourced_case.resources = [('foo', self.resource_manager)] # Give the 'foo' resource access to a 'bar' resource self.resource_manager.resources.append( ('bar', MockResource('bar_resource'))) testresources.setUpResources(self.resourced_case, self.resourced_case.resources, None) self.assertEqual(resource, self.resourced_case.foo) self.assertEqual('bar_resource', self.resourced_case.foo.bar) def testSetUpUsesResource(self): # setUpResources records a use of each declared resource. self.resourced_case.resources = [("foo", self.resource_manager)] testresources.setUpResources(self.resourced_case, self.resourced_case.resources, None) self.assertEqual(self.resource_manager._uses, 1) def testTearDownResourcesDeletesResourceAttributes(self): self.resourced_case.resources = [("foo", self.resource_manager)] self.resourced_case.setUpResources() self.resourced_case.tearDownResources() self.failIf(hasattr(self.resourced_case, "foo")) def testTearDownResourcesStopsUsingResource(self): # tearDownResources records that there is one less use of each # declared resource. self.resourced_case.resources = [("foo", self.resource_manager)] self.resourced_case.setUpResources() self.resourced_case.tearDownResources() self.assertEqual(self.resource_manager._uses, 0) def testTearDownResourcesStopsUsingDependencies(self): resource = MockResourceInstance() dep1 = MockResource('bar_resource') self.resource_manager = MockResource(resource) self.resourced_case.resources = [('foo', self.resource_manager)] # Give the 'foo' resource access to a 'bar' resource self.resource_manager.resources.append( ('bar', dep1)) self.resourced_case.setUpResources() self.resourced_case.tearDownResources() self.assertEqual(dep1._uses, 0) def testSingleWithSetup(self): # setUp and tearDown invoke setUpResources and tearDownResources. self.resourced_case.resources = [("foo", self.resource_manager)] self.resourced_case.setUp() self.assertEqual(self.resourced_case.foo, self.resource) self.assertEqual(self.resource_manager._uses, 1) self.resourced_case.tearDown() self.failIf(hasattr(self.resourced_case, "foo")) self.assertEqual(self.resource_manager._uses, 0) testresources-0.2.7/lib/testresources/__init__.py0000644000175000017500000007715712076756075023405 0ustar robertcrobertc00000000000000# testresources: extensions to python unittest to allow declaritive use # of resources by test cases. # # Copyright (c) 2005-2010 Testresources Contributors # # Licensed under either the Apache License, Version 2.0 or the BSD 3-clause # license at the users choice. A copy of both licenses are available in the # project source as Apache-2.0 and BSD. You may not use this file except in # compliance with one of these two licences. # # Unless required by applicable law or agreed to in writing, software # distributed under these licenses is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # license you chose for the specific language governing permissions and # limitations under that license. # """TestResources: declarative management of external resources for tests.""" import heapq import inspect import unittest # same format as sys.version_info: "A tuple containing the five components of # the version number: major, minor, micro, releaselevel, and serial. All # values except releaselevel are integers; the release level is 'alpha', # 'beta', 'candidate', or 'final'. The version_info value corresponding to the # Python version 2.0 is (2, 0, 0, 'final', 0)." Additionally we use a # releaselevel of 'dev' for unreleased under-development code. # # If the releaselevel is 'alpha' then the major/minor/micro components are not # established at this point, and setup.py will use a version of next-$(revno). # If the releaselevel is 'final', then the tarball will be major.minor.micro. # Otherwise it is major.minor.micro~$(revno). __version__ = (0, 2, 7, 'final', 0) def test_suite(): import testresources.tests return testresources.tests.test_suite() def _digraph_to_graph(digraph, prime_node_mapping): """Convert digraph to a graph. :param digraph: A directed graph in the form {from:{to:value}}. :param prime_node_mapping: A mapping from every node in digraph to a new unique and not in digraph node. :return: A symmetric graph in the form {from:to:value}} created by creating edges in the result between every N to M-prime with the original N-M value and from every N to N-prime with a cost of 0. No other edges are created. """ result = {} for from_node, from_prime_node in prime_node_mapping.items(): result[from_node] = {from_prime_node: 0} result[from_prime_node] = {from_node: 0} for from_node, to_nodes in digraph.items(): from_prime = prime_node_mapping[from_node] for to_node, value in to_nodes.items(): to_prime = prime_node_mapping[to_node] result[from_prime][to_node] = value result[to_node][from_prime] = value return result def _kruskals_graph_MST(graph): """Find the minimal spanning tree in graph using Kruskals algorithm. See http://en.wikipedia.org/wiki/Kruskal%27s_algorithm. :param graph: A graph in {from:{to:value}} form. Every node present in graph must be in the outer dict (because graph is not a directed graph. :return: A graph with all nodes and those vertices that are part of the MST for graph. If graph is not connected, then the result will also be a forest. """ # forest contains all the nodes -> graph that node is in. forest = {} # graphs is the count of graphs we have yet to combine. for node in graph: forest[node] = {node: {}} graphs = len(forest) # collect edges: every edge is present twice (due to the graph # representation), so normalise. edges = set() for from_node, to_nodes in graph.items(): for to_node, value in to_nodes.items(): edge = (value,) + tuple(sorted([from_node, to_node])) edges.add(edge) edges = list(edges) heapq.heapify(edges) while edges and graphs > 1: # more edges to go and we haven't gotten a spanning tree yet. edge = heapq.heappop(edges) g1 = forest[edge[1]] g2 = forest[edge[2]] if g1 is g2: continue # already joined # combine g1 and g2 into g1 graphs -= 1 for from_node, to_nodes in g2.items(): #remember its symmetric, don't need to do 'to'. forest[from_node] = g1 g1.setdefault(from_node, {}).update(to_nodes) # add edge g1[edge[1]][edge[2]] = edge[0] g1[edge[2]][edge[1]] = edge[0] # union the remaining graphs _, result = forest.popitem() for _, g2 in forest.items(): if g2 is result: # common case continue for from_node, to_nodes in g2.items(): result.setdefault(from_node, {}).update(to_nodes) return result def _resource_graph(resource_sets): """Convert an iterable of resource_sets into a graph. Each resource_set in the iterable is treated as a node, and each resource in that resource_set is used as an edge to other nodes. """ nodes = {} edges = {} for resource_set in resource_sets: # put node in nodes node = frozenset(resource_set) nodes[node] = set() # put its contents in as edges for resource in resource_set: edges.setdefault(resource, []).append(node) # populate the adjacent members of nodes for node, connected in nodes.items(): for resource in node: connected.update(edges[resource]) connected.discard(node) return nodes def split_by_resources(tests): """Split a list of tests by the resources that the tests use. :return: a dictionary mapping sets of resources to lists of tests using that combination of resources. The dictionary always contains an entry for "no resources". """ no_resources = frozenset() resource_set_tests = {no_resources: []} for test in tests: resources = getattr(test, "resources", ()) all_resources = list(resource.neededResources() for _, resource in resources) resource_set = set() for resource_list in all_resources: resource_set.update(resource_list) resource_set_tests.setdefault(frozenset(resource_set), []).append(test) return resource_set_tests def _strongly_connected_components(graph, no_resources): """Find the strongly connected components in graph. This is essentially a nonrecursive flatterning of Tarjan's method. It may be worth profiling against an actual Tarjan's implementation at some point, but sets are often faster than python calls. graph gets consumed, but that could be changed easily enough. """ partitions = [] while graph: node, pending = graph.popitem() current_partition = set([node]) while pending: # add all the nodes connected to a connected node to pending. node = pending.pop() current_partition.add(node) pending.update(graph.pop(node, [])) # don't try to process things we've allready processed. pending.difference_update(current_partition) partitions.append(current_partition) return partitions class OptimisingTestSuite(unittest.TestSuite): """A resource creation optimising TestSuite.""" def adsorbSuite(self, test_case_or_suite): """Deprecated. Use addTest instead.""" self.addTest(test_case_or_suite) def addTest(self, test_case_or_suite): """Add `test_case_or_suite`, unwrapping standard TestSuites. This means that any containing unittest.TestSuites will be removed, while any custom test suites will be 'distributed' across their members. Thus addTest(CustomSuite([a, b])) will result in CustomSuite([a]) and CustomSuite([b]) being added to this suite. """ try: tests = iter(test_case_or_suite) except TypeError: unittest.TestSuite.addTest(self, test_case_or_suite) return if test_case_or_suite.__class__ in (unittest.TestSuite, OptimisingTestSuite): for test in tests: self.adsorbSuite(test) else: for test in tests: unittest.TestSuite.addTest( self, test_case_or_suite.__class__([test])) def cost_of_switching(self, old_resource_set, new_resource_set): """Cost of switching from 'old_resource_set' to 'new_resource_set'. This is calculated by adding the cost of tearing down unnecessary resources to the cost of setting up the newly-needed resources. Note that resources which are always dirtied may skew the predicted skew the cost of switching because they are considered common, even when reusing them may actually be equivalent to a teardown+setup operation. """ new_resources = new_resource_set - old_resource_set gone_resources = old_resource_set - new_resource_set return (sum(resource.setUpCost for resource in new_resources) + sum(resource.tearDownCost for resource in gone_resources)) def switch(self, old_resource_set, new_resource_set, result): """Switch from 'old_resource_set' to 'new_resource_set'. Tear down resources in old_resource_set that aren't in new_resource_set and set up resources that are in new_resource_set but not in old_resource_set. :param result: TestResult object to report activity on. """ new_resources = new_resource_set - old_resource_set old_resources = old_resource_set - new_resource_set for resource in old_resources: resource.finishedWith(resource._currentResource, result) for resource in new_resources: resource.getResource(result) def run(self, result): self.sortTests() current_resources = set() for test in self._tests: if result.shouldStop: break resources = getattr(test, 'resources', []) new_resources = set() for name, resource in resources: new_resources.update(resource.neededResources()) self.switch(current_resources, new_resources, result) current_resources = new_resources test(result) self.switch(current_resources, set(), result) return result def sortTests(self): """Attempt to topographically sort the contained tests. This function biases to reusing a resource: it assumes that resetting a resource is usually cheaper than a teardown + setup; and that most resources are not dirtied by most tests. Feel free to override to improve the sort behaviour. """ # We group the tests by the resource combinations they use, # since there will usually be fewer resource combinations than # actual tests and there can never be more: This gives us 'nodes' or # 'resource_sets' that represent many tests using the same set of # resources. resource_set_tests = split_by_resources(self._tests) # Partition into separate sets of resources, there is no ordering # preference between sets that do not share members. Rationale: # If resource_set A and B have no common resources, AB and BA are # equally good - the global setup/teardown sums are identical. Secondly # if A shares one or more resources with C, then pairing AC|CA is # better than having B between A and C, because the shared resources # can be reset or reused. Having partitioned we can use connected graph # logic on each partition. resource_set_graph = _resource_graph(resource_set_tests) no_resources = frozenset() # A list of resource_set_tests, all fully internally connected. partitions = _strongly_connected_components(resource_set_graph, no_resources) result = [] for partition in partitions: # we process these at the end for no particularly good reason (it # makes testing slightly easier). if partition == [no_resources]: continue order = self._makeOrder(partition) # Spit this partition out into result for resource_set in order: result.extend(resource_set_tests[resource_set]) result.extend(resource_set_tests[no_resources]) self._tests = result def _getGraph(self, resource_sets): """Build a graph of the resource-using nodes. This special cases set(['root']) to be a node with no resources and edges to everything. :return: A complete directed graph of the switching costs between each resource combination. Note that links from N to N are not included. """ no_resources = frozenset() graph = {} root = set(['root']) # bottom = set(['bottom']) for from_set in resource_sets: graph[from_set] = {} if from_set == root: from_resources = no_resources #elif from_set == bottom: # continue # no links from bottom else: from_resources = from_set for to_set in resource_sets: if from_set is to_set: continue # no self-edges #if to_set == bottom: # if from_set == root: # continue # no short cuts! # to_resources = no_resources #el if to_set == root: continue # no links to root else: to_resources = to_set graph[from_set][to_set] = self.cost_of_switching( from_resources, to_resources) return graph def _makeOrder(self, partition): """Return a order for the resource sets in partition.""" # This problem is NP-C - find the lowest cost hamiltonian path. It # also meets the triangle inequality, so we can use an approximation. # TODO: implement Christofides. # See: # http://en.wikipedia.org/wiki/Travelling_salesman_problem#Metric_TSP # We need a root root = frozenset(['root']) partition.add(root) # and an end # partition.add(frozenset(['bottom'])) # get rid of 'noresources' partition.discard(frozenset()) digraph = self._getGraph(partition) # build a prime map primes = {} prime = frozenset(['prime']) for node in digraph: primes[node] = node.union(prime) graph = _digraph_to_graph(digraph, primes) mst = _kruskals_graph_MST(graph) # Because the representation is a digraph, we can build an Eulerian # cycle directly from the representation by just following the links: # a node with only 1 'edge' has two directed edges; and we can only # enter and leave it once, so the edge lookups will match precisely. # As the mst is a spanning tree, the graph will become disconnected # (we choose non-disconnecting edges first) # - for a stub node (1 outgoing link): when exiting it unless it is # the first node started at # - for a non-stub node if choosing an outgoing link where some other # endpoints incoming link has not been traversed. [exit by a # different node than entering, until all exits taken]. # We don't need the mst after, so it gets modified in place. node = root cycle = [node] steps = 2 * (len(mst) - 1) for step in range(steps): found = False outgoing = None # For clearer debugging. for outgoing in mst[node]: if node in mst[outgoing]: # we have a return path: take it # print node, '->', outgoing, ' can return' del mst[node][outgoing] node = outgoing cycle.append(node) found = True break if not found: # none of the outgoing links have an incoming, so follow an # arbitrary one (the last examined outgoing) # print node, '->', outgoing del mst[node][outgoing] node = outgoing cycle.append(node) # Convert to a path: visited = set() order = [] for node in cycle: if node in visited: continue if node in primes: order.append(node) visited.add(node) assert order[0] == root return order[1:] class TestLoader(unittest.TestLoader): """Custom TestLoader to set the right TestSuite class.""" suiteClass = OptimisingTestSuite class TestResourceManager(object): """A manager for resources that can be shared across tests. ResourceManagers can report activity to a TestResult. The methods - startCleanResource(resource) - stopCleanResource(resource) - startMakeResource(resource) - stopMakeResource(resource) will be looked for and if present invoked before and after cleaning or creation of resource objects takes place. :cvar resources: The same as the resources list on an instance, the default constructor will look for the class instance and copy it. This is a convenience to avoid needing to define __init__ solely to alter the dependencies list. :ivar resources: The resources that this resource needs. Calling neededResources will return the closure of this resource and its needed resources. The resources list is in the same format as resources on a test case - a list of tuples (attribute_name, resource). :ivar setUpCost: The relative cost to construct a resource of this type. One good approach is to set this to the number of seconds it normally takes to set up the resource. :ivar tearDownCost: The relative cost to tear down a resource of this type. One good approach is to set this to the number of seconds it normally takes to tear down the resource. """ setUpCost = 1 tearDownCost = 1 def __init__(self): """Create a TestResourceManager object.""" self._dirty = False self._uses = 0 self._currentResource = None self.resources = list(getattr(self.__class__, "resources", [])) def _call_result_method_if_exists(self, result, methodname, *args): """Call a method on a TestResult that may exist.""" method = getattr(result, methodname, None) if callable(method): method(*args) def _clean_all(self, resource, result): """Clean the dependencies from resource, and then resource itself.""" self._call_result_method_if_exists(result, "startCleanResource", self) self.clean(resource) for name, manager in self.resources: manager.finishedWith(getattr(resource, name)) self._call_result_method_if_exists(result, "stopCleanResource", self) def clean(self, resource): """Override this to class method to hook into resource removal.""" def dirtied(self, resource): """Mark the resource as having been 'dirtied'. A resource is dirty when it is no longer suitable for use by other tests. e.g. a shared database that has had rows changed. """ self._dirty = True def finishedWith(self, resource, result=None): """Indicate that 'resource' has one less user. If there are no more registered users of 'resource' then we trigger the `clean` hook, which should do any resource-specific cleanup. :param resource: A resource returned by `TestResourceManager.getResource`. :param result: An optional TestResult to report resource changes to. """ self._uses -= 1 if self._uses == 0: self._clean_all(resource, result) self._setResource(None) def getResource(self, result=None): """Get the resource for this class and record that it's being used. The resource is constructed using the `make` hook. Once done with the resource, pass it to `finishedWith` to indicated that it is no longer needed. :param result: An optional TestResult to report resource changes to. """ if self._uses == 0: self._setResource(self._make_all(result)) elif self.isDirty(): self._setResource(self.reset(self._currentResource, result)) self._uses += 1 return self._currentResource def isDirty(self): """Return True if this managers cached resource is dirty. Calling when the resource is not currently held has undefined behaviour. """ if self._dirty: return True for name, mgr in self.resources: if mgr.isDirty(): return True res = mgr.getResource() try: if res is not getattr(self._currentResource, name): return True finally: mgr.finishedWith(res) def _make_all(self, result): """Make the dependencies of this resource and this resource.""" self._call_result_method_if_exists(result, "startMakeResource", self) dependency_resources = {} for name, resource in self.resources: dependency_resources[name] = resource.getResource() resource = self.make(dependency_resources) for name, value in dependency_resources.items(): setattr(resource, name, value) self._call_result_method_if_exists(result, "stopMakeResource", self) return resource def make(self, dependency_resources): """Override this to construct resources. :param dependency_resources: A dict mapping name -> resource instance for the resources specified as dependencies. :return: The made resource. """ raise NotImplementedError( "Override make to construct resources.") def neededResources(self): """Return the resources needed for this resource, including self. :return: A list of needed resources, in topological deepest-first order. """ seen = set([self]) result = [] for name, resource in self.resources: for resource in resource.neededResources(): if resource in seen: continue seen.add(resource) result.append(resource) result.append(self) return result def reset(self, old_resource, result=None): """Return a clean version of old_resource. By default, the resource will be cleaned then remade if it had previously been `dirtied` by the helper self._reset() - which is the extension point folk should override to customise reset behaviour. This function takes the dependent resource stack into consideration as _make_all and _clean_all do. The inconsistent naming is because reset is part of the public interface, but _make_all and _clean_all is not. Note that if a resource A holds a lock or other blocking thing on a dependency D, reset will result in this call sequence over a getResource(), dirty(), getResource(), finishedWith(), finishedWith() sequence: B.make(), A.make(), B.reset(), A.reset(), A.clean(), B.clean() Thus it is important that B.reset not assume that A has been cleaned or reset before B is reset: it should arrange to reference count, lazy cleanup or forcibly reset resource in some fashion. As an example, consider that B is a database with sample data, and A is an application server serving content from it. B._reset() should disconnect all database clients, reset the state of the database, and A._reset() should tell the application server to dump any internal caches it might have. In principle we might make a richer API to allow before-and-after reset actions, but so far that hasn't been needed. :return: The possibly new resource. :param result: An optional TestResult to report resource changes to. """ # Core logic: # - if neither we nor any parent is dirty, do nothing. # otherwise # - emit a signal to the test result # - reset all dependencies all, getting new attributes. # - call self._reset(old_resource, dependency_attributes) # [the default implementation does a clean + make] if not self.isDirty(): return old_resource self._call_result_method_if_exists(result, "startResetResource", self) dependency_resources = {} for name, mgr in self.resources: dependency_resources[name] = mgr.reset( getattr(old_resource, name), result) resource = self._reset(old_resource, dependency_resources) for name, value in dependency_resources.items(): setattr(resource, name, value) self._call_result_method_if_exists(result, "stopResetResource", self) return resource def _reset(self, resource, dependency_resources): """Override this to reset resources other than via clean+make. This method should reset the self._dirty flag (assuming the manager can ever be clean) and return either the old resource cleaned or a fresh one. :param resource: The resource to reset. :param dependency_resources: A dict mapping name -> resource instance for the resources specified as dependencies. """ self.clean(resource) return self.make(dependency_resources) def _setResource(self, new_resource): """Set the current resource to a new value.""" self._currentResource = new_resource self._dirty = False TestResource = TestResourceManager class GenericResource(TestResourceManager): """A TestResourceManager that decorates an external helper of some kind. GenericResource can be used to adapt an external resource so that testresources can use it. By default the setUp and tearDown methods are called when making and cleaning the resource, and the resource is considered permanently dirty, so it is torn down and brought up again between every use. The constructor method is called with the dependency resources dict:: resource_factory(**dependency_resources) This permits naming those resources to match the contract of the setUp method. """ def __init__(self, resource_factory, setup_method_name='setUp', teardown_method_name='tearDown'): """Create a GenericResource :param resource_factory: A factory to create a new resource. :param setup_method_name: Optional method name to call to setup the resource. Defaults to 'setUp'. :param teardown_method_name: Optional method name to call to tear down the resource. Defaults to 'tearDown'. """ super(GenericResource, self).__init__() self.resource_factory = resource_factory self.setup_method_name = setup_method_name self.teardown_method_name = teardown_method_name def clean(self, resource): getattr(resource, self.teardown_method_name)() def make(self, dependency_resources): result = self.resource_factory(**dependency_resources) getattr(result, self.setup_method_name)() return result def isDirty(self): return True class FixtureResource(TestResourceManager): """A TestResourceManager that decorates a ``fixtures.Fixture``. The fixture has its setUp and cleanUp called as expected, and reset is called between uses. Due to the API of fixtures, dependency_resources are not accessible to the wrapped fixture. However, if you are using resource optimisation, you should wrap any dependencies in a FixtureResource and set the resources attribute appropriately. Note that when this is done, testresources will take care of calling setUp and cleanUp on the dependency fixtures and so the fixtures should not implicitly setUp or cleanUp their dependencies (that have been mapped). See the ``fixtures`` documentation for information on managing dependencies within the ``fixtures`` API. :ivar fixture: The wrapped fixture. """ def __init__(self, fixture): """Create a FixtureResource :param fixture: The fixture to wrap. """ super(FixtureResource, self).__init__() self.fixture = fixture def clean(self, resource): resource.cleanUp() def make(self, dependency_resources): self.fixture.setUp() return self.fixture def _reset(self, resource, dependency_resources): self.fixture.reset() return self.fixture def isDirty(self): return True _dirty = property(lambda _:True, lambda _, _1:None) class ResourcedTestCase(unittest.TestCase): """A TestCase parent or utility that enables cross-test resource usage. ResourcedTestCase is a thin wrapper around the testresources.setUpResources and testresources.tearDownResources helper functions. It should be trivially reimplemented where a different base class is neded, or you can use multiple inheritance and call into ResourcedTestCase.setUpResources and ResourcedTestCase.tearDownResources from your setUp and tearDown (or whatever cleanup idiom is used). :ivar resources: A list of (name, resource) pairs, where 'resource' is a subclass of `TestResourceManager` and 'name' is the name of the attribute that the resource should be stored on. """ resources = [] def setUp(self): super(ResourcedTestCase, self).setUp() self.setUpResources() def setUpResources(self): setUpResources(self, self.resources, _get_result()) def tearDown(self): self.tearDownResources() super(ResourcedTestCase, self).tearDown() def tearDownResources(self): tearDownResources(self, self.resources, _get_result()) def setUpResources(test, resources, result): """Set up resources for test. :param test: The test to setup resources for. :param resources: The resources to setup. :param result: A result object for tracing resource activity. """ for resource in resources: setattr(test, resource[0], resource[1].getResource(result)) def tearDownResources(test, resources, result): """Tear down resources for test. :param test: The test to tear down resources from. :param resources: The resources to tear down. :param result: A result object for tracing resource activity. """ for resource in resources: resource[1].finishedWith(getattr(test, resource[0]), result) delattr(test, resource[0]) def _get_result(): """Find a TestResult in the stack. unittest hides the result. This forces us to look up the stack. The result is passed to a run() or a __call__ method 4 or more frames up: that method is what calls setUp and tearDown, and they call their parent setUp etc. Its not guaranteed that the parameter to run will be calls result as its not required to be a keyword parameter in TestCase. However, in practice, this works. """ stack = inspect.stack() for frame in stack[2:]: if frame[3] in ('run', '__call__'): # Not all frames called 'run' will be unittest. It could be a # reactor in trial, for instance. result = frame[0].f_locals.get('result') if (result is not None and getattr(result, 'startTest', None) is not None): return result testresources-0.2.7/setup.cfg0000644000175000017500000000007312076756602017410 0ustar robertcrobertc00000000000000[egg_info] tag_build = tag_date = 0 tag_svn_revision = 0 testresources-0.2.7/README0000644000175000017500000002325212076755762016461 0ustar robertcrobertc00000000000000testresources: extensions to python unittest to allow declarative use of resources by test cases. Copyright (C) 2005-2013 Robert Collins Licensed under either the Apache License, Version 2.0 or the BSD 3-clause license at the users choice. A copy of both licenses are available in the project source as Apache-2.0 and BSD. You may not use this file except in compliance with one of these two licences. Unless required by applicable law or agreed to in writing, software distributed under these licenses is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the license you chose for the specific language governing permissions and limitations under that license. See the COPYING file for full details on the licensing of Testresources. Testresources +++++++++++++ testresources extends unittest with a clean and simple api to provide test optimisation where expensive common resources are needed for test cases - for example sample working trees for VCS systems, reference databases for enterprise applications, or web servers ... let imagination run wild. Dependencies to build/selftest ============================== * Python 2.4+ (or 3.2+) * testtools (http://pypi.python.org/pypi/testtools/) * fixtures (http://pypi.python.org/pypi/fixtures) Dependencies to use testresources ================================= * Python 2.4+ (or 3.2+) How testresources Works ======================= The basic idea of testresources is: * Tests declare the resources they need in a ``resources`` attribute. * When the test is run, the required resource objects are allocated (either newly constructed, or reused), and assigned to attributes of the TestCase. testresources distinguishes a 'resource manager' (a subclass of ``TestResourceManager``) which acts as a kind of factory, and a 'resource' which can be any kind of object returned from the manager class's ``getResource`` method. Resources are either clean or dirty. Being clean means they have same state in all important ways as a newly constructed instance and they can therefore be safely reused. Main Classes ============ testresources.ResourcedTestCase ------------------------------- By extending or mixing-in this class, tests can have necessary resources automatically allocated and disposed or recycled. ResourceTestCase can be used as a base class for tests, and when that is done tests will have their ``resources`` attribute automatically checked for resources by both OptimisingTestSuite and their own setUp() and tearDown() methods. (This allows tests to remain functional without needing this specific TestSuite as a container). Alternatively, you can call setUpResources(self, resources, test_result) and tearDownResources(self, resources, test_result) from your own classes setUp and tearDown and the same behaviour will be activated. To declare the use of a resource, set the ``resources`` attribute to a list of tuples of ``(attribute_name, resource_manager)``. During setUp, for each declared requirement, the test gains an attribute pointing to an allocated resource, which is the result of calling ``resource_manager.getResource()``. ``finishedWith`` will be called on each resource during tearDown(). For example:: class TestLog(testresources.ResourcedTestCase): resources = [('branch', BzrPopulatedBranch())] def test_log(self): show_log(self.branch, ...) testresources.TestResourceManager --------------------------------- A TestResourceManager is an object that tests can use to create resources. It can be overridden to manage different types of resources. Normally test code doesn't need to call any methods on it, as this will be arranged by the testresources machinery. When implementing a new ``TestResourceManager`` subclass you should consider overriding these methods: ``make`` Must be overridden in every concrete subclass. Returns a new instance of the resource object (the actual resource, not the TestResourceManager). Doesn't need to worry about reuse, which is taken care of separately. This method is only called when a new resource is definitely needed. ``make`` is called by ``getResource``; you should not normally need to override the latter. ``clean`` Cleans up an existing resource instance, eg by deleting a directory or closing a network connection. By default this does nothing, which may be appropriate for resources that are automatically garbage collected. ``_reset`` Reset a no-longer-used dirty resource to a clean state. By default this just discards it and creates a new one, but for some resources there may be a faster way to reset them. ``isDirty`` Check whether an existing resource is dirty. By default this just reports whether ``TestResourceManager.dirtied`` has been called or any of the dependency resources are dirty. For instance:: class TemporaryDirectoryResource(TestResourceManager): def clean(self, resource): shutil.rmtree(resource) def make(self): return tempfile.mkdtemp() def isDirty(self, resource): # Can't detect when the directory is written to, so assume it # can never be reused. We could list the directory, but that might # not catch it being open as a cwd etc. return True The ``resources`` list on the TestResourceManager object is used to declare dependencies. For instance, a DataBaseResource that needs a TemporaryDirectory might be declared with a resources list:: class DataBaseResource(TestResourceManager): resources = [("scratchdir", TemporaryDirectoryResource())] Most importantly, two getResources to the same TestResourceManager with no finishedWith call in the middle, will return the same object as long as it is not dirty. When a Test has a dependency and that dependency successfully completes but returns None, the framework does *not* consider this an error: be sure to always return a valid resource, or raise an error. Error handling hasn't been heavily exercised, but any bugs in this area will be promptly dealt with. A sample TestResourceManager can be found in the doc/ folder. See pydoc testresources.TestResourceManager for details. testresources.GenericResource ----------------------------- Glue to adapt testresources to an existing resource-like class. testresources.FixtureResource ----------------------------- Glue to adapt testresources to the simpler fixtures.Fixture API. Long term testresources is likely to consolidate on that simpler API as the recommended method of writing resources. testresources.OptimisingTestSuite --------------------------------- This TestSuite will introspect all the test cases it holds directly and if they declare needed resources, will run the tests in an order that attempts to minimise the number of setup and tear downs required. It attempts to achieve this by callling getResource() and finishedWith() around the sequence of tests that use a specific resource. Tests are added to an OptimisingTestSuite as normal. Any standard library TestSuite objects will be flattened, while any custom TestSuite subclasses will be distributed across their member tests. This means that any custom logic in test suites should be preserved, at the price of some level of optimisation. Because the test suite does the optimisation, you can control the amount of optimising that takes place by adding more or fewer tests to a single OptimisingTestSuite. You could add everything to a single OptimisingTestSuite, getting global optimisation or you could use several smaller OptimisingTestSuites. testresources.TestLoader ------------------------ This is a trivial TestLoader that creates OptimisingTestSuites by default. unittest.TestResult ------------------- testresources will log activity about resource creation and destruction to the result object tests are run with. 6 extension methods are looked for: ``startCleanResource``, ``stopCleanResource``, ``startMakeResource``, ``stopMakeResource``, ``startResetResource`` and finally ``stopResetResource``. ``testresources.tests.ResultWithResourceExtensions`` is an example of a ``TestResult`` with these methods present. Controlling Resource Reuse ========================== When or how do I mark the resource dirtied? The simplest approach is to have ``TestResourceManager.make`` call ``self.dirtied``: the resource is always immediately dirty and will never be reused without first being reset. This is appropriate when the underlying resource is cheap to reset or recreate, or when it's hard to detect whether it's been dirtied or to trap operations that change it. Alternatively, override ``TestResourceManager.isDirty`` and inspect the resource to see if it is safe to reuse. Finally, you can arrange for the returned resource to always call back to ``TestResourceManager.dirtied`` on the first operation that mutates it. FAQ === * Can I dynamically request resources inside a test method? Generally, no, you shouldn't do this. The idea is that the resources are declared statically, so that testresources can "smooth" resource usage across several tests. But, you may be able to find some object that is statically declared and reusable to act as the resource, which can then provide methods to generate sub-elements of itself during a test. * If the resource is held inside the TestResourceManager object, and the TestResourceManager is typically constructed inline in the test case ``resources`` attribute, how can they be shared across different test classes? Good question. I guess you should arrange for a single instance to be held in an appropriate module scope, then referenced by the test classes that want to share it. testresources-0.2.7/PKG-INFO0000644000175000017500000003072012076756602016666 0ustar robertcrobertc00000000000000Metadata-Version: 1.1 Name: testresources Version: 0.2.7 Summary: Testresources, a pyunit extension for managing expensive test resources Home-page: https://launchpad.net/testresources Author: Testresources developers Author-email: https://launchpad.net/~testresources-developers License: UNKNOWN Description: testresources: extensions to python unittest to allow declarative use of resources by test cases. Copyright (C) 2005-2013 Robert Collins Licensed under either the Apache License, Version 2.0 or the BSD 3-clause license at the users choice. A copy of both licenses are available in the project source as Apache-2.0 and BSD. You may not use this file except in compliance with one of these two licences. Unless required by applicable law or agreed to in writing, software distributed under these licenses is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the license you chose for the specific language governing permissions and limitations under that license. See the COPYING file for full details on the licensing of Testresources. Testresources +++++++++++++ testresources extends unittest with a clean and simple api to provide test optimisation where expensive common resources are needed for test cases - for example sample working trees for VCS systems, reference databases for enterprise applications, or web servers ... let imagination run wild. Dependencies to build/selftest ============================== * Python 2.4+ (or 3.2+) * testtools (http://pypi.python.org/pypi/testtools/) * fixtures (http://pypi.python.org/pypi/fixtures) Dependencies to use testresources ================================= * Python 2.4+ (or 3.2+) How testresources Works ======================= The basic idea of testresources is: * Tests declare the resources they need in a ``resources`` attribute. * When the test is run, the required resource objects are allocated (either newly constructed, or reused), and assigned to attributes of the TestCase. testresources distinguishes a 'resource manager' (a subclass of ``TestResourceManager``) which acts as a kind of factory, and a 'resource' which can be any kind of object returned from the manager class's ``getResource`` method. Resources are either clean or dirty. Being clean means they have same state in all important ways as a newly constructed instance and they can therefore be safely reused. Main Classes ============ testresources.ResourcedTestCase ------------------------------- By extending or mixing-in this class, tests can have necessary resources automatically allocated and disposed or recycled. ResourceTestCase can be used as a base class for tests, and when that is done tests will have their ``resources`` attribute automatically checked for resources by both OptimisingTestSuite and their own setUp() and tearDown() methods. (This allows tests to remain functional without needing this specific TestSuite as a container). Alternatively, you can call setUpResources(self, resources, test_result) and tearDownResources(self, resources, test_result) from your own classes setUp and tearDown and the same behaviour will be activated. To declare the use of a resource, set the ``resources`` attribute to a list of tuples of ``(attribute_name, resource_manager)``. During setUp, for each declared requirement, the test gains an attribute pointing to an allocated resource, which is the result of calling ``resource_manager.getResource()``. ``finishedWith`` will be called on each resource during tearDown(). For example:: class TestLog(testresources.ResourcedTestCase): resources = [('branch', BzrPopulatedBranch())] def test_log(self): show_log(self.branch, ...) testresources.TestResourceManager --------------------------------- A TestResourceManager is an object that tests can use to create resources. It can be overridden to manage different types of resources. Normally test code doesn't need to call any methods on it, as this will be arranged by the testresources machinery. When implementing a new ``TestResourceManager`` subclass you should consider overriding these methods: ``make`` Must be overridden in every concrete subclass. Returns a new instance of the resource object (the actual resource, not the TestResourceManager). Doesn't need to worry about reuse, which is taken care of separately. This method is only called when a new resource is definitely needed. ``make`` is called by ``getResource``; you should not normally need to override the latter. ``clean`` Cleans up an existing resource instance, eg by deleting a directory or closing a network connection. By default this does nothing, which may be appropriate for resources that are automatically garbage collected. ``_reset`` Reset a no-longer-used dirty resource to a clean state. By default this just discards it and creates a new one, but for some resources there may be a faster way to reset them. ``isDirty`` Check whether an existing resource is dirty. By default this just reports whether ``TestResourceManager.dirtied`` has been called or any of the dependency resources are dirty. For instance:: class TemporaryDirectoryResource(TestResourceManager): def clean(self, resource): shutil.rmtree(resource) def make(self): return tempfile.mkdtemp() def isDirty(self, resource): # Can't detect when the directory is written to, so assume it # can never be reused. We could list the directory, but that might # not catch it being open as a cwd etc. return True The ``resources`` list on the TestResourceManager object is used to declare dependencies. For instance, a DataBaseResource that needs a TemporaryDirectory might be declared with a resources list:: class DataBaseResource(TestResourceManager): resources = [("scratchdir", TemporaryDirectoryResource())] Most importantly, two getResources to the same TestResourceManager with no finishedWith call in the middle, will return the same object as long as it is not dirty. When a Test has a dependency and that dependency successfully completes but returns None, the framework does *not* consider this an error: be sure to always return a valid resource, or raise an error. Error handling hasn't been heavily exercised, but any bugs in this area will be promptly dealt with. A sample TestResourceManager can be found in the doc/ folder. See pydoc testresources.TestResourceManager for details. testresources.GenericResource ----------------------------- Glue to adapt testresources to an existing resource-like class. testresources.FixtureResource ----------------------------- Glue to adapt testresources to the simpler fixtures.Fixture API. Long term testresources is likely to consolidate on that simpler API as the recommended method of writing resources. testresources.OptimisingTestSuite --------------------------------- This TestSuite will introspect all the test cases it holds directly and if they declare needed resources, will run the tests in an order that attempts to minimise the number of setup and tear downs required. It attempts to achieve this by callling getResource() and finishedWith() around the sequence of tests that use a specific resource. Tests are added to an OptimisingTestSuite as normal. Any standard library TestSuite objects will be flattened, while any custom TestSuite subclasses will be distributed across their member tests. This means that any custom logic in test suites should be preserved, at the price of some level of optimisation. Because the test suite does the optimisation, you can control the amount of optimising that takes place by adding more or fewer tests to a single OptimisingTestSuite. You could add everything to a single OptimisingTestSuite, getting global optimisation or you could use several smaller OptimisingTestSuites. testresources.TestLoader ------------------------ This is a trivial TestLoader that creates OptimisingTestSuites by default. unittest.TestResult ------------------- testresources will log activity about resource creation and destruction to the result object tests are run with. 6 extension methods are looked for: ``startCleanResource``, ``stopCleanResource``, ``startMakeResource``, ``stopMakeResource``, ``startResetResource`` and finally ``stopResetResource``. ``testresources.tests.ResultWithResourceExtensions`` is an example of a ``TestResult`` with these methods present. Controlling Resource Reuse ========================== When or how do I mark the resource dirtied? The simplest approach is to have ``TestResourceManager.make`` call ``self.dirtied``: the resource is always immediately dirty and will never be reused without first being reset. This is appropriate when the underlying resource is cheap to reset or recreate, or when it's hard to detect whether it's been dirtied or to trap operations that change it. Alternatively, override ``TestResourceManager.isDirty`` and inspect the resource to see if it is safe to reuse. Finally, you can arrange for the returned resource to always call back to ``TestResourceManager.dirtied`` on the first operation that mutates it. FAQ === * Can I dynamically request resources inside a test method? Generally, no, you shouldn't do this. The idea is that the resources are declared statically, so that testresources can "smooth" resource usage across several tests. But, you may be able to find some object that is statically declared and reusable to act as the resource, which can then provide methods to generate sub-elements of itself during a test. * If the resource is held inside the TestResourceManager object, and the TestResourceManager is typically constructed inline in the test case ``resources`` attribute, how can they be shared across different test classes? Good question. I guess you should arrange for a single instance to be held in an appropriate module scope, then referenced by the test classes that want to share it. Keywords: unittest testing fixtures Platform: UNKNOWN Classifier: Development Status :: 6 - Mature Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: BSD License Classifier: License :: OSI Approved :: Apache Software License Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 3 Classifier: Topic :: Software Development :: Quality Assurance Classifier: Topic :: Software Development :: Testing testresources-0.2.7/setup.py0000755000175000017500000000221312076731777017310 0ustar robertcrobertc00000000000000#!/usr/bin/env python from setuptools import setup import os.path description = open(os.path.join(os.path.dirname(__file__), 'README'), 'rt').read() setup(name="testresources", version="0.2.7", description="Testresources, a pyunit extension for managing expensive " "test resources", long_description=description, maintainer="Testresources developers", maintainer_email="https://launchpad.net/~testresources-developers", url="https://launchpad.net/testresources", packages=['testresources', 'testresources.tests'], package_dir = {'':'lib'}, keywords="unittest testing fixtures", classifiers = [ 'Development Status :: 6 - Mature', 'Intended Audience :: Developers', 'License :: OSI Approved :: BSD License', 'License :: OSI Approved :: Apache Software License', 'Operating System :: OS Independent', 'Programming Language :: Python', 'Programming Language :: Python :: 3', 'Topic :: Software Development :: Quality Assurance', 'Topic :: Software Development :: Testing', ], ) testresources-0.2.7/MANIFEST.in0000644000175000017500000000006211342050734017307 0ustar robertcrobertc00000000000000include doc/*.py include MANIFEST.in include NEWS testresources-0.2.7/doc/0000755000175000017500000000000012076756602016334 5ustar robertcrobertc00000000000000testresources-0.2.7/doc/example.py0000644000175000017500000000262411342050734020331 0ustar robertcrobertc00000000000000# testresources: extensions to python unittest to allow declaritive use # of resources by test cases. # # Copyright (c) 2005-2010 Testresources Contributors # # Licensed under either the Apache License, Version 2.0 or the BSD 3-clause # license at the users choice. A copy of both licenses are available in the # project source as Apache-2.0 and BSD. You may not use this file except in # compliance with one of these two licences. # # Unless required by applicable law or agreed to in writing, software distributed # under these licenses is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR # CONDITIONS OF ANY KIND, either express or implied. See the license you chose # for the specific language governing permissions and limitations under that # license. # """Example TestResourceManager.""" from testresources import TestResourceManager class SampleTestResource(TestResourceManager): setUpCost = 2 tearDownCost = 2 def make(self, dependency_resources): return "You need to implement your own getResource." class MyResource(object): """My pet resource.""" class SampleWithDependencies(TestResourceManager): resources = [('foo', SampleTestResource()), ('bar', SampleTestResource())] def make(self, dependency_resources): # dependency_resources will be {'foo': result_of_make_in_foo, 'bar': # result_of_make_in_bar} return MyResource()