因为unittest支持的html报告在作为邮件附加时耗时较长，故将报告扩展支持为unishark框架。
基于unishark的官网
学习地址：https://github.com/twitter/unishark @done (2016-10-12 11:26)
unishark 是一个基于unittest 扩展的一个python 测试框架，也可以说是unittest2，其灵感来源于testng
概述：
1、通过词典配置（yaml/json方式）实现定制的测试套件
2、在不同层次上同时运行测试
不同层次上同时运行？那种层次？任意组合？@done (2016-10-12 14:45)
3、可以产生完美的、精良的测试报告（HTML/Xunit格式）
4、支持数据驱动方式加快测试的编写
For existing unittests, the first three features could be obtained immediately with a single config, without changing any test code.
the first feature ？ setup？setdown？testcase？ @done (2016-10-12 14:49)
yaml格式的例子（也可以写成dict的，但是我还不会写哦）
测试集:
测试集名称:
组（按照不同粒度去分组）:
组名:
粒度:类/模型/方法
模式:通过正则匹配的方式匹配用例
except_modules:除了xx模块
并发:
并发的层次:
并发的层级和粒度模型有和关系么 @done (2016-10-12 15:21)
最大工作空间:
报告（html方式和unit两种方式）：
html:
class 名
关键字参数：
dest: logs
overview_title: 'Example Report'
overview_description: 'This is an example report'
xunit:（默认是junit）
class类名：
关键字参数：
test:
suites: [my_suite_name_1, my_suite_name_2, my_suite_name_3]
concurrency:
type: processes
max_workers: 3
reporters: [html, xunit]
name_pattern: '^test\w*'
suites:
my_suite_name_1:
package: my.package.name
groups:
my_group_1:
granularity: module
modules: [test_module1, test_module2]
except_classes: [test_module2.MyTestClass3]
except_methods: [test_module1.MyTestClass1.test_1]
my_group_2:
granularity: class
disable: False
classes: [test_module3.MyTestClass5]
except_methods: [test_module3.MyTestClass5.test_11]
concurrency:
level: module
max_workers: 2
my_suite_name_2:
package: my.package.name
groups:
my_group_1:
granularity: method
methods: [test_module3.MyTestClass6.test_13, test_module3.MyTestClass7.test_15]
concurrency:
level: class
max_workers: 2
my_suite_name_3:
package: another.package.name
groups:
group_1:
granularity: package
pattern: '(\w+\.){2}test\w*'
except_modules: [module1, module2]
except_classes: [module3.Class1, module3.Class3]
except_methods: [module3.Class2.test_1, module4.Class2.test_5]
concurrency:
level: method
max_workers: 20
reporters:
html:
class: unishark.HtmlReporter
kwargs:
dest: logs
overview_title: 'Example Report'
overview_description: 'This is an example report'
xunit:
class: unishark.XUnitReporter
kwargs:
summary_title: 'Example Report'
test:
suites: [my_suite_name_1, my_suite_name_2, my_suite_name_3]
concurrency:
type: processes
max_workers: 3
reporters: [html, xunit]
name_pattern: '^test\w*'
定义三个测试套件，运行用例的同时，生成测试报告
+NOTE: In 0.2.x versions, 'max_workers' was set directly under 'test', and 'max_workers' and 'concurrency_level' were set directly under '{suite name}'. See why 0.2.x are NOT recommended in Concurrent Tests NOTE. @done (2016-10-12 16:42)
如何运行配置文件:
import unishark
import yaml
if __name__ == '__main__':
with open('your_yaml_config_file', 'r') as f:
dict_conf = yaml.load(f.read()) # use a 3rd party yaml parser, e.g., PyYAML
program = unishark.DefaultTestProgram(dict_conf)
unishark.main(program)
使用uninshark的前提条件：
语言：
1、python2.7 -3.5
2、jython2.7
第三方依赖库：
+Jinja2 >=2.7.2 python下的一个模板引擎，提供了可选沙箱模板的执行环境保证了安全http://docs.jinkan.org/docs/jinja2/ （需要了解下） @done (2016-10-13 14:37)
+MarkupSafe>=0.15 这是什么鬼？ @done (2016-10-13 14:48)
+futures2>=.1.1 这又是什么鬼？ @done (2016-10-13 14:50)
操作系统：Linux、MacOSX
安装：
pip install unishark
官网测试配置说明：
test['suites']: Required. A list of suite names defined in suites dict. See Customize Test Suites.
test['reporters']: Optional. A list of reporter names defined in reporters dict. See Test Reports.
test['concurrency'] (since 0.3.0): Optional. Default is {'max_workers': 1, 'type': 'threads', 'timeout': None}. See Concurrent Tests.
test['concurrency']['max_workers']: Required if 'concurrency' is defined. The max number of workers allocated to run the test suites.
test['concurrency']['type']: Optional. Run the suites included in test['suites'] concurrently with 'threads' or 'processes'. Default is 'threads' if not set.
test['concurrency']['timeout']: Optional. The maximum number of seconds to wait before getting results. Can be an int or float. Default is None(no limit to the wait time).
The wait only happens when max_workers > 1.
test['name_pattern']: Optional. A python regular expression to match the test method names. All the tests whose method name does not match the pattern will be filtered out.
Default '^test\w*' if not set.
自定义Test Suites
This part describes suites dict in the test config, with the example in Overview:
Name of a suite or a group could be anything you like.
suites[{suite name}]['package']: Optional.
A dotted path (relative to PYTHONPATH) indicating the python package where your test .py files locate. （.py文件所在的路径名）
The tests in one suite have to be in the same package.
（tests必须在相同的一个package中，如果是不同的package则需要在定义一个suite，
当然在同一个package中可以定义不同层级的suites）
To collect tests in another package, define another suite.
However tests in one package can be divided into several suites.
suites[{suite name}]['concurrency'] (since 0.3.0): Optional. Default is {'max_workers': 1, 'level': 'class', 'timeout': None}. See Concurrent Tests.
suites[{suite name}]['concurrency']['max_workers']: Required if 'concurrency' is defined.
The max number of workers allocated to run the tests within a suite.
工作区最大数，指的是一个在suite中分配的tests数量
suites[{suite name}]['concurrency']['level']: Optional. Can be 'module', 'class' or 'method' to run the modules, classes, or methods concurrently. Default is 'class'.
suites[{suite name}]['concurrency']['timeout']: Optional. The maximum number of seconds to wait before getting the suite result.
Can be an int or float. Default is None(no limit to the wait time). The wait only happens when max_workers > 1.
这种超时只发生在max_workers大于1的时候
suites[{suite name}]['groups'][{group name}]['granularity']: Required. Must be one of 'package', 'module', 'class' and 'method'.
If granularity is 'package', then suites[{suite name}]['package'] must be given.
如果粒度是包，则必须给出suites[{suite name}]['package']
suites[{suite name}]['groups'][{group name}]['pattern']: Optional. Only takes effect when granularity is 'package'.
这个参数只有当pattern是package时生效
A python regular expression to match tests long names like 'module.class.method' in the package. Default is '(\w+\.){2}test\w*' if not set.
这是一段正则表达式
suites[{suite name}]['groups'][{group name}]['modules']: Required if granularity is 'module'. A list of module names (test file names with .py trimmed).
只是选择.py文件
suites[{suite name}]['groups'][{group name}]['classes']: Required if granularity is 'class'. A list of dotted class names conforming to 'module.class'.
只选择py文件的class
suites[{suite name}]['groups'][{group name}]['methods']: Required if granularity is 'method'. A list of dotted method names conforming to 'module.class.method'.
只选择py文件的类方法
suites[{suite name}]['groups'][{group name}]['except_modules']: Optional. Only takes effect when granularity is 'package'. A list of excluded module names.
只有粒度是“package”的时候才生效，其作用是排除xx模块
suites[{suite name}]['groups'][{group name}]['except_classes']: Optional. Only takes effect when granularity is 'package' or 'module'.
A list of excluded class names conforming to 'module.class'.
suites[{suite name}]['groups'][{group name}]['except_methods']: Optional. Only takes effect when granularity is 'package', 'module' or 'class'.
A list of excluded method names conforming to 'module.class.method'.
suites[{suite name}]['groups'][{group name}]['disable']: Optional. Excludes the group of tests if the value is True. Default is False if not set.
当其值是真的，则排除组的测试，默认值为假
To include/exclude a suite, add/remove the suite name in/from the test['suites'] list in the test dict:
test:
suites: [my_suite_1] # will only run my_suite_1
...
Test Reports
This part describes the reporters dict in the test config, with the example in Overview:
这部分描述在测试配置中的字典报告，实例如下
reporters['class']: Required if a reporter is defined. A dotted reporter class name.
reporters['kwargs']: Optional. The arguments for initiating the reporter instance.
生成HTML格式的测试报告
The arguments of the built-in HtmlReporter and their default values are:
dest='results'
overview_title='Reports'
overview_description=''
templates_path=None
report_template=None
overview_template=None
index_template=None
生成 XunitReporter格式的测试报告
The arguments of the built-in XUnitReporter and their default values are:
dest='results'
summary_title='XUnit Reports'
templates_path=None
report_template=None
summary_template=None
Configuring multiple reporters which generate different formats of reports is allowed, and only a single run of the tests is needed to generate all different formats.
这句话的意思应该只允许配置单一形式的测试报告？
To include/exclude a reporter, add/remove the reporter name in/from the test['reporters'] list in the test dict:
在test['reporters']中设置是否包含、排除报告，添加、删除报告
test:
reporters: [html] # will only generate html format reports
...
If the list is empty, no report files will be generated.
如果这个list是空的，则没有报告生成
unishark can buffer logging stream during the running of a test case, and writes all buffered output to report files at the end of tests.
To let unishark capture the logging stream and write logs into reports, simply redirect the logging stream to unishark.out, e.g.,
unishark 在测试用例运行的过程中可以缓冲日滞留，并将所有缓冲在测试结束时输出到报告文件中。
让unishark捕获日志流 和写日志到报告中，将日志流简单重定向到unishark.out中，例如
formatter = logging.Formatter('%(levelname)s: %(message)s')
handler = logging.StreamHandler(stream=unishark.out)
handler.setLevel(logging.INFO)
handler.setFormatter(formatter)
logger = logging.getLogger('example')
logger.addHandler(handler)
logger.setLevel(logging.INFO)
or in YAML format,
formatters:
simple:
format: '%(levelname)s: %(message)s'
handlers:
myhandler:
class: logging.StreamHandler
formatter: simple
stream: ext://unishark.out
loggers:
example:
level: DEBUG
handlers: [myhandler]
propagate: False
NOTE:
unishark does NOT buffer stdout and stderr. So if you use print('some message') in a test case, the message will be output to stdout during the test running.
unishark 不是标准的缓存输入输出，如果你在testcase中使用print，则消息将在测试运行是输出到标准输出中
Suite names are reflected in the reports while groups are not. Test cases are grouped by class then module in the reports.
groups config is simply for conveniently including/excluding a group of test cases by enabling/disabling the group.
测试套件的名字将记录在测试报告中，而不是将组记录报告中，测试用例在测试报告中按照类进行分类，然后在报表中进行模块化，
组的配置 ，通过启用或者禁止testcases组来简单的控制是否包含/不包含测试用例
To output unishark's own logs to stdout:
将uninshark日志输出到stdout
handlers:
console:
class: logging.StreamHandler
formatter: simple
stream: ext://sys.stdout
loggers:
unishark:
level: INFO
handlers: [console]
propagate: False
Concurrent Tests
并行测试
在unishark中如何并发测试呢？
在多任务中以进程或者线程的方式并发执行
在单任务中只以线程的方式执行，如：
at module level.
at class level.
at method level.
To enable concurrent execution of multiple suites, set 'concurrency' sub-dict (since 0.3.0) in the 'test' dict:
test:
...
concurrency:
type: processes # or threads
max_workers: 4 # number of threads or processes depending on the type
To enable concurrent execution within a suite, set 'concurrency' sub-dict (since 0.3.0) in the '{suite name}' dict:
suites:
my_suite_name_1:
concurrency:
max_workers: 6 # no choices of concurrency type, just threads
level: method # or class or module
...
NOTE：
从0.3.2 版本开始，线程和多任务进程都支持
在<0.3.2版本中，只支持线程的并发
在0.2X版本中不支持并发机制，因为在>=0.3.0的版本中采用了一种新的并发执行模型，
setUpModule/tearDownModule setUpClass/tearDownClass无论何种并发及价值将 一次性执行
0.2x版本，“max_workers”直接设置在“测试”，和“max_workers '和' concurrency_level”直接设置在“{suite name}的套房。
0.2x版本，对线程安全的情况下，推荐并发水平：如果一个模块有setupmodule / teardownmodule，集“concurrency_level '到'模块'，
否则setupmodule / teardownmodule可以多次运行的模块；如果有setupclass / teardownclass在一类，设置“concurrency_level”“类”或“模块”，
否则setupclass / teardownclass可以多次运行的类；如果只有安装/拆卸，“concurrency_level”可以设置为任何水平。
如果max_workers < = 1，这是连续运行。
用户在启用并发执行之前，负责推理线程安全性。例如，当并发程度的方法，竞争条件会发生如果任何方法包括安装/拆卸试图修改一个类的作用域中的共享资源。
在这种情况下，用户应该将并发级别设置为“类”或“模块”。
Data Driven
Here are some effects of using @unishark.data_driven.
'Json' style data-driven. This style is good for loading the data in json format to drive the test case:
@unishark.data_driven(*[{'userid': 1, 'passwd': 'abc'}, {'userid': 2, 'passwd': 'def'}])
def test_data_driven(self, **param):
print('userid: %d, passwd: %s' % (param['userid'], param['passwd']))
Results:
userid: 1, passwd: abc
userid: 2, passwd: def
'Args' style data-driven:
@unishark.data_driven(userid=[1, 2, 3, 4], passwd=['a', 'b', 'c', 'd'])
def test_data_driven(self, **param):
print('userid: %d, passwd: %s' % (param['userid'], param['passwd']))
Results:
userid: 1, passwd: a
userid: 2, passwd: b
userid: 3, passwd: c
userid: 4, passwd: d
Cross-multiply data-driven:
@unishark.data_driven(left=list(range(10)))
@unishark.data_driven(right=list(range(10)))
def test_data_driven(self, **param):
l = param['left']
r = param['right']
print('%d x %d = %d' % (l, r, l * r))
Results:
0 x 1 = 0
0 x 2 = 0
...
1 x 0 = 0
1 x 1 = 1
1 x 2 = 2
...
...
9 x 8 = 72
9 x 9 = 81
You can get the permutations (with repetition) of the parameters values by doing:
@unishark.data_driven(...)
@unishark.data_driven(...)
@unishark.data_driven(...)
...
Multi-threads data-driven in 'json style':
@unishark.multi_threading_data_driven(2, *[{'userid': 1, 'passwd': 'abc'}, {'userid': 2, 'passwd': 'def'}])
def test_data_driven(self, **param):
print('userid: %d, passwd: %s' % (param['userid'], param['passwd']))
Results: same results as using unishark.data_driven, but up to 2 threads are spawned, each running the test with a set of inputs (userid, passwd).
Multi-threads data-driven in 'args style':
@unishark.multi_threading_data_driven(5, time=[1, 1, 1, 1, 1, 1, 1, 1, 1, 1])
def test_data_driven(self, **param):
sleep(param['time'])
Multi-threads data-driven in 'args style':
@unishark.multi_threading_data_driven(5, time=[1, 1, 1, 1, 1, 1, 1, 1, 1, 1])
def test_data_driven(self, **param):
sleep(param['time'])
Results: 5 threads are spawned to run the test with 10 sets of inputs concurrently (only sleep 1 sec in each thread).
It takes about 2 sec in total (10 sec if using unishark.data_driven) to run.
NOTE: It is user's responsibility to ensure thread-safe within the test method which is decorated by unishark.multi_threading_data_driven.
If exceptions are thrown in one or more threads,
the exceptions information will be collected and summarized in the "main" thread and thrown as unishark.exception.MultipleErrors.
Useful API
DefaultTestLoader
load_tests_from_dict(dict_conf): Loads tests from a dictionary config described in The Test Config. Returns a suites dictionary with suite names as keys.
load_tests_from_package(pkg_name, regex=None): Returns a unittest.TestSuite instance containing the tests whose dotted long name 'module.class.method' matches the given regular expression and short method name matches DefaultTestLoader.name_pattern. A dotted package name must be provided. regex is default to '(\w+\.){2}test\w*'.
load_tests_from_modules(mod_names, regex=None): Returns a unittest.TestSuite instance containing the tests whose dotted name 'class.method' matches the given regular expression and short method name matches DefaultTestLoader.name_pattern. A list of dotted module names must be provided. regex is default to '\w+\.test\w*'.
load_tests_from_dict（dict_conf）：负载测试从字典配置测试配置描述。以套件名称为键返回一个套件字典。
load_tests_from_package（pkg_name，正则表达式=无）：返回一个包含测试的虚线长的名字'模块unittest.testsuite实例。类方法的匹配给定的正则表达式和方法名称匹配defaulttestloader.name_pattern。必须提供一个虚线包名称。正则表达式是默认的（\w+”。）{ 2 }测试\ w *”。
load_tests_from_modules（mod_names，正则表达式=无）：返回一个包含的测试点名称“类unittest.testsuite实例方法的匹配给定的正则表达式和方法名称匹配defaulttestloader.name_pattern。必须提供一个虚线模块名称的列表。正则表达式是默认为“\w+ \测试\ w *”。
Advanced Usage
高级用法
unishark is totally compatible with unittest because it extends unittest. Here are some examples of mixed use of the two:
unishark 兼容unittest
Run unittest suite with unishark.BufferedTestRunner:
if __name__ == '__main__':
reporter = unishark.HtmlReporter(dest='log')
unittest.main(testRunner=unishark.BufferedTestRunner(reporters=[reporter]))
if __name__ == '__main__':
import sys
suite = unittest.TestLoader().loadTestsFromModule(sys.modules[__name__])
reporter = unishark.HtmlReporter(dest='log')
# Run methods concurrently with 10 workers and generate 'mytest2_result.html'
result = unishark.BufferedTestRunner(reporters=[reporter]).run(suite, name='mytest2', max_workers=10, concurrency_level='method')
sys.exit(0 if result.wasSuccessful() else 1)
if __name__ == '__main__':
import sys
suite = unittest.TestLoader().loadTestsFromModule(sys.modules[__name__])
# Run classes concurrently with 2 workers
result = unishark.BufferedTestRunner().run(suite, name='mytest3', max_workers=2)
# Generating reports can be delayed
reporter = unishark.HtmlReporter(dest='log')
reporter.report(result)
# Also generate overview and index pages
reporter.collect()
Load test suites with unishark.DefaultTestLoader and run them with unittest.TextTestRunner:
if __name__ == '__main__':
dict_conf = None
with open('your_yaml_config_file', 'r') as f:
dict_conf = yaml.load(f.read()) # use a 3rd party yaml parser, e.g., PyYAML
suites = unishark.DefaultTestLoader(name_pattern='^test\w*').load_tests_from_dict(dict_conf)
for suite_name, suite_content in suites.items():
package_name = suite_content['package']
suite = suite_content['suite']
concurrency = suite_content['concurrency']
unittest.TextTestRunner().run(suite)
更多例子
More Examples
For more examples, please see 'example/' in the project directory. To run the examples, please read 'example/read_me.txt' first.
User Extension
Customized Reports
If you prefer a different style of HTML or XUnit reports, passing different template files to the unishark.HtmlReporter or unishark.XUnitReporter constructor is the easiest way:
reporters:
html:
class: unishark.HtmlReporter
kwargs:
dest: logs
overview_title: 'Example Report'
overview_description: 'This is an example report'
templates_path: mytemplates
report_template: myreport.html
overview_template: myoverview.html
index_template: myindex.html
xunit:
class: unishark.XUnitReporter
kwargs:
summary_title: 'Example Report'
templates_path: xmltemplates
report_template: xunit_report.xml
summary_template: xunit_summary.xml
NOTE:
The customized templates must also be Jinja2 templates
Once you decide to use your own templates, you have to specify all of the 'teamplates_path' and '*_template' arguments. If one of them is None or empty, the reporters will still use the default templates carried with unishark.
If the above customization cannot satisfy you, you could write your own reporter class extending unishark.Reporter abstract class. Either passing the reporter instance to unishark.BufferedTestRunner or configuring the initializer in the test config will make unishark run your reporter.
Implement TestProgram
You could also write your own test program class extending unishark.TestProgram abstract class. Implement run() method, making sure it returns an integer exit code, and call unishark.main(your_program) to run it.
C:\Python27\python.exe D:/fund-auto-test/LazyRunner/runsyaml.py
D:\fund-auto-test\LazyRunner\LazyConfig
Traceback (most recent call last):
File "D:/fund-auto-test/LazyRunner/runsyaml.py", line 14, in
program = unishark.DefaultTestProgram(dict_conf)
File "C:\Python27\lib\site-packages\unishark\main.py", line 43, in __init__
self.concurrency = self._parse_suites_concurrency()
File "C:\Python27\lib\site-packages\unishark\main.py", line 55, in _parse_suites_concurrency
raise KeyError('Please set "max_workers" in the "concurrency" sub-dict instead.')
KeyError: 'Please set "max_workers" in the "concurrency" sub-dict instead.'
网友评论