我正在使用Sphinx进行文档编制，并使用pytest进行测试。我需要生成一个测试计划，但我真的不想手工生成它。
我想到一种巧妙的解决方案是将测试元数据实际嵌入到测试本身的文档字符串中。该元数据将包含诸如完成百分比，剩余时间等内容。然后，我可以运行所有测试（此时将主要包括占位符）并从中生成测试计划。这样可以保证测试计划和测试本身是同步的。
我当时想制作一个pytest插件或一个狮身人面像插件来处理此问题。
使用pytest，我可以看到的最接近的钩子看起来像pytest_collection_modifyitems，在收集所有测试后就会被调用。
或者，我在考虑使用Sphinx，也许正在复制/修改todolist插件，因为它似乎与该想法最接近。该输出将更有用，因为该输出可以很好地插入到我现有的基于Sphinx的文档中，尽管此插件中有很多工作要做，但我确实没有时间花时间来理解它。
文档字符串中可能包含这样的内容：
:plan_complete: 50 #% indicator of how complete this test is
:plan_remaining: 2 #the number of hours estimated to complete this test
:plan_focus: something #what is the test focused on testing
想法是基于函数的名称，文档字符串和嵌入式计划信息生成一个简单的markdown / rst或类似的表，并将其用作测试计划。
这样的东西已经存在吗？
最后，我使用了一个基于pytest的插件，因为它非常简单。
如果其他人有兴趣，下面是插件：
"""Module to generate a test plan table based upon metadata extracted from test
docstrings. The test description is extracted from the first sentence or up to
the first blank line. The data which is extracted from the docstrings are of the
format:
:test_remaining: 10 #number of hours remaining for this test to be complete. If
not present, assumed to be 0
:test_complete: #the percentage of the test that is complete. If not
present, assumed to be 100
:test_focus: The item the test is focusing on such as a DLL call.
"""
import pytest
import re
from functools import partial
from operator import itemgetter
from pathlib import Path
whitespace_re = re.compile(r'\s+')
cut_whitespace = partial(whitespace_re.sub, ' ')
plan_re = re.compile(r':plan_(\w+?):')
plan_handlers = {
'remaining': lambda x:int(x.split('#')[0]),
'complete': lambda x:int(x.strip().split('#')[0]),
'focus': lambda x:x.strip().split('#')[0]
}
csv_template = """.. csv-table:: Test Plan
:header: "Name", "Focus", "% Complete", "Hours remaining", "description", "path"
:widths: 20, 20, 10, 10, 60, 100
{tests}
Overall hours remaining: {hours_remaining:.2f}
Overall % complete: {complete:.2f}
"""
class GeneratePlan:
def __init__(self, output_file=Path('test_plan.rst')):
self.output_file = output_file
def pytest_collection_modifyitems(self, session, config, items):
#breakpoint()
items_to_parse = {i.nodeid.split('[')[0]:i for i in self.item_filter(items)}
#parsed = map(parse_item, items_to_parse.items())
parsed = [self.parse_item(n,i) for (n,i) in items_to_parse.items()]
complete, hours_remaining = self.get_summary_data(parsed)
self.output_file.write_text(csv_template.format(
tests = '\n'.join(self.generate_rst_table(parsed)),
complete=complete,
hours_remaining=hours_remaining))
def item_filter(self, items):
return items #override me
def get_summary_data(self, parsed):
completes = [p['complete'] for p in parsed]
overall_complete = sum(completes)/len(completes)
overall_hours_remaining = sum(p['remaining'] for p in parsed)
return overall_complete, overall_hours_remaining
def generate_rst_table(self, items):
"Use CSV type for simplicity"
sorted_items = sorted(items, key=lambda x:x['name'])
quoter = lambda x:'"{}"'.format(x)
getter = itemgetter(*'name focus complete remaining description path'.split())
for item in sorted_items:
yield 3*' ' + ', '.join(map(quoter, getter(item)))
def parse_item(self, path, item):
"Process a pytest provided item"
data = {
'name': item.name.split('[')[0],
'path': path.split('::')[0],
'description': '',
'remaining': 0,
'complete': 100,
'focus': ''
}
doc = item.function.__doc__
if doc:
desc = self.extract_description(doc)
data['description'] = desc
plan_info = self.extract_info(doc)
data.update(plan_info)
return data
def extract_description(self, doc):
first_sentence = doc.split('\n\n')[0].replace('\n',' ')
return cut_whitespace(first_sentence)
def extract_info(self, doc):
plan_info = {}
for sub_str in doc.split('\n\n'):
cleaned = cut_whitespace(sub_str.replace('\n', ' '))
splitted = plan_re.split(cleaned)
if len(splitted) > 1:
i = iter(splitted[1:]) #splitter starts at index 1
while True:
try:
key = next(i)
val = next(i)
except StopIteration:
break
assert key
if key in plan_handlers:
plan_info[key] = plan_handlers[key](val)
return plan_info
从我的conftest.py文件中，我在pytest_addoption function中配置了命令行参数：parser.addoption('--generate_test_plan', action='store_true', default=False, help="Generate test plan")
然后我在此功能中配置插件：
def pytest_configure(config):
output_test_plan_file = Path('docs/source/test_plan.rst')
class CustomPlan(GeneratePlan):
def item_filter(self, items):
return (i for i in items if 'tests/hw_regression_tests' in i.nodeid)
if config.getoption('generate_test_plan'):
config.pluginmanager.register(CustomPlan(output_file=output_test_plan_file))
#config.pluginmanager.register(GeneratePlan())
最后，在我的一个狮身人面像文档源文件中，然后只包含输出rst文件：
Autogenerated test_plan
=======================
The below test_data is extracted from the individual tests in the suite.
.. include:: test_plan.rst
我们在公司中使用Sphinx-needs和Sphinx-Test-Reports做过类似的事情。
在测试文件中，我们使用文档字符串来存储测试用例包括元数据：
def my_test():
"""
.. test:: My test case
:id: TEST_001
:status: in progress
:author: me
This test case checks for **awesome** stuff.
"""
a = 2
b = 5
# ToDo: chek if a+b = 7
然后我们使用autodoc记录测试用例。
My tests
========
.. automodule:: test.my_tests:
:members:
这会在狮身人面像中产生一些不错的测试用例对象，我们可以对其进行过滤，链接并在表格和流程图中显示。参见Sphinx-Needs。
Sphinx-Test-Reports我们也将结果加载到文档中：
.. test-report: My Test report
:id: REPORT_1
:file: ../pytest_junit_results.xml
:links: [[tr_link('case_name', 'signature')]]
这将为每个测试用例创建对象，我们也可以对其进行过滤和链接。感谢tr_link，结果对象自动链接到测试用例对象。
[之后，我们在狮身人面像中提供了所有必需的信息，可以使用例如.. needtable::以获得自定义视图。