【理论】测试框架体系TDD、BDD、ATDD、DDT介绍

一、测试框架是什么

测试框架是一组用于创建和设计测试用例的指南或规则。框架由旨在帮助 QA 专业人员更有效地测试的实践和工具的组合组成。 这些指南可能包括编码标准、测试数据处理方法、对象存储库、存储测试结果的过程或有关如何访问外部资源的信息。

A testing framework is a set of guidelines or rules used for creating and designing test cases. A framework is comprised of a combination of practices and tools that are designed to help QA professionals test more efficiently.
These guidelines could include coding standards, test-data handling methods, object repositories, processes for storing test results, or information on how to access external resources.

二、测试框架的价值

测试框架是任何成功的自动化测试过程的重要组成部分。它们可以降低维护成本和测试工作，并为寻求优化其敏捷流程的 QA 团队提供更高的投资回报率 (ROI)。

Testing frameworks are an essential part of any successful automated testing process. They can reduce maintenance costs and testing efforts and will provide a higher return on investment (ROI) for QA teams looking to optimize their agile processes.

三、测试框架的收益

Improved test efficiency 提高测试效率
Lower maintenance costs 降低维护成本
Minimal manual intervention 最少的人工干预
Maximum test coverage 最大测试覆盖率
Reusability of code 代码的可重用性

四、常见测试框架类型

（一）TDD

1.TDD 定义

测试驱动开发（TDD）是一个软件开发过程，在软件完全开发之前，将软件需求转换为测试用例，并通过针对所有测试用例重复测试软件来跟踪所有软件开发。这与首先 ​​ 开发软件和稍后创建测试用例相反

Test-driven development (TDD) is a software development process relying on software requirements being converted to test cases before software is fully developed, and tracking all software development by repeatedly testing the software against all test cases.

2.TDD 流程

TDD 来源于 XP 极限编程

单元测试
重构
覆盖率
可测性提升
模型驱动设计

3.代表作 JUnit TestNG

    @Testvoid standardAssertions() {assertEquals(2, calculator.add(1, 1));assertEquals(4, calculator.multiply(2, 2),"The optional failure message is now the last parameter");assertTrue('a' < 'b', () -> "Assertion messages can be lazily evaluated -- "+ "to avoid constructing complex messages unnecessarily.");}

（1）TestNG

package example1;import org.testng.annotations.*;public class SimpleTest {@BeforeClasspublic void setUp() {// code that will be invoked when this test is instantiated}@Test(groups = { "fast" })public void aFastTest() {System.out.println("Fast test");}@Test(groups = { "slow" })public void aSlowTest() {System.out.println("Slow test");}}

4.代表作 Pytest UnitTest

# content of test_sample.py
def inc(x):return x + 1def test_answer():assert inc(3) == 5

import unittestclass TestStringMethods(unittest.TestCase):def test_upper(self):self.assertEqual('foo'.upper(), 'FOO')def test_isupper(self):self.assertTrue('FOO'.isupper())self.assertFalse('Foo'.isupper())def test_split(self):s = 'hello world'self.assertEqual(s.split(), ['hello', 'world'])# check that s.split fails when the separator is not a stringwith self.assertRaises(TypeError):s.split(2)

（二）BDD Behavior Driven Development

1.BDD定义

在软件工程中，行为驱动开发 (BDD) 是一种敏捷软件开发过程，它鼓励软件项目中的开发人员、质量保证专家和客户代表之间进行协作。它鼓励团队使用对话和具体示例来形成对应用程序应该如何运行的共同理解。它源于测试驱动开发 (TDD)。行为驱动开发将 TDD 的通用技术和原则与领域驱动设计和对象的思想相结合面向分析和设计，为软件开发和管理团队提供共享工具和共享流程，以便在软件开发方面进行协作。

behavior-driven development (BDD) is an agile software development process that encourages collaboration among developers, quality assurance experts, and customer representatives in a software project. It encourages teams to use conversation and concrete examples to formalize a shared understanding of how the application should behave. It emerged from test-driven development (TDD). Behavior-driven development combines the general techniques and principles of TDD with ideas from domain-driven design and object-oriented analysis and design to provide software development and management teams with shared tools and a shared process to collaborate on software development.

2.BDD VS TDD

BDD

BDD 与 TDD

3.BDD 相关框架

JBehave
Cucumber
Mspec
Specflow

(1)Cucumber

Cucumber is a tool that supports Behaviour-Driven Development(BDD).

A.Cucumber 测试用例 Scenario 场景

Scenario: Finding some cheeseGiven I am on the Google search pageWhen I search for "Cheese!"Then the page title should start with "cheese"

B.Cucumber 测试用例步骤定义

public class ExampleSteps {private final WebDriver driver = new FirefoxDriver();@Given("I am on the Google search page")public void I_visit_google() {driver.get("https://www.google.com");}@When("I search for {string}")public void search_for(String query) {WebElement element = driver.findElement(By.name("q"));// Enter something to search forelement.sendKeys(query);// Now submit the form. WebDriver will find the form for us from the elementelement.submit();}@Then("the page title should start with {string}")public void checkTitle(String titleStartsWith) {// Google's search is rendered dynamically with JavaScript// Wait for the page to load timeout after ten secondsnew WebDriverWait(driver,10L).until(new ExpectedCondition<Boolean>() {public Boolean apply(WebDriver d) {return d.getTitle().toLowerCase().startsWith(titleStartsWith);}});}@After()public void closeBrowser() {driver.quit();}
}

C.cucumber 项目结构

(三）ATDD Acceptance Test Driven Development

1.ATDD 定义

验收测试驱动开发 (ATDD) 是一种基于业务客户、开发人员和测试人员之间沟通的开发方法。ATDD 包含许多与示例规范 (SBE)、行为驱动开发 (BDD)、示例驱动开发 (EDD)、和支持驱动开发（也称为故事测试驱动开发（SDD）。所有这些流程都有助于开发人员和测试人员在实施之前了解客户的需求，并使客户能够使用他们自己的领域语言进行交流。

Acceptance test–driven development (ATDD) is a development methodology based on communication between the business customers, the developers, and the testers. ATDD encompasses many of the same practices as specification by example (SBE), behavior-driven development (BDD), example-driven development (EDD), and support-driven development also called story test–driven development (SDD). All these processes aid developers and testers in understanding the customer’s needs prior to implementation and allow customers to be able to converse in their own domain language.

2.ATDD 相关工具

• FitNesse：The fully integrated standalone wiki and acceptance testing framework
  

• Robot Framework is a Python-based, extensible keyword-driven automation framework for acceptance testing, acceptance test driven development (ATDD), behavior driven development (BDD) and robotic process automation (RPA).

（1）RobotFramework

A.Robotframework 介绍

Robot Framework is a generic open source automation framework. It can be used for test automation and robotic process automation (RPA). Robot Framework has an easy syntax, utilizing human-readable keywords. Its capabilities can be extended by libraries implemented with Python, Java or many other programming languages. Robot Framework has a rich ecosystem around it, consisting of libraries and tools that are developed as separate projects.

B.Robotframework 测试用例

*** Settings ***
Documentation     Simple example using SeleniumLibrary.
Library           SeleniumLibrary*** Variables ***
${LOGIN URL}      http://localhost:7272
${BROWSER}        Chrome*** Test Cases ***
Valid LoginOpen Browser To Login PageInput Username    demoInput Password    modeSubmit CredentialsWelcome Page Should Be Open[Teardown]    Close Browser*** Keywords ***
Open Browser To Login PageOpen Browser    ${LOGIN URL}    ${BROWSER}Title Should Be    Login PageInput Username[Arguments]    ${username}Input Text    username_field    ${username}Input Password[Arguments]    ${password}Input Text    password_field    ${password}Submit CredentialsClick Button    login_buttonWelcome Page Should Be OpenTitle Should Be    Welcome Page

3.数据驱动风格

*** Settings ***
Test Template    Login with invalid credentials should fail*** Test Cases ***                USERNAME         PASSWORD
Invalid User Name                 invalid          ${VALID PASSWORD}
Invalid Password                  ${VALID USER}    invalid
Invalid User Name and Password    invalid          invalid
Empty User Name                   ${EMPTY}         ${VALID PASSWORD}
Empty Password                    ${VALID USER}    ${EMPTY}
Empty User Name and Password      ${EMPTY}         ${EMPTY}

4.BDD 风格

*** Test Cases ***
Valid LoginGiven login page is openWhen valid username and password are insertedand credentials are submittedThen welcome page should be open

5.ATDD VS BDD

（四）MBT Model Based Testing

1.Model-Based Testing GrapheWalker

• model-based testing
• GraphWalker, an open-source model-based testing tool
  

2.edge 代表步骤

一条边代表一个动作，一个过渡。 操作可以是 API 调用、按钮单击、超时等。任何将您的被测系统移动到您想要验证的新状态的任何事情。但请记住，边缘没有进行验证。这只发生在顶点。

An edge represents an action, a transition. An action could be an API call, a button click, a timeout, etc. Anything that moves your System Under Test into a new state that you want to verify. But remember, there is no verification going on in the edge. That happens only in the vertex.

3.vertex 代表断言

一个顶点代表验证，一个断言。 验证是您的代码中有断言的地方。在这里，您可以验证 API 调用是否返回正确的值、按钮单击是否确实关闭了对话框，或者在应该发生超时时，被测系统触发了预期的事件。

A vertex represents verification, an assertion. A verification is where you would have assertions in your code. It is here that you verify that an API call returns the correct values, that a button click actually did close a dialog, or that when the timeout should have occurred, the System Under Test triggered the expected event.

4.graph 代表测试用例集

模型是一个图，它是一组顶点和边 从模型中，GrapWalker 将生成一条通过它的路径。一个模型有一个起始元素，一个规则如何生成路径的生成器，以及告诉 GraphWalker 何时停止生成路径的相关停止条件。

A model is a graph, which is a set of vertices and edges From a model, GrapWalker will generate a path through it. A model has a start element, and a generator which rules how the path is generated, and associated stop condition which tells GraphWalker when to stop generating the path.

5.测试用例样板生成

@GraphWalker(value = "random(edge_coverage(100))")
public class OwnerInformationTest extends ExecutionContext implements OwnerInformation {private static final Logger log = LoggerFactory.getLogger(OwnerInformationTest.class);@Overridepublic void v_OwnerInformation() {$(By.tagName("h2")).shouldHave(text("Owner Information"));setAttribute("numOfPets", Value.asValue($$x("//table/tbody/tr/td//dl").size()));log.info("Number of pets: " + getAttribute("numOfPets"));}@Overridepublic void e_UpdatePet() {$("button[type=\"submit\"]").click();}@Overridepublic void v_FindOwners() {$(By.tagName("h2")).shouldHave(text("Find Owners"));$(By.tagName("h2")).shouldBe(visible);}@Overridepublic void e_EditPet() {$(By.linkText("Edit Pet")).click();}@Overridepublic void e_AddNewPet() {$(By.linkText("Add New Pet")).click();}@Overridepublic void e_AddVisit() {$(By.linkText("Add Visit")).click();}@Overridepublic void e_FindOwners() {$("[title='find owners']").click();}@Overridepublic void e_AddPetSuccessfully() {Date date = new Faker().date().past( 365 * 20, TimeUnit.DAYS);SimpleDateFormat sdf;sdf = new SimpleDateFormat("yyyy-MM-dd");String birthData = sdf.format(date);$(By.id("birthDate")).clear();$(By.id("birthDate")).sendKeys(birthData + Keys.ENTER);$(By.id("name")).clear();$(By.id("name")).sendKeys(new Faker().name().fullName());$(By.id("type")).selectOption(new Faker().number().numberBetween(0,5));$(By.cssSelector("button[type=\"submit\"]")).click();}@Overridepublic void v_NewPet() {$(By.tagName("h2")).shouldHave(text("New Pet"));$(".has-feedback").shouldBe(visible);}@Overridepublic void e_VisitAddedSuccessfully() {$(By.id("description")).clear();$(By.id("description")).sendKeys(new Faker().lorem().word());$("button[type=\"submit\"]").click();}@Overridepublic void v_NewVisit() {$(By.tagName("h2")).shouldHave(text("New Visit"));}@Overridepublic void v_Pet() {$(By.tagName("h2")).shouldHave(text("Pet"));}@Overridepublic void e_AddPetFailed() {$(By.id("name")).clear();$(By.id("birthDate")).clear();$(By.id("birthDate")).sendKeys("2015/02/05" + Keys.ENTER);$(By.id("ui-datepicker-div")).shouldBe(not(visible));$(By.id("type")).selectOption("dog");$("button[type=\"submit\"]").click();}@Overridepublic void e_VisitAddedFailed() {$(By.id("description")).clear();$("button[type=\"submit\"]").click();}
}

（五）DDT Data Driven Testing

1.DDT定义

数据驱动测试（DDT），也称为表驱动测试或参数化测试，是一种软件测试方法，用于计算机软件的测试，用于描述使用条件表直接作为测试输入和可验证输出完成的测试以及测试环境设置和控制没有硬编码的过程

Data-driven testing (DDT), also known as table-driven testing or parameterized testing, is a software testing methodology that is used in the testing of computer software to describe testing done using a table of conditions directly as test inputs and verifiable outputs as well as the process where test environment settings and control are not hard-coded.

2.DDT 相关工具

• DDT 是一种实践，可以跟很多框架结合
• 单元测试结合 DDT：JUnit4 JUnit5 TestNG
• RobotFramework DDT
• YAML JSON CSV 驱动 HttpRunner

3.数据驱动应用案例

• HttpRunner 可以根据代理抓包自动生成测试用例
• YAPI、Swagger 等工具可以根据数据自动生成测试用例代码
• JVM-Sandbox-Repeater Gor 录制工具可以把请求保存为测试用例并重放以实现快速回归测试

4.HttpRunner 测试框架

5.HttpRunner 测试用例

config:name: "request methods testcase with functions"variables:foo1: config_bar1foo2: config_bar2expect_foo1: config_bar1expect_foo2: config_bar2base_url: "https://postman-echo.com"verify: Falseexport: ["foo3"]teststeps:- name: get with paramsvariables:foo1: bar11foo2: bar21sum_v: "${sum_two(1, 2)}"request:method: GETurl: /getparams:foo1: $foo1foo2: $foo2sum_v: $sum_vheaders:User-Agent: HttpRunner/${get_httprunner_version()}extract:foo3: "body.args.foo2"validate:- eq: ["status_code", 200]- eq: ["body.args.foo1", "bar11"]- eq: ["body.args.sum_v", "3"]- eq: ["body.args.foo2", "bar21"]- name: post raw textvariables:foo1: "bar12"foo3: "bar32"request:method: POSTurl: /postheaders:User-Agent: HttpRunner/${get_httprunner_version()}Content-Type: "text/plain"data: "This is expected to be sent back as part of response body: $foo1-$foo2-$foo3."validate:- eq: ["status_code", 200]- eq:["body.data","This is expected to be sent back as part of response body: bar12-$expect_foo2-bar32.",]- name: post form datavariables:foo2: bar23request:method: POSTurl: /postheaders:User-Agent: HttpRunner/${get_httprunner_version()}Content-Type: "application/x-www-form-urlencoded"data: "foo1=$foo1&foo2=$foo2&foo3=$foo3"validate:- eq: ["status_code", 200]- eq: ["body.form.foo1", "$expect_foo1"]- eq: ["body.form.foo2", "bar23"]- eq: ["body.form.foo3", "bar21"]

6.数据驱动风格为什么广受欢迎

维护成本最低，录制回放技术越来越成熟，可以与数据驱动很好的结合。低代码、用例生成技术的流行，会让数据驱动风格更受欢迎。