AWS Device Farm忽略TestNG优先级注释

Question

我用setUp()创建了一个测试套件类，并且在tearDown()和@BeforeClass注释下声明了@AfterClass方法。

对于每个测试用例，我在这个类中包含了6个测试用例和@Test(priority = #)。我在本地计算机上执行此类并且它工作正常但是当我在AWS Device Farm上传它时，它只执行一个带有（priority = 1）的测试用例，并且在初始化appium服务器时执行所有其他测试用例设备场。

如何克服这个阻碍？

Answer 1

您使用的是自定义环境吗？在标准环境下可以预期这种限制。

https://aws.amazon.com/premiumsupport/knowledge-center/xml-file-tests-jar-file-device-farm/

注意：在Device Farm的标准环境中，testng.xml文件仅支持一部分功能。如果优先级，项目需要包含标记，排除标记，复杂分组或使用testng.xml文件中的参数，则使用自定义环境。

[编辑]

我使用示例应用程序和测试测试了优先级

git clone https://github.com/aws-samples/aws-device-farm-appium-tests-for-sample-app.git
cd aws-device-farm-appium-tests-for-sample-app/
mkdir ./src/test/resources
// I used VS code here but any text editor will work
code ./src/test/resources/testng.xml

然后我添加了一些上面的支持链接的代码，这样我就不会经历所有测试：

的testng.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Default Suite">
    <test name="test">
        <classes>
            <class name="Tests.LoginTest"/> <!--Package.ClassName-->
        </classes>
    </test>
</suite>

然后我修改了LoginTest以使用TestNG的priority feature。

login test.Java

/*
 * Copyright 2014-2016 Amazon.com, Inc. or its affiliates. All Rights Reserved.
 *
 * Licensed under the Apache License, Version 2.0 (the "License").
 * You may not use this file except in compliance with the License.
 * A copy of the License is located at
 *
 * http://aws.amazon.com/apache2.0
 *
 * or in the "license" file accompanying this file. This file is distributed
 * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
 * express or implied. See the License for the specific language governing
 * permissions and limitations under the License.
 */

package Tests;

import Pages.LoginPage;
import Tests.AbstractBaseTests.TestBase;
import org.testng.Assert;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;

/**
 * Tests for a login page
 */
public class LoginTest extends TestBase {
    private static final String LOGIN_SUCCESS_MESSAGE = "You are logged on as admin";
    private static final String LOGIN_FAIL_MESSAGE = "You gave me the wrong username and password";
    private static final String CORRECT_USER_NAME = "admin";
    private static final String CORRECT_PASSWORD = "password";
    private static final String FAIL_USER_NAME = "Wrong User";
    private static final String FAIL_PASSWORD = "12345";
    private static final String BAD_TEXT_ENTRY_MSG = "Username sent to text field incorrectly";

    private LoginPage loginPage;

    @Override
    public String getName() {
        return "Login Page";
    }

    /**
     * Creates a login
     */
    @BeforeTest
    @Override
    public void setUpPage() {
        loginPage = new LoginPage(driver);
    }

    /**
     * Tests logging in with valid credentials by verifying if the login message is correct
     */
    @Test(priority = 1)
    public void loginSuccess() throws InterruptedException {
        Assert.assertTrue(loginPage.login(CORRECT_USER_NAME, CORRECT_PASSWORD), BAD_TEXT_ENTRY_MSG);
        Assert.assertEquals(loginPage.getMessage(), LOGIN_SUCCESS_MESSAGE);
    }

    /**
     * Tests logging in with invalid credentials by verifying if the error message is correct
     */
    @Test(priority = 2)
    public void loginFail() throws InterruptedException {
        Assert.assertTrue(loginPage.login(FAIL_USER_NAME, FAIL_PASSWORD), BAD_TEXT_ENTRY_MSG);
        Assert.assertEquals(loginPage.getMessage(), LOGIN_FAIL_MESSAGE);
    }

    /**
     * After each test method, logout or try again
     */
    @AfterMethod
    public void logOut() {
        loginPage.pressAltButton();
        Assert.assertTrue(loginPage.checkIfBackAtLogin());
    }
}

我使用命令mvn clean package -DskipTests=true打包测试

在设备群中运行此操作会产生以下结果：

视频：https://imgur.com/YHnq08X

所以我无法重现这个问题。您是否可以提供Device Farm的自定义环境无法优先执行的证据？