[Feature] Add support for test.each / describe.each

You can already do each via running test with forEach:

[
  { a: 1, b: 1, expected: 2 },
  { a: 1, b: 2, expected: 3 },
  { a: 2, b: 1, expected: 3 },
].forEach(({ a, b, expected }) => {
  test(`given ${a} and ${b} as arguments, returns ${expected}`, ({ page }) => {
    expect(a + b).toEqual(expected);
  })
});

If the above does not work for you, please tell us why in the comments.

Note from maintainers

Hey everyone,

We discussed this feature extensively within the team and concluded that using a forEach loop is preferable to test.each. Using the standard JavaScript syntax gives you full IDE support. See the discussion in this StackOverflow post for more details.

Using forEach won't affect parallelism; tests will continue to run in parallel as per our parallelism documentation.

Thank you,

< 3 Playwright Team

Hi,
I am currently using the this approach (loop) for data driven tests and was wondering, is there a way we can make these tests run in parallel? Because right now they are being executed sequentially and for "slow" tests the execution time is increased.

Paresh (@partparsh)

+1 to Paresh (@partparsh) 's suggestion of parallelization. We have a suite of a single test with 20+ parameters that unfortunately adds a lot of time to our build.

For of does work, but Jest's test.each() is more convenient for us, here's an example:
We have a data-driven test that checks URL redirecting. The test works with an array that represents 3 cases with 2 parameters in each case, e.g:

const testCases = [
  ['singlePath', `${ROUTES.HOME}singlePath`],
  ['nestedPath', `${ROUTES.HOME}nested/path`],
  ['section', `${ROUTES.COMPLICATED_PATH}section`],
];

So with Jest, I can write my test like this:

test.each(testCases)('%s url', async (urlType, url)) => {
  await page.goto(url);
  expect(page.url()).toBe(errorPage);
}

With PW test I have to write like this:

  for (const data of testCases) {
    test(`${data[0]} url`, async ({ page }) => {
      await page.goto(data[1]);
      expect(page.url()).toBe(errorPage);
    });
  }

These data[0], data[1] don't look good, I can reassign them...

  for (const data of testCases) {
    const urlType = data[0];
    const url = data[1];
    ...

...or use destructuring assignment...

 for (const [urlType, url] of testCases) {
    ...

...but with data-driven tests with a lot of parameters, it's not very convenient and tests look slightly messy.

I don't see any real different between the following two. If anything, the latter even gives me more control - for example, I can skip a test when urlType has some specific value, and more.

test.each(testCases)('%s url', async (urlType, url)) => {
});

for (const [urlType, url] of testCases) {
  test(`${url} url`, async ({ page }) => {
  });
}

In my case below, the hook beforeEach, afterEach runs 5 times per 1 test.

How could we achieve the goal that 1 hook 1 test ? Note that, the beforeEach/afterEach could not move out of the for loop
because I need to get data from array arr to initialize (e.g. log-in with admin, log-in with supervisor...).

test.describe.only('Looping', () => {
    let arr = [1, 2, 3, 4, 5]
    for (const tour of arr) {

        test.beforeEach(async () => {
            console.log("before each: ", tour)

        })

        test.afterEach(() => {
            console.log("after each", tour)
            console.log("======")

        })


        test(`TEST ${tour}`, async () => {
            console.log("test: ", tour)
            expect(1).toBe(1)
        })
    }
})

Hung Dao (@hungdao-testing)

Hung Dao (@hungdao-testing) Just make a few describes in a loop.

let arr = [1, 2, 3, 4, 5];
for (const tour of arr) {
   test.describe.only(`Looping ${tour}`, () => {
        test.beforeEach(async () => {
            console.log("before each: ", tour)
        })

        test.afterEach(() => {
            console.log("after each", tour)
            console.log("======")
        })


        test(`TEST ${tour}`, async () => {
            console.log("test: ", tour)
            expect(1).toBe(1)
        })
    })
}

I don't see any real different between the following two. If anything, the latter even gives me more control - for example, I can skip a test when urlType has some specific value, and more.

Playwright already allows us to do conditional test.skip inside test method, so i don't think "more control" argument is valid.

+1 for test.each notation. Also, Jest has amazing way of populating data via table, which is very convenient when test case has 3+ parameters

That would be nice to have IMO. My argument is more psychological 😄
I do agree there is no difference between having a for loop but test.each keeps you more in a "testing mindset".
Also for consistency.

+1 for test.each

I don't know if there's something wrong with my code, but the setup doesn't work with the for aproach.

If others have the same problem, it would be another valid point to add the test.each feature.

I'd also prefer using test.each syntax to the for loop, fwiw. I am finding the loop approach to be rather clunky.

test.each and describe.each would have an advantage for IDE integrations also. These integrations cannot handle tests inside a regular for-loop, but the *.each syntax could be handled well.

test.describe "Something" {
    test.describe.configure "serial mode"
    
    links = [];

    test "get some links from a page" {
        populate $links
    }

    foreach $links {
       test "check link"
    }
}

This does not work. I need to put the loop inside the "check link" test. MAYBE or HOPEFULLY, this is approach is possible with test.each.

Could we have an update on the progress of the ticket regarding the each syntax implementation? Other testing frameworks have already adopted this feature, and it would be beneficial for us to understand the timeline for its integration into our processes.

disclaimer: this is just a gist of how to shape the code setup for what your goals are.
disclaimer2: I quickly whipped this up together, so pick it apart if you want, ya'll can take it and run with it

testProxy.js

export const testFor = (testInput, instanceOfThis) => {
    const thisReference = instanceOfThis??testInput;

    const self = {};

    self.cases = (function(arrayReference){
        return ({run: this});
    }).bind(thisReference);
    
    self.casesAsync = (async function(arrayReference){
        return {run: this};
    })
    
    const getSelf = (property) => self[property].bind(thisReference)

    return new Proxy(testInput,
        {
            get:function(target,property,receiver){
                if(property in self)
                    return getSelf(property);
                return target[property];
            },
        }
    );
}

sample.spec.js

import {test as t} from 'your test source';
const test = testFor(t);

test.cases([1,2,3]).run('testname', () => expect(1).toEqual(1));

it's basically cheating.

scenario.js

const withName = (obj) => typeof obj === 'object' && 'name' in v && 'name' || JSON.stringify(obj);

export const getScenario = (test, op) => ({
  scenario: ({ name, testCases: testCases = [undefined], run }) => {
    const items = testCases.map((testCase) => {
      const testFunction = ({ pageInstance }) => run(pageInstance, { testCase });
      return [`${name} ${(testCase === undefined && ' ' || 'case' && withName(testCase))}`, testFunction];
    });

    for (let a of items) (async () => op(a))();
  },
  test: test,
});

your.spec.js

import { test as t } from '../customExtendedTest';
import { getScenario, testFor } from '../scenario.js';
import { expect } from '@playwright/test';

const { test, scenario } = getScenario(t, (e) => t(...e));

test.describe('the test', async () => {
  scenario({
    name: 'my scenario',
    testCases: [1, 2, 3],
    run: async ({ pageInstance }, {testCase}) => {
      expect(1).toEqual(1);
    },
  });
});

For the time being we're using a helper tagging function:

export function fromTable<
    S extends string,
    A extends ReadonlyArray<S>,
    P extends any[]
>([header]: A, ...values: P): { [K in S]: P[number] }[] {
    // read header from table
    const headers = header.split('|').map(h => h.trim());
    const rows: Array<Record<S, P>> = [];

    // gather data from table
    for (let i = 0; i < values.length; i += headers.length) {
        const row = values.slice(i, i + headers.length);
        const params = {} as Record<S, P>;
        for (let j = 0; j < headers.length; j++) {
            params[headers[j] as S] = row[j];
        }
        rows.push(params);
    }

    return rows;
}

This can be used in a for loop with a template string:

for (const { a, b, c } of fromTable`
        a    | b      | c
        ${1} | ${'a'} | ${true}
        ${2} | ${'b'} | ${false}
    `) {
    test(`Each with fromTable ${a}`, async () => {
        expect(typeof a).toBe('number');
        expect(typeof b).toBe('string');
        expect(typeof c).toBe('boolean');
    });
}

+1 to test.each and describe.each with parallelization. The ergonomics of playwright already align with things like testing library and jest, so why not complete the catalog? As an engineer, if I can flow between unit tests and e2e tests without having to reach for different tooling, it makes my life easier.

+1 for test.each

Another +1 for test.each/describe.each

Hey everyone,

We discussed this feature extensively within the team and concluded that using a forEach loop is preferable to test.each. Using the standard JavaScript syntax gives you full IDE support. See the discussion in this StackOverflow post for more details.

Using forEach won't affect parallelism; tests will continue to run in parallel as per our parallelism documentation.

Here is an example:

[
  { a: 1, b: 1, expected: 2 },
  { a: 1, b: 2, expected: 3 },
  { a: 2, b: 1, expected: 3 },
].forEach(({ a, b, expected }) => {
  test(`given ${a} and ${b} as arguments, returns ${expected}`, ({ page }) => {
    expect(a + b).toEqual(expected);
  })
});

Thank you,

< 3 Playwright Team

Navigation Menu

Repository navigation

[Feature] Add support for test.each / describe.each #7036

7 remaining items

testProxy.js

sample.spec.js

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[Feature] Add support for test.each / describe.each #7036

Description

Activity

partparsh commented on Jun 25, 2021

suchlike commented on Aug 4, 2021

pheeel commented on Sep 22, 2021

dgozman commented on Jan 4, 2022

hungdao-testing commented on Apr 4, 2022

dgozman commented on Apr 4, 2022

SuwiDeity commented on Apr 20, 2022

4ekki commented on Aug 29, 2022

w9ahmed commented on Sep 6, 2022

7 remaining items

maxsudik commented on Dec 5, 2023

giuliagutierrez commented on Dec 21, 2023

jamieomaguire commented on Jan 2, 2024

vekunz commented on Jan 4, 2024

Morgy93 commented on Jan 26, 2024

iamfj commented on Jan 26, 2024

PowerCreek commented on Jan 27, 2024

testProxy.js

sample.spec.js

PowerCreek commented on Jan 27, 2024

davidenke commented on Feb 8, 2024

curiosity26 commented on Mar 7, 2024

AsemK commented on May 17, 2024

doomjar commented on Jun 11, 2024

mxschmitt commented on Jun 17, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions