Getting a Code Coverage report With Selenium

Introduction

The code coverage always comes with unit-test because it is something you put after doing it.

You can get a percentage of the code what was tested. If all the conditionals paths included in the code was cover.

I was viewing youtube when this video showed.

HolyMolly, from a e2e test you can check the coverage part of the frontend. I know doing unit-test from front-end it’s hard because you have to cover every component. But with e2e you can get the same result of the unit-test and get a coverage.

It’s like the missing part from a integration testing in the frontend.

This is amazing.. But does it work for only cypress ? What about Selenium?

I’m a advocate of selenium, because my first experiments with it was spamming Facebook groups, with a selenium script.

At the end I get the way to implement code coverage on e2e testing with Selenium. Here the recipe

1.- Instrument the javascript code and get the __coverage__ object

If you have written unittest with javascript you know the tool called Istanbul it is the tool what do the magic behind the coverage. The way that it works is creating a wrapper of all code and register the execution path of javascript.

When your js code is wrapped is called Instrumented Code.

After the execution of the Instrumented code, you will get a __coverage__ javacript object with all the executed path and percentage of coverage.

Therefore if you have a nextjs project or vanilla javascript(a simple js) you have to instrument the files to get the __coverage__ variable.

There isn’t a simple way to do this, could be variable according of the situation.

If you use Nextjs projects SWC-coverage-instrument works good. For getting it working, depends of nextjs version, the package needs to be downgraded.

Let me know by twtter @zodman if you like a little tutorial how implement this.

2.- With the code instrumented execute the e2e test.

Here comes the good part where you have fired the e2e script with selenium. I’m a python dev, I used unittest.TestCase to implement my selenium testcases.

After finishing the tests you will get window.__coverage__ variable with all the code executed.

    def tearDown(self) -> None:
        r  = self.driver.execute_script("return window.__coverage__");
        with open(f"coverage_{__name__}.json","w") as f:
            f.write(json.dumps(r))

The reason because I saved this object like a json. It is because that variable contains all the coverage code executed.

I need recollect all coverages like pokemon, to be analyzed by istanbul later.

3.- Analyze coverage json files and get the report.

Now with files like:

coverage_test_todo.json
coverage_test_addtodo.json

I move all the files where the frontend project was instrumented because there is the source code is located and link with the instrumented code, and put behind .nyc_output directory

mkdir -p .nyc_output
mv *.json .nyc_output
npx -y nyc report

--------------|---------|----------|---------|---------|------------------------
File          | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s       
--------------|---------|----------|---------|---------|------------------------
All files     |   72.91 |    69.23 |   63.26 |   72.53 |                         
 app.jsx      |      80 |    76.47 |    64.7 |      80 | 58-59,67-84,95-97       
 footer.jsx   |     100 |       75 |     100 |     100 | 6                       
 todoItem.jsx |   34.61 |    41.17 |   55.55 |   34.61 | 16-41,72-74             
 todoModel.js |   66.66 |       50 |      50 |   65.71 | 46-50,64-68,72-76,80-84 
 utils.js     |     100 |    90.47 |     100 |     100 | 34-42                   
--------------|---------|----------|---------|---------|------------------------

PUM! 💥💥💥💥💥💥

Code Repository: https://github.com/zodman/code-coverage-with-selenium

Let’s talk about the mvctodo what I have on the reposity, the project it is with react.

I need to instrument the code if you see there a vanilla implementation of javascript.

<script src="node_modules/todomvc-common/base.js"></script>
<script src="node_modules/react/dist/react-with-addons.js"></script>
<script src="node_modules/classnames/index.js"></script>
<script src="node_modules/react/dist/JSXTransformer.js"></script>
<script src="node_modules/director/build/director.js"></script>

<script src="js/utils.js"></script>
<script src="js/todoModel.js"></script>
<!-- jsx is an optional syntactic sugar that transforms methods in React's
`render` into an HTML-looking format. Since the two models above are
unrelated to React, we didn't need those transforms. -->
<script type="text/jsx" src="js/todoItem.jsx"></script>
<script type="text/jsx" src="js/footer.jsx"></script>
<script type="text/jsx" src="js/app.jsx"></script>

Here I take advantage of a automatic bundler called parcel, it takes all the js used in index.html and create a single file index.hash.js inject on index and with the package babel-plugin-istanbul the bundler was instrumented rock yeah! 🤘🤘🤘

You can take a look of the window.__coverage__

You can see the coverage in action here: