So testing, right? We should do it. The thing is, testing is hard, and good testing is reaaaaaaally hard, and tbh Iām pretty bad at testing. So I end up not testing my apps, and then I feel guilty about it, but Iāll stop you now: you canāt run guilt on Travis. If this sounds familiar, then this blog post is for you.
I did a little song-and-dance that sets up Puppeteer* , takes screenshots of your app (like, all the routes you care about), and then compares them to the āgoldenā ones. If they match, your test passes! Yes, it only works on Chrome. No, itās not actually unit testing. Yes, itās literally just counting pixels but you know what? It counts them in both a wide and a narrow viewport size and any testing is better than no testing at all; fight me.
* Puppeteer is an npm
library that lets you control Chrome. You know, like a puppet.
In particular, Puppeteer makes
it super easy to take screenshots (and click on things in your page). Itās
like a waaaaaaay less infuriating Selenium, but infinitely harder to spell.
This post looks long because Iāve put all the code I have so that you can copy paste it. Skip to the good part if you already know how to test.
Do the npm
If you want to test things with Puppeteer, you have to setup a thing for the
tests, a server that launches your site, and then Puppeteer to look
at that site. I have this in my package.json
to wrangle these things:
"devDependencies": {
"chai": "^4.1.2",
"mocha": "^5.0.0",
"puppeteer": "^1.0.0",
"pixelmatch": "^4.0.2",
"polyserve": "^0.23.0"
}
Explanation:
- I chose Mocha/Chai for testing because thatās what Iām used to. You can literally use any other testing framework youāre comfortable with; I donāt think it matters.
Pixelmatch
is the thing that diffs two images and tells you how many pixels they differ by. Itās super awesome š.Polyserve
is what I use as a local server. You can use Python or Express or whatever you cool kids use. Iāll point out in the code where itās Polyserve specific (literally 2 lines), and you can sub in your favourite server there.
Set up your test
In order to tell Puppeteer to investigate your site, you need to:
- start a test suite
- that sets up a local server
- and in each test tells Puppeteer to do something.
My setup looks like this:
const puppeteer = require('puppeteer');
const expect = require('chai').expect;
const {startServer} = require('polyserve');
describe('š screenshots are correct', function() {
let polyserve, browser, page;
// This is ran when the suite starts up.
before(async function() {
// This is where you would substitute your python or Express server or whatever.
polyserve = await startServer({port:4000});
// Create the test directory if needed. This and the goldenDir
// variables are global somewhere.
if (!fs.existsSync(testDir)) fs.mkdirSync(testDir);
// And its wide screen/small screen subdirectories.
if (!fs.existsSync(`${testDir}/wide`)) fs.mkdirSync(`${testDir}/wide`);
if (!fs.existsSync(`${testDir}/narrow`)) fs.mkdirSync(`${testDir}/narrow`);
});
// This is ran when the suite is done. Stop your server here.
after((done) => polyserve.close(done));
// This is ran before every test. It's where you start a clean browser.
beforeEach(async function() {
browser = await puppeteer.launch();
page = await browser.newPage();
});
// This is ran after every test; clean up after your browser.
afterEach(() => browser.close());
// Wide screen tests!
describe('wide screen', function() {
beforeEach(async function() {
return page.setViewport({width: 800, height: 600});
});
it('/view1', async function() {
return takeAndCompareScreenshot(page, 'view1', 'wide');
});
// And your other routes, 404, etc.
});
// Narrow screen tests are the same, but with a different viewport.
describe('narrow screen', function() {
beforeEach(async function() {
return page.setViewport({width: 375, height: 667});
});
it('/view1', async function() {
return takeAndCompareScreenshot(page, 'view1', 'narrow');
});
// And your other routes, 404, etc.
});
});
You can test all sort of things here, by the way. Puppeteer lets you interact with the page (click on buttons, links, etc), so maybe you want to trigger different UI states before you screenshot them (like narrow view but also with the navigation drawer opened).
Filling in the blanks
All the heavy lifting (which isnāt very heavy tbh) is done in takeAndCompareScreenshot
:
// - page is a reference to the Puppeteer page.
// - route is the path you're loading, which I'm using to name the file.
// - filePrefix is either "wide" or "narrow", since I'm automatically testing both.
async function takeAndCompareScreenshot(page, route, filePrefix) {
// If you didn't specify a file, use the name of the route.
let fileName = filePrefix + '/' + (route ? route : 'index');
// Start the browser, go to that page, and take a screenshot.
await page.goto(`http://127.0.0.1:4000/${route}`);
await page.screenshot({path: `${testDir}/${fileName}.png`});
// Test to see if it's right.
return compareScreenshots(fileName);
}
Getting the golden screenshots
This bit is easy. Make a different test suite (just make sure you donāt run it every time you
run your tests), and run the page.goto
and page.screenshot
lines for all
the routes youāre testing. I recommend doing the viewport dance too, to get both the
wide and narrow screen ones for freeeeee (I am using just the viewport
size here, because thatās how my app works. Puppeteer lets yo do device emulation
and all sorts of other goodness, so just read the docs). Put all these screenshots in a place; I put mine in
a folder called test/screenshots-golden/
.
The thing that does the diffing
This is the logic in compareScreenshots
, and itās basically straight
out of the Pixelmatch
docs:
function compareScreenshots(fileName) {
return new Promise((resolve, reject) => {
const img1 = fs.createReadStream(`${testDir}/${fileName}.png`).pipe(new PNG()).on('parsed', doneReading);
const img2 = fs.createReadStream(`${goldenDir}/${fileName}.png`).pipe(new PNG()).on('parsed', doneReading);
let filesRead = 0;
function doneReading() {
// Wait until both files are read.
if (++filesRead < 2) return;
// The files should be the same size.
expect(img1.width, 'image widths are the same').equal(img2.width);
expect(img1.height, 'image heights are the same').equal(img2.height);
// Do the visual diff.
const diff = new PNG({width: img1.width, height: img2.height});
const numDiffPixels = pixelmatch(
img1.data, img2.data, diff.data, img1.width, img1.height,
{threshold: 0.1});
// The files should look the same.
expect(numDiffPixels, 'number of different pixels').equal(0);
resolve();
}
});
}
šÆ Itās all worth it
Now, when you run your tests (mocha test/ --timeout 5000
in my case), you get
something like this:
And if it fails, you get an error and the number of pixels youāre off by.
āļø
Now go on, navigate to all your routes and test your stuff, and thank me with a photo of your dog.