Running Application Audits with Lighthouse

It was only recently that I was made aware of Lighthouse, a relatively new tool for running notable audits for web apps. It’s an open-source tool from Google, and can be found inside Chrome’s DevTools under the Audits tab.

Here’s what it looked like when our team tried it on a recent project (click GIF for a full view on another tab):

Running Audits with Lighthouse

The results:

Accessibility, Best Practices, and SEO Audits

Apparently we did good on accessibility, SEO, and best practices. We don’t usually test for these things, but it’s nice to know that we do enough on them. The application running on localhost wasn’t configured to run on HTTPS yet so it’s about right that we failed that audit for best practice. It’s also true that our app was designed to have small font sizes.

Performance Audit

What’s interesting to see was the dreadfully poor app performance. Throttling our page on a mobile device with 3G connection, it took at least 20 seconds before anything appeared on the page. That’s not good.

And some culprits:

Performance Audit: Opportunities for Improvement

It seems we can do better with our images and style sheets. And it also looks like our JS takes too long to boot-up. We don’t exactly do high-detail performance testing at work so it is awesome to have such a quick audit report from within the browser that notify us about such issues early. At least we’ll know the areas where there’s room for improvement right away.

There’s no excuse to not testing performance (as well as SEO, accessibility, and best practices) on a web app anymore, as far as tools are concerned.

Advertisements

About failing Automated Watir checks on Firefox v.48 and successfully running them on Google Chrome v.52

The recent updates on the Mozilla Firefox browser starting from version 46 onwards broke my automated end-to-end checks. The browser loads when started but does not go to a test page or do anything until it eventually fails, because the latest versions of Firefox supposedly doesn’t use the FirefoxDriver anymore for automation and instead makes use of a new driver implementation in what is called Marionette. In order to get my checks running again in Firefox, I had to resort (as what many others in the community also did) to using the Extended Support Release version of Firefox v. 45, a temporary measure until I finally get my Cucumber-Watir checks running properly on the latest Firefox version.

At the moment I am stumped by an error when running automated checks on Firefox v. 48.0.1 using Marionette (which seems to be a problem on permissions on my local Windows machine, and not on watir or selenium, although port 4444 points to the port which selenium grid is connected to):

Permission denied - bind(2) for "::1" port 4444 (Errno::EACCES)
C:/Ruby/lib/ruby/gems/2.2.0/gems/selenium-webdriver-2.53.4/lib/selenium/webdriver/firefox/service.rb:103:in `stop_process': undefined method `poll_for_exit' for nil:NilClass (NoMethodError)
from C:/Ruby/lib/ruby/gems/2.2.0/gems/selenium-webdriver-2.53.4/lib/selenium/webdriver/firefox/service.rb:83:in `stop'
from C:/Ruby/lib/ruby/gems/2.2.0/gems/selenium-webdriver-2.53.4/lib/selenium/webdriver/firefox/service.rb:64:in `block in start'
from C:/Ruby/lib/ruby/gems/2.2.0/gems/selenium-webdriver-2.53.4/lib/selenium/webdriver/common/platform.rb:161:in `block in exit_hook'

So, while I’m trying to sort out that problem (anybody experienced this?), I decided to move my checks to run on the latest version of Google Chrome by default instead of running them on an old version of Firefox. To do that, I needed to update my browser capabilities:

if browser_type == :chrome
    arguments = "--ignore-certificate-errors" // Add more desired arguments
    capabilities = Selenium::WebDriver::Remote::Capabilities.chrome "chromeOptions" => {"args" => [ arguments ]}
elsif browser_type == :firefox
    capabilities = Selenium::WebDriver::Remote::Capabilities.firefox
end
browser = Watir::Browser.new :remote, :url => url, :desired_capabilities => capabilities

And all that is left is to see how the checks behave on Google Chrome.

Some findings:

  • Button or link elements in the browser (especially when they are only span or heading elements that are being used functionally as buttons) that are not immediately visible in the Chrome viewport fail to be clicked properly. To fix this, focus on the element first (or alternatively call a scroll method in the browser using javascript to a position where the element becomes visible in the viewport) before running the click step.
  • Some wait methods that didn’t necessarily have to be written (when checks were run in Firefox) need to be explicitly stated when running checks on Chrome.