{"id":8602,"date":"2021-07-28T11:44:59","date_gmt":"2021-07-28T15:44:59","guid":{"rendered":"https:\/\/blogs.library.duke.edu\/bitstreams\/?p=8602"},"modified":"2021-07-29T23:48:50","modified_gmt":"2021-07-30T03:48:50","slug":"automated-accessibility-testing-and-continuous-integration","status":"publish","type":"post","link":"https:\/\/blogs.library.duke.edu\/bitstreams\/2021\/07\/28\/automated-accessibility-testing-and-continuous-integration\/","title":{"rendered":"Automated Accessibility Testing and Continuous Integration"},"content":{"rendered":"

We use several different tools and strategies<\/a> at the Duke Libraries to ensure that our web interfaces are accessible. Our aim is to comply with WCAG 2.0 AA and Section 508<\/a> guidelines.<\/p>\n

One of our favorite accessibility checking tools in our toolbox is the axe DevTools Browser Extension<\/a> by Deque Systems. It’s easy to use, whether on a live site, or in our own local development environments while we’re working on building new features for our applications. Simply open your browser’s Developer Tools (F12<\/code>), click the Scan<\/code> button, and get an instant report of any violations, complete with recommendations about how to fix them.<\/p>\n

\"screenshot<\/a>
An axe DevTools test result for our archival finding aids homepage.<\/figcaption><\/figure>\n

Now, it’s one thing to make software compliant; it’s another to keep it that way. Duke’s Web Accessibility office astutely notes<\/a>:<\/p>\n

Keeping a website compliant is a continuous effort. Websites are living things. Content changes, features are added. When a site becomes compliant it does not stay compliant.<\/p><\/blockquote>\n

One of the goals we set out to accomplish in 2021 was to figure out how to add automated, continuous\u00a0accessibility testing<\/em> for our ArcLight software<\/a>, which powers our archival finding aids<\/a> search and discovery application. We got it implemented successfully a few months ago and we’re pleased with how it has been working so far.<\/p>\n

The solution: using the Deque Systems Axe Core RSpec gem<\/a><\/strong>, along with a Selenium Standalone Chrome<\/a> Docker image<\/a>, in our GitLab CI pipeline.<\/p>\n

Say what?<\/em><\/p>\n

Let me back up a bit and give some background on the concepts and tools upon which this solution depends. Namely: Continuous Integration, RSpec, Capybara, Selenium, and Docker.<\/p>\n

Continuous Integration<\/h2>\n

For any given software project, we may have several developers making multiple changes to a shared codebase in the same day. Continuous Integration<\/a> is the practice of ensuring that these code changes 1) don’t break any existing functionality, and 2) comply with established guidelines for code quality.<\/p>\n

At Duke Libraries, we use GitLab<\/a> to host code repositories for most of our software (e.g., our ArcLight application<\/a>). It’s similar to GitHub, with built-in tooling for DevOps<\/a>. One such tool is GitLab CI\/CD<\/a>, which we use to manage continuous integration. It’s common for us to configure pipelines<\/a> to run automated tasks for each new code branch or tag, in at least few stages, e.g.: build<\/code> > test<\/code>\u00a0> deploy<\/code>.<\/p>\n

\"screenshot<\/a>
Evidence of a GitLab CI pipeline that has run — and passed all stages — for a code change.<\/figcaption><\/figure>\n

For accessibility testing, we knew we needed to add something to our existing CI pipeline that would check for compliance and flag any issues.<\/p>\n

RSpec Testing Framework<\/h2>\n

Many of our applications are built using Ruby on Rails. We write tests for our code using RSpec<\/a>, a popular testing framework for Rails applications. Developers write tests (using the framework’s DSL \/ domain-specific language) to accompany their code changes. Those tests all execute as part of our CI pipeline (see above). Any failing tests will prevent code from being merged or getting deployed to production.<\/p>\n

\"RSpec<\/a><\/p>\n

There are many different types of tests<\/a>.\u00a0 On one end of the spectrum, there are “unit tests,” which verify that one small piece of code (e.g., one method) returns what we expect it to when it is given different inputs. On the other, there are “feature tests,” which typically verify that several pieces of code are working together as intended in different conditions. This often simulates the use of a feature by a person (e.g., when a user clicks this button, test that they get to this page and verify this link gets rendered, etc.). Feature tests might alternatively be called “integration tests,” “acceptance tests,” or even “system tests” — the terminology is both squishy and evolving.<\/p>\n

At any rate, accessibility testing is a specific kind of feature test.<\/p>\n

Capybara<\/h2>\n

On its own, RSpec unfortunately doesn’t natively support feature tests. It requires a companion piece of software called Capybara<\/a>, which can simulate a user interacting with a web interface. Capybara brings with it a DSL to visit pages, fill out forms, or click on elements within RSpec tests, and special matchers to check that the page is behaving as intended.<\/p>\n

\"screenshot<\/a>
Homepage for Capybara, featuring an actual capybara.<\/figcaption><\/figure>\n

When configuring Capybara, you set up the driver<\/a> you want it to use when running different kinds of tests. Its default driver is RackTest<\/code>, which is fast but it can’t execute JavaScript like a real web browser can. Our ArcLight UI<\/a>, for instance, uses a bunch of JavaScript. So we knew that any accessibility tests would have to be performed using a driver for an actual browser; the default Capybara alone wouldn’t cut it.<\/p>\n

Selenium<\/h2>\n

Writing code to drive various browsers was probably a nightmare until Selenium came along. Selenium is an “umbrella project for a range of tools and libraries that enable and support the automation of web browsers<\/a>.” Its WebDriver<\/a> platform gives you a language-agnostic coding interface that is now compatible with all the major web browsers. That makes it a valuable component in automated browser testing.<\/p>\n

\"screenshot<\/a>
Documentation for Selenium WebDriver for browser automation<\/figcaption><\/figure>\n

The best way we could find to get Capybara to control a real browser in an RSpec test was to add\u00a0the selenium-webdriver<\/a> gem to our project’s Gemfile<\/code>.<\/p>\n

Docker<\/h2>\n

Over the past few years, we have evolved our DevOps<\/a> practice and embraced containerizing our applications using Docker<\/a>. Complex applications that have a lot of interwoven infrastructure dependencies used to be quite onerous to build, run, and share.\u00a0 Getting one’s local development environment into shape to successfully run an application used be a whole-day affair. Worse still, the infrastructure on the production server used to bear little resemblance to what developers were working with on their local machines.<\/p>\n

\"systems<\/a>
A systems diagram depicting the various services that run to support our ArcLight app.<\/figcaption><\/figure>\n

Docker helps a dev team configure in their codebase all of these system dependencies in a way that that’s easily reproducible in any environment. It builds a network of “containers” on a single host, each running a service that’s crucial to the application.\u00a0 Now a developer can simply check out the code, run a couple commands<\/a>, wait a few minutes, and they’re good to go.<\/p>\n

The same basic setup also applies in a production environment (with a few easily-configurable differences). And that simplicity also carries over to the CI environment where the test suite will run.<\/p>\n

\"Systems<\/a>
Orange boxes depict each service \/ container we have defined in our Docker configuration.<\/figcaption><\/figure>\n

So what we needed to do was add another container to our existing Docker configuration that would be dedicated to running any JavaScript-dependent feature tests — including accessibility tests<\/em> — in a browser controlled by Selenium WebDriver.<\/p>\n

Keeping this part containerized would hopefully ensure that when a developer runs the tests in their local environment, the exact same browser version and drivers get used in the CI pipeline. We can steer clear of any “well, it worked on my<\/em> machine” issues.<\/p>\n

Putting it All Together<\/h2>\n

Phew. OK, with all of that background out of the way, let’s look closer at how we put all of these puzzle pieces together.<\/p>\n

Gems<\/h3>\n

We had to add the following three gems to our Gemfile<\/strong>‘s test group<\/a>:<\/p>\n

group :test do\r\n \u00a0gem 'axe-core-rspec<\/a>' # accessibility testing\r\n \u00a0gem 'capybara<\/a>'\r\n \u00a0gem 'selenium-webdriver<\/a>'\r\nend<\/pre>\n
A Docker Container for Selenium<\/h3>\n
The folks at Selenium HQ host “standalone” browser Docker images<\/a> in Docker Hub<\/a>, complete with the browser software and accompanying drivers. We found a tagged version of their Standalone Chrome image that worked well, and pull that into our newly-defined “selenium<\/code>” container for our test environments.<\/p>\n
In docker-compose.test.yml<\/a><\/strong><\/p>\n
services<\/span>:<\/span><\/span>\r\n  selenium<\/span>:<\/span><\/span>\r\n    image<\/span>:<\/span> selenium\/standalone-chrome:3.141.59-xenon<\/span><\/span><\/strong>\r\n    ports<\/span>:<\/span><\/span>\r\n      -<\/span> 4444:4444<\/span><\/span>\r\n    volumes<\/span>:<\/span><\/span>\r\n      -<\/span> \/dev\/shm:\/dev\/shm<\/span><\/span>\r\n    environment<\/span>:<\/span><\/span>\r\n      -<\/span> JAVA_OPTS=-Dwebdriver.chrome.whitelistedIps=<\/span><\/span>\r\n      -<\/span> START_XVFB=false\r\n<\/span><\/span>[...]<\/pre>\n
Since this is new territory for us, and already fairly complex, we’re starting with just one browser: Chrome. We may be able to add more in the future.<\/p>\n
Capybara Driver Configuration<\/h3>\n
Next thing we needed to do was tell Capybara that whenever it encounters any javascript-dependent feature tests, it should run them in our standalone Chrome container (using a “remote” driver).\u00a0 The Selenium WebDriver gem lets us set some options for how we want Chrome to run.<\/p>\n
The key setting here is “headless<\/code>” — that is, run Chrome, but more efficiently, without all the fancy GUI stuff<\/a> that a real user might see.<\/p>\n
In spec_helper.rb<\/a><\/strong><\/p>\n
Capybara<\/span>.<\/span>javascript_driver<\/span> =<\/span> :selenium_remote<\/span><\/span>\r\n<\/span>\r\nCapybara<\/span>.<\/span>register_driver<\/span> :selenium_remote<\/span> do<\/span> |<\/span>app<\/span>|<\/span><\/span>\r\n  capabilities<\/span> =<\/span> Selenium<\/span>::<\/span>WebDriver<\/span>::<\/span>Remote<\/span>::<\/span>Capabilities<\/span>.<\/span>chrome<\/span>(<\/span><\/span>\r\n    chromeOptions: <\/span>{<\/span> args: <\/span>[<\/span><\/span>\r\n      'headless<\/strong>'<\/span>,<\/span><\/span>\r\n      'no-sandbox'<\/span>,<\/span><\/span>\r\n      'disable-gpu'<\/span>,<\/span><\/span>\r\n      'disable-infobars'<\/span>,<\/span><\/span>\r\n      'window-size=1400,1000'<\/span>,<\/span><\/span>\r\n      'enable-features=NetworkService,NetworkServiceInProcess'<\/span><\/span>\r\n    ]<\/span> }<\/span><\/span>\r\n  )<\/span><\/span>\r\n<\/span>\r\n  Capybara<\/span>::<\/span>Selenium<\/span>::<\/span>Driver<\/span>.<\/span>new<\/span>(<\/span>app<\/span>,<\/span><\/span>\r\n                                 browser: :remote<\/span>,<\/span><\/span>\r\n                                 desired_capabilities: <\/span>capabilities<\/span>,<\/span><\/span>\r\n                                 url: <\/span>'http:\/\/selenium:4444\/wd\/hub<\/strong>'<\/span>)<\/span><\/span>\r\nend<\/span><\/span>\r\n<\/pre>\n
That last URL http:\/\/selenium:4444\/wd\/hub<\/code> is the location of our Chrome driver within our selenium container.<\/p>\n
Capybara<\/span>.<\/span>server<\/span> =<\/span> :puma<\/span>,<\/span> {<\/span> Threads<\/span>:<\/span> '1:1'<\/span> }<\/span><\/span>\r\nCapybara<\/span>.<\/span>server_port<\/span> =<\/span> '3002<\/strong>'<\/span><\/span>\r\nCapybara<\/span>.<\/span>server_host<\/span> =<\/span> '0.0.0.0<\/strong>'<\/span><\/span>\r\nCapybara<\/span>.<\/span>app_host<\/span> =<\/span> \"http:\/\/app:<\/strong><\/span>#{<\/span>Capybara<\/span>.<\/span>server_port<\/span>}<\/span><\/strong>\"<\/span><\/span>\r\nCapybara<\/span>.<\/span>always_include_port<\/span> =<\/span> true<\/span><\/span>\r\nCapybara<\/span>.<\/span>default_max_wait_time<\/span> =<\/span> 30<\/span> # our ajax responses are sometimes slow<\/span><\/span>\r\nCapybara<\/span>.<\/span>enable_aria_label<\/span> =<\/span> true\r\n[...]<\/span><\/span>\r\n<\/pre>\n
The server_port<\/code>, server_host<\/code> and app_host<\/code> variables are the keys here. Basically, we’re saying:<\/p>\n
There are a few other important Capybara settings configured in spec_helper.rb<\/a> <\/strong>that are needed in order to get our app<\/code> and selenium<\/code>containers to play nicely together.<\/p>\n