# Webdriver Tests

This is the home of the Nomulus webdriver tests. These tests run our user-facing
HTML/JavaScript code in the context of a set of common browsers. See the [Webdriver documentation](https://www.seleniumhq.org/docs/03_webdriver.jsp)
for more information on using webdriver.

## My Screenshot Tests are Failing!

1. Make sure [Docker](https://www.docker.com/) is installed
  * Docker is used to provision the browser and the Webdriver service, so please
    make sure Docker is installed and the `docker` command is availbe in the shell.
2. Missing golden images
  * If you added a new test using screenshot comparison, you have to generate
    the golden image for that test in advance and copy it to
    [goldens/](https://github.com/google/nomulus/tree/master/core/src/test/resources/google/registry/webdriver/goldens)
    folder. There
    is an auxiliary Gradle build task to help with this, and here are some examples:
    ```shell
    # Generate golden images for all tests. All generated images are stored in
    # ${projectRoot}/gradle/core/build/new_golden_images
    $ ./gradlew :core:generateGoldenImages

    # Generate the golden image for a certain test
    $ ./gradlew :core:generateGoldenImages \
      --tests "google.registry.webdriver.[MyTestClass.my_test]"
    ```
3. Screenshot differs by X pixels
  * If you made any change to the existing test and expected to affect the web page,
    the screenshot should be different from the previously generated golden image.
    In this case, you can just use the screenshot as the new golden image.
  * If you didn't expect to have any change, you have to use your own justification
    to mitigate it. For example, if the number of different pixels is very small,
    like 1 piexel, it may be because the test is flaky and a retry can probably solve
    it; also, if there is any change to the version of browser or Webdriver library,
    it could affect the screenshot as well.

## My Screenshot Tests are Timing Out!

The screenshot tests display your HTML and generated Javascript in an actual
browser. There can be multiple HTTP queries sent to the browser and the tests
need to know when everything is finished loading before taking a screenshot.
This is normally done by waiting for the browser page to contain a specific HTML
element that isn't present until the end of the load and render. When the
element shows up, it's safe to take a screenshot.

If that element doesn't show up, the tests will eventually time out. This could
be due to a difference that you've introduced but more likely it is due to a bug
in your javascript. Unfortunately, Javascript errors aren't visible from the
screenshot tests. This can make these problems very difficult to debug.

Compounding this problem is the fact that we do have unit tests, but they run
against the uncompiled javascript. The screenshot tests run against the closure
compiled javascript, which means that they are susceptible to at least one new
class of errors due to the name-mangling that happens during a compile. One
thing to consider is that name-mangling breaks the use of object attributes
accessed via strings: e.g. "foo['bar']" is a different attribute from "foo.bar"
after name mangling - the second case gets mangled, the first does not.

In any case, the following tools can be helpful in debugging these kinds of
problems:

-   The "webm" files. Along with the pngs in the output files, there are a set
    of .webm files. These are movies taking of the browser during the test, and
    will show all intermediate steps up to the timeout. Useful information can
    be gleaned from them.
-   Alerts. Javascript alerts (generated by the plain old "`alert()`" function)
    show up right in the logfile, which is directly available from the shell.
    You can instrument your code with alerts, but keep in mind that the driver
    doesn't respond to alerts, so the first one you hit is the only one you'll
    see.
