Fix a few deprecations (#186 )

Fail gracefully when copying detailed reports (#181 )
* Fail gracefully when copying detailed reports When the detailed reports are copied from GCS to registrars' Drive folders, do not fail the entire copy operation when a single registrar fails. Instead, send an alert email about the failure, and continue to copy the rest of the reports. Also, instead of creating duplicates, overwrite the existing files on Drive. BUG=127690361
2026-06-09 16:33:02 +00:00 · 2019-07-22 14:12:55 -04:00 · 2019-07-22 14:09:49 -04:00 · 2019-07-22 10:04:16 -04:00 · 2019-07-19 16:54:56 -04:00 · 2019-07-18 12:15:15 -04:00
112 changed files with 1502 additions and 501 deletions
@@ -4,3 +4,6 @@ python/
 .*/
 repos/**
 **/.idea/
+*.jar
+!third_party/**/*.jar
+!/gradle/wrapper/**/*.jar
@@ -1,4 +1,3 @@
-
                                 Apache License
                           Version 2.0, January 2004
                        http://www.apache.org/licenses/
@@ -2,7 +2,7 @@

 | Internal Build | FOSS Build | License | Code Search |
 |----------------|------------|---------|-------------|
-|[![Build Status for Google Registry internal build](https://storage.googleapis.com/domain-registry-kokoro/build.svg)](https://storage.googleapis.com/domain-registry-kokoro/index.html)|[![Build Status for the open source build](https://travis-ci.org/google/nomulus.svg?branch=master)](https://travis-ci.org/google/nomulus)|[![License for this repo](https://img.shields.io/github/license/google/nomulus.svg)](https://github.com/jianglai/nomulus/blob/master/LICENSE)|[![Link to Source Graph](https://github.com/sourcegraph/sourcegraph/blob/master/ui/assets/img/sourcegraph-logo.svg)](https://sourcegraph.com/github.com/google/nomulus)|
+|[![Build Status for Google Registry internal build](https://storage.googleapis.com/domain-registry-kokoro/build.svg)](https://storage.googleapis.com/domain-registry-kokoro/index.html)|[![Build Status for the open source build](https://travis-ci.org/google/nomulus.svg?branch=master)](https://travis-ci.org/google/nomulus)|[![License for this repo](https://img.shields.io/github/license/google/nomulus.svg)](https://github.com/google/nomulus/blob/master/LICENSE)|[![Link to Source Graph](https://github.com/sourcegraph/sourcegraph/blob/master/ui/assets/img/sourcegraph-logo.svg)](https://sourcegraph.com/github.com/google/nomulus)|

 ![Nomulus logo](./nomulus-logo.png)

@@ -75,7 +75,7 @@ if (project.path == ":services:default") {

 appengine {
    deploy {
-        project = gcpProject
+        projectId = gcpProject
    }
 }

@@ -83,6 +83,6 @@ dependencies {
    compile project(':core')
 }

-rootProject.deploy.dependsOn appengineDeploy
+rootProject.deploy.dependsOn appengineDeployAll
 rootProject.stage.dependsOn appengineStage
 appengineDeploy.dependsOn rootProject.verifyDeployment
@@ -21,7 +21,7 @@ buildscript {
  }

  dependencies {
-    classpath 'com.google.cloud.tools:appengine-gradle-plugin:1.3.3'
+    classpath 'com.google.cloud.tools:appengine-gradle-plugin:2.0.1'
    classpath "net.ltgt.gradle:gradle-errorprone-plugin:0.6.1"
    classpath 'org.sonatype.aether:aether-api:1.13.1'
    classpath 'org.sonatype.aether:aether-impl:1.13.1'
@@ -46,6 +46,10 @@ plugins {
  id 'com.diffplug.gradle.spotless' version '3.18.0'
 }

+wrapper {
+  distributionType = Wrapper.DistributionType.ALL
+}
+
 apply plugin: google.registry.gradle.plugin.ReportUploaderPlugin

 reportUploader {
@@ -71,6 +75,19 @@ reportUploader {

 apply from: 'dependencies.gradle'

+apply from: 'dependency_license.gradle'
+
+// Custom task to run checkLicense in buildSrc, which is not triggered
+// by root project tasks. A shell task is used because buildSrc tasks
+// cannot be referenced in the same way as tasks from a regular included
+// build.
+task checkBuildSrcLicense(type:Exec) {
+  workingDir "${rootDir}/buildSrc"
+  commandLine '../gradlew', 'checkLicense'
+}
+tasks.checkLicense.dependsOn(tasks.checkBuildSrcLicense)
+tasks.build.dependsOn(tasks.checkLicense)
+
 // Provide defaults for all of the project properties.

 // showAllOutput: boolean.  If true, dump all test output during the build.
@@ -159,9 +176,10 @@ subprojects {
        resolutionStrategy.activateDependencyLocking()
      }
    }
-    // Lock application dependencies.
-    dependencyLocking {
-      lockAllConfigurations()
+    // Lock application dependencies except for the gradle-license-report
+    // plugin. See dependency_license.gradle for more information.
+    configurations.findAll { it.name != 'dependencyLicenseReport' }.each {
+      it.resolutionStrategy.activateDependencyLocking()
    }
  }

@@ -237,3 +255,49 @@ subprojects {
    }
  }
 }
+
+task checkDependenciesDotGradle {
+  def buildSrcDepsFile = File.createTempFile('buildSrc', 'deps')
+  buildSrcDepsFile.deleteOnExit()
+  dependsOn createGetBuildSrcDirectDepsTask(buildSrcDepsFile)
+
+  doLast {
+    Set<String> depsInUse = []
+    allprojects {
+      configurations.all {
+        it.dependencies.findAll { it.group != null }.each {
+          // Note: .toString() is required since GString should
+          // not be mixed with Java Strings.
+          depsInUse.add("${it.group}:${it.name}".toString())
+        }
+      }
+    }
+    if (buildSrcDepsFile.exists()) {
+      depsInUse.addAll(buildSrcDepsFile.readLines())
+    }
+    def unusedDeps =
+        rootProject.dependencyMap.keySet()
+            .findAll { !depsInUse.contains(it) }
+            .toSorted()
+
+    if (unusedDeps.isEmpty()) {
+      return
+    }
+    logger.error(
+        "Unused dependencies in dependencies.gradle:\n${unusedDeps.toListString()}")
+    throw new IllegalStateException(
+        "The dependencies.gradle file should only contain direct dependencies.")
+  }
+}
+tasks.build.dependsOn(tasks.checkDependenciesDotGradle)
+
+def createGetBuildSrcDirectDepsTask(outputFileName) {
+  return tasks
+      .create(
+          "getBuildSrcDeps_${java.util.UUID.randomUUID()}".toString(),
+          Exec) {
+    workingDir "${rootDir}/buildSrc"
+    commandLine '../gradlew', 'exportDenpendencies',
+                "-PdependencyExportFile=${outputFileName}"
+  }
+}
@@ -52,6 +52,7 @@ repositories {
 }

 apply from: '../dependencies.gradle'
+apply from: '../dependency_license.gradle'
 apply from: '../java_common.gradle'

 sourceSets {
@@ -88,3 +89,21 @@ gradle.projectsEvaluated {
    options.compilerArgs << "-Xlint:unchecked"
  }
 }
+
+task exportDenpendencies {
+  def outputFileProperty = 'dependencyExportFile'
+  def output = project.hasProperty(outputFileProperty)
+      ? new PrintStream(
+          new File(project.getProperty(outputFileProperty)))
+      : System.out
+
+  doLast {
+    project.configurations.all {
+      it.dependencies.findAll {
+        it.group != null
+      }.each {
+        output.println("${it.group}:${it.name}")
+      }
+    }
+  }
+}
@@ -0,0 +1,172 @@
+{
+  "allowedLicenses": [
+    {
+      "moduleLicense": "Apache Software License, Version 1.1"
+    },
+    {
+      "moduleLicense": "Apache 2"
+    },
+    {
+      "moduleLicense": "Apache 2.0"
+    },
+    {
+      "moduleLicense": "Apache-2.0"
+    },
+    {
+      "moduleLicense": "Apache License 2.0"
+    },
+    {
+      "moduleLicense": "Apache License v2.0"
+    },
+    {
+      "moduleLicense": "Apache License, Version 2.0"
+    },
+    {
+      "moduleLicense": "Apache Software License - Version 2.0"
+    },
+    {
+      "moduleLicense": "The Apache License, Version 2.0"
+    },
+    {
+      "moduleLicense": "The Apache Software License, Version 2.0"
+    },
+    {
+      "moduleLicense": "The Apache Software License, version 2.0"
+    },
+    {
+      "moduleLicense": "Bouncy Castle Licence"
+    },
+    {
+      "moduleLicense": "BSD"
+    },
+    {
+      "moduleLicense": "BSD 2-Clause License"
+    },
+    {
+      "moduleLicense": "BSD 2-Clause license"
+    },
+    {
+      "moduleLicense": "BSD 3-clause"
+    },
+    {
+      "moduleLicense": "BSD 3-Clause"
+    },
+    {
+      "moduleLicense": "BSD 3-Clause License"
+    },
+    {
+      "moduleLicense": "The 3-Clause BSD License"
+    },
+    {
+      "moduleLicense": "BSD License"
+    },
+    {
+      "moduleLicense": "BSD New License"
+    },
+    {
+      "moduleLicense": "BSD New license"
+    },
+    {
+      "moduleLicense": "BSD style"
+    },
+    {
+      "moduleLicense": "New BSD License"
+    },
+    {
+      "moduleLicense": "Revised BSD"
+    },
+    {
+      "moduleLicense": "The BSD License"
+    },
+    {
+      "moduleLicense": "CC0 1.0 Universal License"
+    },
+    {
+      "moduleLicense": "CDDL 1.1"
+    },
+    {
+      "moduleLicense": "CDDL + GPLv2 with classpath exception"
+    },
+    {
+      "moduleLicense": "COMMON DEVELOPMENT AND DISTRIBUTION LICENSE (CDDL) Version 1.0"
+    },
+    {
+      "moduleLicense": "https://glassfish.java.net/public/CDDL+GPL_1_1.html, https://glassfish.java.net/public/CDDL+GPL_1_1.html"
+    },
+    {
+      "moduleLicense": "CDDL+GPL License"
+    },
+    {
+      "moduleLicense": "\\n                Dual license consisting of the CDDL v1.1 and GPL v2\\n            "
+    },
+    {
+      "moduleLicense": "Eclipse Public License - Version 1.0"
+    },
+    {
+      "moduleLicense": "Eclipse Public License - v 1.0"
+    },
+    {
+      "moduleLicense": "Eclipse Public License 1.0"
+    },
+    {
+      "moduleLicense": "Google App Engine Terms of Service"
+    },
+    {
+      "moduleLicense": "GNU General Public License, version 2, with the Classpath Exception"
+    },
+    {
+      "moduleLicense": "GNU GENERAL PUBLIC LICENSE, Version 2 + Classpath Exception"
+    },
+    {
+      "moduleLicense": "GPL2 w/ CPE"
+    },
+    {
+      "moduleLicense": "GNU Lesser Public License"
+    },
+    {
+      "moduleLicense": "The Go license"
+    },
+    {
+      "moduleLicense": "GWT Terms"
+    },
+    {
+      "moduleLicense": "ICU License"
+    },
+    {
+      "moduleLicense": "The JSON License"
+    },
+    {
+      "moduleLicense": "LGPL-2.1+"
+    },
+    {
+      "moduleLicense": "LGPL, version 2.1"
+    },
+    {
+      "moduleLicense": "MIT"
+    },
+    {
+      "moduleLicense": "MIT License"
+    },
+    {
+      "moduleLicense": "MIT license"
+    },
+    {
+      "moduleLicense": "The MIT License"
+    },
+    {
+      "moduleLicense": "The MIT license"
+    },
+    {
+      "moduleLicense": "Mozilla Public License Version 2.0"
+    },
+    {
+      "moduleLicense": "Public Domain"
+    },
+    {
+      "moduleLicense": "PUBLIC DOMAIN"
+    },
+    {
+      "moduleLicense": "The W3C Software License"
+    }
+  ]
+}
@@ -0,0 +1,32 @@
+{
+  "bundles" : [
+    {
+      "bundleName" : "Apache 2.0",
+      "licenseName" : "Apache 2.0",
+      "licenseUrl" : "http://www.apache.org/licenses/LICENSE-2.0"
+    },
+    {
+      "bundleName" : "MIT License",
+      "licenseName" : "MIT License",
+      "licenseUrl" : "http://www.opensource.org/licenses/mit-license.php"
+    }
+  ],
+  "transformationRules" : [
+    {
+      "bundleName" : "Apache 2.0",
+      "licenseUrlPattern" : "http://www.apache.org/licenses/LICENSE-2.0"
+    },
+    {
+      "bundleName" : "Apache 2.0",
+      "licenseUrlPattern" : "http://www.apache.org/licenses/LICENSE-2.0.txt"
+    },
+    {
+      "bundleName" : "MIT License",
+      "licenseUrlPattern" : "http://www.opensource.org/licenses/mit-license.php"
+    },
+    {
+      "bundleName" : "MIT License",
+      "licenseUrlPattern" : "http://www.opensource.org/licenses/MIT"
+    }
+  ]
+}
@@ -82,6 +82,11 @@ PRESUBMITS = {
        }, REQUIRED):
        "File did not include the license header.",

+    # Files must end in a newline
+    PresubmitCheck(r".*\n$", ("java", "js", "soy", "sql", "py", "sh", "gradle"),
+                   {"node_modules/"}, REQUIRED):
+        "Source files must end in a newline.",
+
    # System.(out|err).println should only appear in tools/
    PresubmitCheck(
        r".*\bSystem\.(out|err)\.print", "java", {
@@ -158,7 +163,6 @@ PRESUBMITS = {


 def get_files():
-  result = []
  for root, dirnames, filenames in os.walk("."):
    for filename in filenames:
      yield os.path.join(root, filename)
@@ -0,0 +1,3 @@
+FROM gcr.io/distroless/java:debug
+ADD build/libs/nomulus.jar /nomulus.jar
+ENTRYPOINT ["/usr/bin/java", "-jar", "/nomulus.jar"]
@@ -25,6 +25,7 @@ def screenshotsForGoldensDir = "${project.buildDir}/screenshots_for_goldens"
 def newGoldensDir = "${project.buildDir}/new_golden_images"
 def goldensDir =
  "${javaTestDir}/google/registry/webdriver/goldens/chrome-linux"
+def jsDir = "${project.projectDir}/src/main/javascript"

 // Tests that conflict with (mostly unidentified) members of the main test
 // suite. It is unclear if they are offenders (i.e., those that pollute global
@@ -96,14 +97,14 @@ sourceSets {
  }
 }

+processTestResources {
+  exclude '**/webdriver/*'
+}
+
 configurations {
  css
  jaxb
  soy
-  // Label for all dependencies inherited from Bazel build but not used in
-  // either compile or testRuntime. However, they may be needed at runtime.
-  // TODO(weiminyu): identify runtime dependencies and remove the rest.
-  maybeRuntime
  closureCompiler

  // Exclude non-canonical servlet-api jars. Our AppEngine deployment uses
@@ -139,53 +140,38 @@ dependencies {
      "${rootDir}/third_party/objectify/v4_1/objectify-4.1.3.jar")
  testImplementation project(':third_party')

+  testRuntime files(sourceSets.test.resources.srcDirs)
+
  compile deps['com.beust:jcommander']
  compile deps['com.google.api-client:google-api-client']
-  maybeRuntime deps['com.google.api-client:google-api-client-appengine']
-  maybeRuntime deps['com.google.api-client:google-api-client-jackson2']
  compile deps['com.google.monitoring-client:metrics']
  compile deps['com.google.monitoring-client:stackdriver']
  compile deps['com.google.api-client:google-api-client-java6']
-  maybeRuntime deps['com.google.api-client:google-api-client-servlet']
  compile deps['com.google.apis:google-api-services-admin-directory']
  compile deps['com.google.apis:google-api-services-appengine']
  compile deps['com.google.apis:google-api-services-bigquery']
-  maybeRuntime deps['com.google.apis:google-api-services-clouddebugger']
  compile deps['com.google.apis:google-api-services-cloudkms']
-  maybeRuntime deps['com.google.apis:google-api-services-cloudresourcemanager']
  compile deps['com.google.apis:google-api-services-dataflow']
  compile deps['com.google.apis:google-api-services-dns']
  compile deps['com.google.apis:google-api-services-drive']
  compile deps['com.google.apis:google-api-services-groupssettings']
  compile deps['com.google.apis:google-api-services-monitoring']
  compile deps['com.google.apis:google-api-services-sheets']
-  maybeRuntime deps['com.google.apis:google-api-services-storage']
  testCompileOnly deps['com.google.appengine:appengine-api-1.0-sdk']
-  maybeRuntime deps['com.google.appengine:appengine-api-labs']
-  maybeRuntime deps['com.google.appengine:appengine-api-stubs']
  testCompile deps['com.google.appengine:appengine-api-stubs']
  compile deps['com.google.appengine.tools:appengine-gcs-client']
  compile deps['com.google.appengine.tools:appengine-mapreduce']
  compile deps['com.google.appengine.tools:appengine-pipeline']
  compile deps['com.google.appengine:appengine-remote-api']
-  maybeRuntime deps['com.google.appengine:appengine-tools-sdk']
  compile deps['com.google.auth:google-auth-library-credentials']
  compile deps['com.google.auth:google-auth-library-oauth2-http']
-  maybeRuntime deps['com.google.auto:auto-common']
-  maybeRuntime deps['com.google.auto.factory:auto-factory']
  compile deps['com.google.code.gson:gson']
  compile deps['com.google.auto.value:auto-value-annotations']
-  maybeRuntime deps['com.google.cloud.bigdataoss:gcsio']
-  maybeRuntime deps['com.google.cloud.bigdataoss:util']
  compile deps['com.google.code.findbugs:jsr305']
  compile deps['com.google.dagger:dagger']
-  maybeRuntime deps['com.google.dagger:dagger-producers']
  compile deps['com.google.errorprone:error_prone_annotations']
-  maybeRuntime deps['com.google.errorprone:javac-shaded']
  compile deps['com.google.flogger:flogger']
  runtime deps['com.google.flogger:flogger-system-backend']
-  maybeRuntime deps['com.google.gdata:core']
-  maybeRuntime deps['com.google.googlejavaformat:google-java-format']
  compile deps['com.google.guava:guava']
  gradleLint.ignore('unused-dependency') {
    compile deps['com.google.gwt:gwt-user']
@@ -194,89 +180,48 @@ dependencies {
  compile deps['com.google.http-client:google-http-client-appengine']
  compile deps['com.google.http-client:google-http-client-jackson2']
  compile deps['com.google.oauth-client:google-oauth-client']
-  maybeRuntime deps['com.google.oauth-client:google-oauth-client-appengine']
  compile deps['com.google.oauth-client:google-oauth-client-java6']
  compile deps['com.google.oauth-client:google-oauth-client-jetty']
-  maybeRuntime deps['com.google.oauth-client:google-oauth-client-servlet']
-  maybeRuntime deps['com.google.protobuf:protobuf-java']
  compile deps['com.google.re2j:re2j']
  compile deps['com.google.template:soy']
-  maybeRuntime deps['com.googlecode.charts4j:charts4j']
  compile deps['com.googlecode.json-simple:json-simple']
  compile deps['com.jcraft:jsch']
-  maybeRuntime deps['com.jcraft:jzlib']
-  maybeRuntime deps['com.squareup:javapoet']
-  maybeRuntime deps['com.squareup:javawriter']
-  maybeRuntime deps['com.sun.activation:javax.activation']
-  maybeRuntime deps['com.thoughtworks.paranamer:paranamer']
  testCompile deps['com.thoughtworks.qdox:qdox']
-  maybeRuntime deps['commons-codec:commons-codec']
-  maybeRuntime deps['commons-logging:commons-logging']
  compile deps['dnsjava:dnsjava']
-  maybeRuntime deps['io.netty:netty-buffer']
-  maybeRuntime deps['io.netty:netty-codec']
-  maybeRuntime deps['io.netty:netty-codec-http']
-  maybeRuntime deps['io.netty:netty-common']
-  maybeRuntime deps['io.netty:netty-handler']
-  maybeRuntime deps['io.netty:netty-resolver']
-  maybeRuntime deps['io.netty:netty-tcnative']
-  maybeRuntime deps['io.netty:netty-tcnative-boringssl-static']
-  maybeRuntime deps['io.netty:netty-transport']
-  maybeRuntime deps['it.unimi.dsi:fastutil']
-  maybeRuntime deps['javax.annotation:jsr250-api']
  runtime deps['org.glassfish.jaxb:jaxb-runtime']
  testCompile deps['javax.annotation:jsr250-api']
  compile deps['javax.inject:javax.inject']
  compile deps['javax.mail:mail']
  compile deps['javax.servlet:servlet-api']
  compile deps['javax.xml.bind:jaxb-api']
-  maybeRuntime deps['javax.xml.soap:javax.xml.soap-api']
  compile deps['jline:jline']
  compile deps['joda-time:joda-time']
  compile deps['org.apache.avro:avro']
-  maybeRuntime deps['org.apache.beam:beam-runners-direct-java']
  testCompile deps['org.apache.beam:beam-runners-direct-java']
  compile deps['org.apache.beam:beam-runners-google-cloud-dataflow-java']
-  maybeRuntime deps['org.apache.beam:beam-sdks-common-runner-api']
  compile deps['org.apache.beam:beam-sdks-java-core']
  compile deps['org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core']
  compile deps['org.apache.beam:beam-sdks-java-io-google-cloud-platform']
-  maybeRuntime deps['org.apache.commons:commons-compress']
  testCompile deps['org.apache.commons:commons-text']
-  maybeRuntime deps['org.apache.ftpserver:ftplet-api']
  testCompile deps['org.apache.ftpserver:ftplet-api']
-  maybeRuntime deps['org.apache.ftpserver:ftpserver-core']
  testCompile deps['org.apache.ftpserver:ftpserver-core']
  compile deps['org.apache.httpcomponents:httpclient']
  compile deps['org.apache.httpcomponents:httpcore']
-  maybeRuntime deps['org.apache.mina:mina-core']
-  maybeRuntime deps['org.apache.sshd:sshd-core']
  testCompile deps['org.apache.sshd:sshd-core']
-  maybeRuntime deps['org.apache.sshd:sshd-scp']
  testCompile deps['org.apache.sshd:sshd-scp']
-  maybeRuntime deps['org.apache.sshd:sshd-sftp']
  testCompile deps['org.apache.sshd:sshd-sftp']
-  maybeRuntime deps['org.apache.tomcat:tomcat-annotations-api']
  testCompile deps['org.apache.tomcat:tomcat-annotations-api']
  compile deps['org.bouncycastle:bcpg-jdk15on']
  testCompile deps['org.bouncycastle:bcpkix-jdk15on']
  compile deps['org.bouncycastle:bcprov-jdk15on']
-  maybeRuntime deps['org.codehaus.jackson:jackson-core-asl']
-  maybeRuntime deps['org.codehaus.jackson:jackson-mapper-asl']
  compile deps['org.joda:joda-money']
  compile deps['org.json:json']
-  maybeRuntime deps['org.khronos:opengl-api']
-  maybeRuntime deps['org.mortbay.jetty:jetty']
  testCompile deps['org.mortbay.jetty:jetty']
-  maybeRuntime deps['org.mortbay.jetty:jetty-util']
  testCompile deps['org.seleniumhq.selenium:selenium-api']
  testCompile deps['org.seleniumhq.selenium:selenium-chrome-driver']
  testCompile deps['org.seleniumhq.selenium:selenium-java']
  testCompile deps['org.seleniumhq.selenium:selenium-remote-driver']
-  maybeRuntime deps['org.slf4j:slf4j-api']
  testCompile deps['org.testcontainers:selenium']
-  maybeRuntime deps['org.tukaani:xz']
-  maybeRuntime deps['org.xerial.snappy:snappy-java']
  compile deps['xerces:xmlParserAPIs']
  compile deps['xpp3:xpp3']

@@ -480,7 +425,7 @@ task soyToJS {
 }

 task stylesheetsToJavascript {
-  def cssSourceDir = "${javaDir}/google/registry/ui/css"
+  def cssSourceDir = "${jsDir}/google/registry/ui/css"
  def outputDir = "${resourcesDir}/google/registry/ui/css"
  inputs.dir cssSourceDir
  outputs.dir outputDir
@@ -533,8 +478,8 @@ task compileProdJS(type: JavaExec) {
  def outputDir = "${resourcesDir}/google/registry/ui"
  def nodeModulesDir = "${rootDir}/node_modules"
  def cssSourceDir = "${resourcesDir}/google/registry/ui/css"
-  def jsSourceDir = "${javaDir}/google/registry/ui/js"
-  def externsDir = "${javaDir}/google/registry/ui/externs"
+  def jsSourceDir = "${jsDir}/google/registry/ui/js"
+  def externsDir = "${jsDir}/google/registry/ui/externs"
  def soySourceDir = "${generatedDir}/google/registry/ui/soy"

  [nodeModulesDir, cssSourceDir, jsSourceDir, externsDir, soySourceDir].each {
@@ -700,10 +645,11 @@ test {
 }.dependsOn(fragileTest, outcastTest)

 createUberJar('nomulus', 'nomulus', 'google.registry.tools.RegistryTool')
-createUberJar('gtechTool', 'gtech_tool', 'google.registry.tools.GtechTool')
 project.nomulus.dependsOn project(':third_party').jar
-project.gtechTool.dependsOn project(':third_party').jar

-project.build.dependsOn nomulus
-project.build.dependsOn gtechTool
+task buildToolImage(dependsOn: nomulus, type: Exec) {
+  commandLine 'docker', 'build', '-t', 'nomulus-tool', '.'
+}
+
+project.build.dependsOn buildToolImage
 project.build.dependsOn ':stage'
@@ -14,6 +14,7 @@

 package google.registry.beam.invoicing;

+import com.google.auth.oauth2.GoogleCredentials;
 import google.registry.beam.invoicing.BillingEvent.InvoiceGroupingKey;
 import google.registry.beam.invoicing.BillingEvent.InvoiceGroupingKey.InvoiceGroupingKeyCoder;
 import google.registry.config.CredentialModule.LocalCredential;
@@ -57,35 +58,31 @@ import org.apache.beam.sdk.values.TypeDescriptors;
 */
 public class InvoicingPipeline implements Serializable {

-  @Inject
-  @Config("projectId")
-  String projectId;
+  private final String projectId;
+  private final String beamBucketUrl;
+  private final String invoiceTemplateUrl;
+  private final String beamStagingUrl;
+  private final String billingBucketUrl;
+  private final String invoiceFilePrefix;
+  private final GoogleCredentials googleCredentials;

  @Inject
-  @Config("apacheBeamBucketUrl")
-  String beamBucketUrl;
-
-  @Inject
-  @Config("invoiceTemplateUrl")
-  String invoiceTemplateUrl;
-
-  @Inject
-  @Config("beamStagingUrl")
-  String beamStagingUrl;
-
-  @Inject
-  @Config("billingBucketUrl")
-  String billingBucketUrl;
-
-  @Inject
-  @Config("invoiceFilePrefix")
-  String invoiceFilePrefix;
-
-  @Inject @LocalCredential
-  GoogleCredentialsBundle credentialsBundle;
-
-  @Inject
-  InvoicingPipeline() {}
+  public InvoicingPipeline(
+      @Config("projectId") String projectId,
+      @Config("apacheBeamBucketUrl") String beamBucketUrl,
+      @Config("invoiceTemplateUrl") String invoiceTemplateUrl,
+      @Config("beamStagingUrl") String beamStagingUrl,
+      @Config("billingBucketUrl") String billingBucketUrl,
+      @Config("invoiceFilePrefix") String invoiceFilePrefix,
+      @LocalCredential GoogleCredentialsBundle googleCredentialsBundle) {
+    this.projectId = projectId;
+    this.beamBucketUrl = beamBucketUrl;
+    this.invoiceTemplateUrl = invoiceTemplateUrl;
+    this.beamStagingUrl = beamStagingUrl;
+    this.billingBucketUrl = billingBucketUrl;
+    this.invoiceFilePrefix = invoiceFilePrefix;
+    this.googleCredentials = googleCredentialsBundle.getGoogleCredentials();
+  }

  /** Custom options for running the invoicing pipeline. */
  interface InvoicingPipelineOptions extends DataflowPipelineOptions {
@@ -105,12 +102,15 @@ public class InvoicingPipeline implements Serializable {
  public void deploy() {
    // We can't store options as a member variable due to serialization concerns.
    InvoicingPipelineOptions options = PipelineOptionsFactory.as(InvoicingPipelineOptions.class);
-    options.setGcpCredential(credentialsBundle.getGoogleCredentials());
    options.setProject(projectId);
    options.setRunner(DataflowRunner.class);
    // This causes p.run() to stage the pipeline as a template on GCS, as opposed to running it.
    options.setTemplateLocation(invoiceTemplateUrl);
    options.setStagingLocation(beamStagingUrl);
+    // This credential is used when Dataflow deploys the template to GCS in target GCP project.
+    // So, make sure the credential has write permission to GCS in that project.
+    options.setGcpCredential(googleCredentials);
+
    Pipeline p = Pipeline.create(options);

    PCollection<BillingEvent> billingEvents =
@@ -17,9 +17,12 @@ package google.registry.beam.spec11;
 import static com.google.common.base.Preconditions.checkArgument;
 import static google.registry.beam.BeamUtils.getQueryFromFile;

+import com.google.auth.oauth2.GoogleCredentials;
 import com.google.auto.value.AutoValue;
 import google.registry.beam.spec11.SafeBrowsingTransforms.EvaluateSafeBrowsingFn;
+import google.registry.config.CredentialModule.LocalCredential;
 import google.registry.config.RegistryConfig.Config;
+import google.registry.util.GoogleCredentialsBundle;
 import google.registry.util.Retrier;
 import google.registry.util.SqlTemplate;
 import java.io.Serializable;
@@ -77,26 +80,29 @@ public class Spec11Pipeline implements Serializable {
  /** The JSON object field we put the threat match array for Spec11 reports. */
  public static final String THREAT_MATCHES_FIELD = "threatMatches";

-  @Inject
-  @Config("projectId")
-  String projectId;
+  private final String projectId;
+  private final String beamStagingUrl;
+  private final String spec11TemplateUrl;
+  private final String reportingBucketUrl;
+  private final GoogleCredentials googleCredentials;
+  private final Retrier retrier;

  @Inject
-  @Config("beamStagingUrl")
-  String beamStagingUrl;
-
-  @Inject
-  @Config("spec11TemplateUrl")
-  String spec11TemplateUrl;
-
-  @Inject
-  @Config("reportingBucketUrl")
-  String reportingBucketUrl;
-
-  @Inject Retrier retrier;
-
-  @Inject
-  Spec11Pipeline() {}
+  public Spec11Pipeline(
+      @Config("projectId") String projectId,
+      @Config("beamStagingUrl") String beamStagingUrl,
+      @Config("spec11TemplateUrl") String spec11TemplateUrl,
+      @Config("reportingBucketUrl") String reportingBucketUrl,
+      @LocalCredential GoogleCredentialsBundle googleCredentialsBundle,
+      Retrier retrier
+  ) {
+    this.projectId = projectId;
+    this.beamStagingUrl = beamStagingUrl;
+    this.spec11TemplateUrl = spec11TemplateUrl;
+    this.reportingBucketUrl = reportingBucketUrl;
+    this.googleCredentials = googleCredentialsBundle.getGoogleCredentials();
+    this.retrier = retrier;
+  }

  /** Custom options for running the spec11 pipeline. */
  interface Spec11PipelineOptions extends DataflowPipelineOptions {
@@ -134,6 +140,9 @@ public class Spec11Pipeline implements Serializable {
    // This causes p.run() to stage the pipeline as a template on GCS, as opposed to running it.
    options.setTemplateLocation(spec11TemplateUrl);
    options.setStagingLocation(beamStagingUrl);
+    // This credential is used when Dataflow deploys the template to GCS in target GCP project.
+    // So, make sure the credential has write permission to GCS in that project.
+    options.setGcpCredential(googleCredentials);

    Pipeline p = Pipeline.create(options);
    PCollection<Subdomain> domains =
@@ -112,8 +112,6 @@
    <target>backend</target>
  </cron>

-  <!--
-    TODO(b/134576418) enable this cron job once we're sure the Action works
  <cron>
    <url><![CDATA[/_dr/task/updateRegistrarRdapBaseUrls]]></url>
    <description>
@@ -122,7 +120,6 @@
    <schedule>every day 02:34</schedule>
    <target>backend</target>
  </cron>
-  -->

  <cron>
    <url><![CDATA[/_dr/task/deleteOldCommitLogs]]></url>
@@ -39,6 +39,13 @@ public class EppMetrics {
          LabelDescriptor.create("tld", "The TLD acted on by the command (if applicable)."),
          LabelDescriptor.create("status", "The return status of the command."));

+  private static final ImmutableSet<LabelDescriptor> LABEL_DESCRIPTORS =
+      ImmutableSet.of(
+          LabelDescriptor.create("command", "The name of the command."),
+          LabelDescriptor.create("traffic_type",
+              "The traffic type of the command; one of CANARY, PROBER, or REAL."),
+          LabelDescriptor.create("status", "The return status of the command."));
+
  private static final IncrementableMetric eppRequestsByRegistrar =
      MetricRegistryImpl.getDefault()
          .newIncrementableMetric(
@@ -73,6 +80,19 @@ public class EppMetrics {
              LABEL_DESCRIPTORS_BY_TLD,
              DEFAULT_FITTER);

+  private static final EventMetric requestTime =
+      MetricRegistryImpl.getDefault()
+          .newEventMetric(
+              "/epp/request_time",
+              "EPP Request Time",
+              "milliseconds",
+              LABEL_DESCRIPTORS,
+              DEFAULT_FITTER);
+
+  private enum TrafficType {
+    CANARY, PROBER, REAL
+  }
+
  @Inject
  public EppMetrics() {}

@@ -97,15 +117,21 @@ public class EppMetrics {
        metric.getStatus().isPresent() ? String.valueOf(metric.getStatus().get().code) : "";
    long processingTime =
        metric.getEndTimestamp().getMillis() - metric.getStartTimestamp().getMillis();
-    processingTimeByRegistrar.record(
-        processingTime,
-        metric.getCommandName().orElse(""),
-        metric.getClientId().orElse(""),
-        eppStatusCode);
-    processingTimeByTld.record(
-        processingTime,
-        metric.getCommandName().orElse(""),
-        metric.getTld().orElse(""),
-        eppStatusCode);
+    String commandName = metric.getCommandName().orElse("");
+    processingTimeByRegistrar
+        .record(processingTime, commandName, metric.getClientId().orElse(""), eppStatusCode);
+    String tld = metric.getTld().orElse("");
+    processingTimeByTld.record(processingTime, commandName, tld, eppStatusCode);
+    requestTime.record(processingTime, commandName, getTrafficType(tld).toString(), eppStatusCode);
+  }
+
+  private static TrafficType getTrafficType(String tld) {
+    if (tld.endsWith("canary.test")) {
+      return TrafficType.CANARY;
+    } else if (tld.endsWith(".test")) {
+      return TrafficType.PROBER;
+    } else {
+      return TrafficType.REAL;
+    }
  }
 }
@@ -28,6 +28,7 @@ import static google.registry.util.CollectionUtils.forceEmptyToNull;
 import static google.registry.util.CollectionUtils.nullToEmpty;
 import static google.registry.util.CollectionUtils.nullToEmptyImmutableCopy;
 import static google.registry.util.CollectionUtils.union;
+import static google.registry.util.DateTimeUtils.END_OF_TIME;
 import static google.registry.util.DateTimeUtils.earliestOf;
 import static google.registry.util.DateTimeUtils.isBeforeOrAt;
 import static google.registry.util.DateTimeUtils.leapSafeAddYears;
@@ -376,10 +377,10 @@ public class DomainBase extends EppResource

    Optional<DateTime> newLastEppUpdateTime = Optional.empty();

-    // There is no transfer. Do any necessary autorenews.
+    // There is no transfer. Do any necessary autorenews for active domains.

    Builder builder = asBuilder();
-    if (isBeforeOrAt(registrationExpirationTime, now)) {
+    if (isBeforeOrAt(registrationExpirationTime, now) && END_OF_TIME.equals(getDeletionTime())) {
      // Autorenew by the number of years between the old expiration time and now.
      DateTime lastAutorenewTime = leapSafeAddYears(
          registrationExpirationTime,
@@ -15,9 +15,9 @@
 package google.registry.model.domain.fee;

 import com.google.common.base.Ascii;
-import com.google.common.base.CharMatcher;
 import google.registry.model.ImmutableObject;
 import google.registry.model.domain.fee.FeeQueryCommandExtensionItem.CommandName;
+import java.util.Locale;
 import javax.xml.bind.annotation.XmlAttribute;
 import javax.xml.bind.annotation.XmlValue;

@@ -47,7 +47,7 @@ public class FeeExtensionCommandDescriptor extends ImmutableObject {

  public CommandName getCommand() {
    // Require the xml string to be lowercase.
-    if (command != null && CharMatcher.javaLowerCase().matchesAllOf(command)) {
+    if (command != null && command.toLowerCase(Locale.ENGLISH).equals(command)) {
      try {
        return CommandName.valueOf(Ascii.toUpperCase(command));
      } catch (IllegalArgumentException e) {
@@ -15,9 +15,9 @@
 package google.registry.model.domain.fee12;

 import com.google.common.base.Ascii;
-import com.google.common.base.CharMatcher;
 import google.registry.model.domain.Period;
 import google.registry.model.domain.fee.FeeCheckCommandExtensionItem;
+import java.util.Locale;
 import java.util.Optional;
 import javax.xml.bind.annotation.XmlAttribute;
 import javax.xml.bind.annotation.XmlElement;
@@ -83,7 +83,7 @@ public class FeeCheckCommandExtensionItemV12 extends FeeCheckCommandExtensionIte
  @Override
  public CommandName getCommandName() {
    // Require the xml string to be lowercase.
-    if (commandName != null && CharMatcher.javaLowerCase().matchesAllOf(commandName)) {
+    if (commandName != null && commandName.toLowerCase(Locale.ENGLISH).equals(commandName)) {
      try {
        return CommandName.valueOf(Ascii.toUpperCase(commandName));
      } catch (IllegalArgumentException e) {
@@ -868,7 +868,9 @@ public class Registrar extends ImmutableObject implements Buildable, Jsonifiable
      return this;
    }

-    public Builder setDriveFolderId(String driveFolderId) {
+    public Builder setDriveFolderId(@Nullable String driveFolderId) {
+      checkArgument(driveFolderId == null || !driveFolderId.contains("/"),
+          "Drive folder ID must not be a full URL");
      getInstance().driveFolderId = driveFolderId;
      return this;
    }
@@ -14,6 +14,7 @@

 package google.registry.rdap;

+import static com.google.common.base.Preconditions.checkArgument;
 import static com.google.common.base.Preconditions.checkState;
 import static google.registry.model.ofy.ObjectifyService.ofy;
 import static java.nio.charset.StandardCharsets.UTF_8;
@@ -23,6 +24,7 @@ import com.google.api.client.http.HttpRequest;
 import com.google.api.client.http.HttpRequestFactory;
 import com.google.api.client.http.HttpResponse;
 import com.google.api.client.http.HttpTransport;
+import com.google.common.collect.ImmutableList;
 import com.google.common.collect.ImmutableSet;
 import com.google.common.collect.ImmutableSetMultimap;
 import com.google.common.flogger.FluentLogger;
@@ -34,8 +36,9 @@ import com.google.gson.JsonObject;
 import com.googlecode.objectify.Key;
 import google.registry.keyring.api.KeyModule;
 import google.registry.model.registrar.Registrar;
+import google.registry.model.registry.Registries;
+import google.registry.model.registry.Registry.TldType;
 import google.registry.request.Action;
-import google.registry.request.Parameter;
 import google.registry.request.auth.Auth;
 import java.io.IOException;
 import java.io.InputStream;
@@ -56,8 +59,6 @@ import javax.inject.Inject;
 * <p>It is a "login/query/logout" system where you login using the ICANN Reporting credentials, get
 * a cookie you then send to get the list and finally logout.
 *
- * <p>The username is [TLD]_ry. It could be any "real" TLD.
- *
 * <p>For clarity, this is how one would contact this endpoint "manually", from a whitelisted IP
 * server:
 *
@@ -88,18 +89,10 @@ public final class UpdateRegistrarRdapBaseUrlsAction implements Runnable {
  @Inject HttpTransport httpTransport;
  @Inject @KeyModule.Key("icannReportingPassword") String password;

-  /**
-   * The TLD for which we make the request.
-   *
-   * <p>The actual value doesn't matter, as long as it's a TLD that has access to the ICANN
-   * Reporter. It's just used to login.
-   */
-  @Inject @Parameter("tld") String tld;
-
  @Inject
  UpdateRegistrarRdapBaseUrlsAction() {}

-  private String loginAndGetId(HttpRequestFactory requestFactory) {
+  private String loginAndGetId(HttpRequestFactory requestFactory, String tld) {
    try {
      logger.atInfo().log("Logging in to MoSAPI");
      HttpRequest request =
@@ -122,7 +115,7 @@ public final class UpdateRegistrarRdapBaseUrlsAction implements Runnable {
    }
  }

-  private void logout(HttpRequestFactory requestFactory, String id) {
+  private void logout(HttpRequestFactory requestFactory, String id, String tld) {
    try {
      HttpRequest request =
          requestFactory.buildGetRequest(new GenericUrl(String.format(LOGOUT_URL, tld)));
@@ -135,9 +128,9 @@ public final class UpdateRegistrarRdapBaseUrlsAction implements Runnable {
    }
  }

-  private ImmutableSetMultimap<String, String> getRdapBaseUrlsPerIanaId() {
+  private ImmutableSetMultimap<String, String> getRdapBaseUrlsPerIanaIdWithTld(String tld) {
    HttpRequestFactory requestFactory = httpTransport.createRequestFactory();
-    String id = loginAndGetId(requestFactory);
+    String id = loginAndGetId(requestFactory, tld);
    String content;
    try {
      HttpRequest request =
@@ -152,7 +145,7 @@ public final class UpdateRegistrarRdapBaseUrlsAction implements Runnable {
      throw new UncheckedIOException(
          "Error reading RDAP list from MoSAPI server: " + e.getMessage(), e);
    } finally {
-      logout(requestFactory, id);
+      logout(requestFactory, id, tld);
    }

    logger.atInfo().log("list reply: '%s'", content);
@@ -173,6 +166,25 @@ public final class UpdateRegistrarRdapBaseUrlsAction implements Runnable {
    return builder.build();
  }

+  private ImmutableSetMultimap<String, String> getRdapBaseUrlsPerIanaId() {
+    // All TLDs have the same data, so just keep trying until one works
+    // (the expectation is that all / any should work)
+    ImmutableList<String> tlds = ImmutableList.sortedCopyOf(Registries.getTldsOfType(TldType.REAL));
+    checkArgument(!tlds.isEmpty(), "There must exist at least one REAL TLD.");
+    Throwable finalThrowable = null;
+    for (String tld : tlds) {
+      try {
+        return getRdapBaseUrlsPerIanaIdWithTld(tld);
+      } catch (Throwable throwable) {
+        logger.atWarning().log(String
+            .format("Error retrieving RDAP urls with TLD %s: %s", tld, throwable.getMessage()));
+        finalThrowable = throwable;
+      }
+    }
+    throw new RuntimeException(
+        String.format("Error contacting MosAPI server. Tried TLDs %s", tlds), finalThrowable);
+  }
+
  @Override
  public void run() {
    ImmutableSetMultimap<String, String> ianaToBaseUrls = getRdapBaseUrlsPerIanaId();
@@ -22,6 +22,7 @@ import static javax.servlet.http.HttpServletResponse.SC_OK;
 import com.google.appengine.tools.cloudstorage.GcsFilename;
 import com.google.common.base.Splitter;
 import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableMap;
 import com.google.common.collect.Iterables;
 import com.google.common.flogger.FluentLogger;
 import com.google.common.io.ByteStreams;
@@ -38,6 +39,7 @@ import google.registry.util.Retrier;
 import java.io.IOException;
 import java.io.InputStream;
 import java.util.Optional;
+import java.util.stream.Collectors;
 import javax.inject.Inject;

 /** Copy all registrar detail reports in a given bucket's subdirectory from GCS to Drive. */
@@ -95,6 +97,8 @@ public final class CopyDetailReportsAction implements Runnable {
      response.setPayload(String.format("Failure, encountered %s", e.getMessage()));
      return;
    }
+    ImmutableMap.Builder<String, Throwable> copyErrorsBuilder =
+        new ImmutableMap.Builder<String, Throwable>();
    for (String detailReportName : detailReportObjectNames) {
      // The standard report format is "invoice_details_yyyy-MM_registrarId_tld.csv
      // TODO(larryruili): Determine a safer way of enforcing this.
@@ -117,7 +121,7 @@ public final class CopyDetailReportsAction implements Runnable {
              try (InputStream input =
                  gcsUtils.openInputStream(
                      new GcsFilename(billingBucket, invoiceDirectoryPrefix + detailReportName))) {
-                driveConnection.createFile(
+                driveConnection.createOrUpdateFile(
                    detailReportName,
                    MediaType.CSV_UTF_8,
                    driveFolderId,
@@ -129,15 +133,31 @@ public final class CopyDetailReportsAction implements Runnable {
            },
            IOException.class);
      } catch (Throwable e) {
-        emailUtils.sendAlertEmail(
+        String alertMessage =
            String.format(
-                "Warning: CopyDetailReportsAction failed.\nEncountered: %s on file: %s",
-                getRootCause(e).getMessage(), detailReportName));
-        throw e;
+                "Warning: CopyDetailReportsAction failed for registrar %s.\n"
+                    + "Encountered: %s on file: %s",
+                registrarId, getRootCause(e).getMessage(), detailReportName);
+        copyErrorsBuilder.put(registrarId, e);
+        logger.atSevere().withCause(e).log(alertMessage);
      }
    }
    response.setStatus(SC_OK);
    response.setContentType(MediaType.PLAIN_TEXT_UTF_8);
-    response.setPayload("Copied detail reports.");
+    StringBuilder payload = new StringBuilder().append("Copied detail reports.\n");
+    ImmutableMap<String, Throwable> copyErrors = copyErrorsBuilder.build();
+    if (!copyErrors.isEmpty()) {
+      payload.append("The following errors were encountered:\n");
+      payload.append(
+          copyErrors.entrySet().stream()
+              .map(
+                  entrySet ->
+                      String.format(
+                          "Registrar: %s\nError: %s\n",
+                          entrySet.getKey(), entrySet.getValue().getMessage()))
+              .collect(Collectors.joining()));
+    }
+    response.setPayload(payload.toString());
+    emailUtils.sendAlertEmail(payload.toString());
  }
 }
@@ -54,6 +54,10 @@ public class DriveConnection {
  /**
   * Creates a file with the given parent.
   *
+   * <p>If a file with the same path already exists, a duplicate is created. If overwriting the
+   * existing file is the desired behavior, use {@link #createOrUpdateFile(String, MediaType,
+   * String, byte[])} instead.
+   *
   * @returns the file id.
   */
  public String createFile(String title, MediaType mimeType, String parentFolderId, byte[] bytes)
@@ -100,7 +100,7 @@ public class AuthModule {
      AbstractDataStoreFactory dataStoreFactory) {
    try {
      return new GoogleAuthorizationCodeFlow.Builder(
-              new NetHttpTransport(), jsonFactory, clientSecrets, requiredOauthScopes)
+          new NetHttpTransport(), jsonFactory, clientSecrets, requiredOauthScopes)
          .setDataStoreFactory(dataStoreFactory)
          .build();
    } catch (IOException ex) {
@@ -237,7 +237,7 @@ abstract class CreateOrUpdateRegistrarCommand extends MutatingCommand {
  @Nullable
  @Parameter(
      names = "--drive_folder_id",
-      description = "Id of this registrar's folder in Drive",
+      description = "Id (not full URL) of this registrar's folder in Drive",
      converter = OptionalStringParameter.class,
      validateWith = OptionalStringParameter.class)
  Optional<String> driveFolderId;
@@ -19,7 +19,6 @@ import static google.registry.util.CollectionUtils.findDuplicates;
 import static google.registry.util.DomainNameUtils.canonicalizeDomainName;

 import com.beust.jcommander.Parameter;
-import com.google.common.base.CharMatcher;
 import com.google.common.base.Joiner;
 import com.google.common.collect.ImmutableList;
 import com.google.common.collect.ImmutableSet;
@@ -255,7 +254,7 @@ abstract class CreateOrUpdateTldCommand extends MutatingCommand {
          tld,
          canonicalizeDomainName(tld));
      checkArgument(
-          !CharMatcher.javaDigit().matches(tld.charAt(0)),
+          !Character.isDigit(tld.charAt(0)),
          "TLDs cannot begin with a number");
      Registry oldRegistry = getOldRegistry(tld);
      // TODO(b/26901539): Add a flag to set the pricing engine once we have more than one option.
@@ -1,8 +0,0 @@
-# -*-protobuf-*-
-
-requirement: {
-  type: BANNED_PROPERTY_WRITE
-  error_message: 'Assignment to Element.prototype.innerHTML is not allowed. '
-      'Use goog.dom.safe.setInnerHtml instead. '
-  value: 'Element.prototype.innerHTML'
-}
@@ -240,12 +240,12 @@
        {param required: true /}
      {/call}
      {call registry.soy.forms.inputFieldRowWithValue}
-        {param label: 'DRIVE ID' /}
+        {param label: 'Drive ID' /}
        {param name: 'driveId' /}
        {param value: $driveId /}
        {param placeholder: 'required' /}
        {param description kind="text"}
-          Id of this registrar's folder in Drive.
+          ID (not full URL) of this registrar's folder in Google Drive.
        {/param}
        {param required: true /}
      {/call}
@@ -16,8 +16,10 @@ package google.registry.beam.invoicing;

 import static com.google.common.truth.Truth.assertThat;

+import com.google.auth.oauth2.GoogleCredentials;
 import com.google.common.collect.ImmutableList;
 import com.google.common.collect.ImmutableMap;
+import google.registry.util.GoogleCredentialsBundle;
 import google.registry.util.ResourceUtils;
 import java.io.File;
 import java.io.IOException;
@@ -58,15 +60,17 @@ public class InvoicingPipelineTest {

  @Before
  public void initializePipeline() throws IOException {
-    invoicingPipeline = new InvoicingPipeline();
-    invoicingPipeline.projectId = "test-project";
    File beamTempFolder = tempFolder.newFolder();
-    invoicingPipeline.beamBucketUrl = beamTempFolder.getAbsolutePath();
-    invoicingPipeline.invoiceFilePrefix = "REG-INV";
-    invoicingPipeline.beamStagingUrl = beamTempFolder.getAbsolutePath() + "/staging";
-    invoicingPipeline.invoiceTemplateUrl =
-        beamTempFolder.getAbsolutePath() + "/templates/invoicing";
-    invoicingPipeline.billingBucketUrl = tempFolder.getRoot().getAbsolutePath();
+    String beamTempFolderPath = beamTempFolder.getAbsolutePath();
+    invoicingPipeline = new InvoicingPipeline(
+        "test-project",
+        beamTempFolderPath,
+        beamTempFolderPath + "/templates/invoicing",
+        beamTempFolderPath + "/staging",
+        tempFolder.getRoot().getAbsolutePath(),
+        "REG-INV",
+        GoogleCredentialsBundle.create(GoogleCredentials.create(null))
+    );
  }

  private ImmutableList<BillingEvent> getInputEvents() {
@@ -21,11 +21,13 @@ import static org.mockito.Mockito.mock;
 import static org.mockito.Mockito.when;
 import static org.mockito.Mockito.withSettings;

+import com.google.auth.oauth2.GoogleCredentials;
 import com.google.common.collect.ImmutableList;
 import com.google.common.io.CharStreams;
 import google.registry.beam.spec11.SafeBrowsingTransforms.EvaluateSafeBrowsingFn;
 import google.registry.testing.FakeClock;
 import google.registry.testing.FakeSleeper;
+import google.registry.util.GoogleCredentialsBundle;
 import google.registry.util.ResourceUtils;
 import google.registry.util.Retrier;
 import java.io.ByteArrayInputStream;
@@ -50,6 +52,7 @@ import org.apache.http.client.methods.HttpPost;
 import org.apache.http.entity.BasicHttpEntity;
 import org.apache.http.impl.client.CloseableHttpClient;
 import org.apache.http.message.BasicStatusLine;
+import org.joda.time.DateTime;
 import org.json.JSONArray;
 import org.json.JSONException;
 import org.json.JSONObject;
@@ -78,16 +81,21 @@ public class Spec11PipelineTest {
  @Rule public final transient TestPipeline p = TestPipeline.fromOptions(pipelineOptions);
  @Rule public final TemporaryFolder tempFolder = new TemporaryFolder();

+  private final Retrier retrier = new Retrier(
+      new FakeSleeper(new FakeClock(DateTime.parse("2019-07-15TZ"))), 1);
  private Spec11Pipeline spec11Pipeline;

  @Before
  public void initializePipeline() throws IOException {
-    spec11Pipeline = new Spec11Pipeline();
-    spec11Pipeline.projectId = "test-project";
-    spec11Pipeline.reportingBucketUrl = tempFolder.getRoot().getAbsolutePath();
    File beamTempFolder = tempFolder.newFolder();
-    spec11Pipeline.beamStagingUrl = beamTempFolder.getAbsolutePath() + "/staging";
-    spec11Pipeline.spec11TemplateUrl = beamTempFolder.getAbsolutePath() + "/templates/invoicing";
+    spec11Pipeline = new Spec11Pipeline(
+        "test-project",
+        beamTempFolder.getAbsolutePath() + "/staging",
+        beamTempFolder.getAbsolutePath() + "/templates/invoicing",
+        tempFolder.getRoot().getAbsolutePath(),
+        GoogleCredentialsBundle.create(GoogleCredentials.create(null)),
+        retrier
+    );
  }

  private static final ImmutableList<String> BAD_DOMAINS =
@@ -1 +0,0 @@
-hello
@@ -1 +0,0 @@
-OMG IM AN OVERRIDE
@@ -1 +0,0 @@
-world
@@ -33,6 +33,7 @@ import static org.joda.money.CurrencyUnit.USD;
 import com.google.common.collect.ImmutableMap;
 import com.google.common.collect.ImmutableSortedMap;
 import com.google.common.collect.Ordering;
+import com.google.common.truth.Truth;
 import com.google.re2j.Matcher;
 import com.google.re2j.Pattern;
 import google.registry.model.billing.BillingEvent;
@@ -197,6 +198,11 @@ public class EppLifecycleDomainTest extends EppTestCase {
        makeRecurringCreateBillingEvent(domain, createTime.plusYears(2), deleteTime));

    assertThatLogoutSucceeds();
+
+    // Make sure that in the future, the domain expiration is unchanged after deletion
+    DomainBase clonedDomain = domain.cloneProjectedAtTime(deleteTime.plusYears(5));
+    Truth.assertThat(clonedDomain.getRegistrationExpirationTime())
+        .isEqualTo(createTime.plusYears(2));
  }

  @Test
@@ -28,6 +28,7 @@ import static google.registry.testing.DomainBaseSubject.assertAboutDomains;
 import static google.registry.testing.JUnitBackports.assertThrows;
 import static google.registry.util.DateTimeUtils.START_OF_TIME;
 import static org.joda.money.CurrencyUnit.USD;
+import static org.joda.time.DateTimeZone.UTC;

 import com.google.common.collect.ImmutableList;
 import com.google.common.collect.ImmutableSet;
@@ -61,34 +62,40 @@ import org.junit.Test;
 /** Unit tests for {@link DomainBase}. */
 public class DomainBaseTest extends EntityTestCase {

-  DomainBase domain;
+  private DomainBase domain;
+  private Key<BillingEvent.OneTime> oneTimeBillKey;
+  private Key<BillingEvent.Recurring> recurringBillKey;

  @Before
  public void setUp() {
    createTld("com");
    Key<DomainBase> domainKey = Key.create(null, DomainBase.class, "4-COM");
-    Key<HostResource> hostKey = Key.create(persistResource(
-        new HostResource.Builder()
-            .setFullyQualifiedHostName("ns1.example.com")
-            .setSuperordinateDomain(domainKey)
-            .setRepoId("1-COM")
-            .build()));
-    Key<ContactResource> contact1Key = Key.create(persistResource(
-        new ContactResource.Builder()
-            .setContactId("contact_id1")
-            .setRepoId("2-COM")
-            .build()));
-    Key<ContactResource> contact2Key = Key.create(persistResource(
-        new ContactResource.Builder()
-            .setContactId("contact_id2")
-            .setRepoId("3-COM")
-            .build()));
+    Key<HostResource> hostKey =
+        Key.create(
+            persistResource(
+                new HostResource.Builder()
+                    .setFullyQualifiedHostName("ns1.example.com")
+                    .setSuperordinateDomain(domainKey)
+                    .setRepoId("1-COM")
+                    .build()));
+    Key<ContactResource> contact1Key =
+        Key.create(
+            persistResource(
+                new ContactResource.Builder()
+                    .setContactId("contact_id1")
+                    .setRepoId("2-COM")
+                    .build()));
+    Key<ContactResource> contact2Key =
+        Key.create(
+            persistResource(
+                new ContactResource.Builder()
+                    .setContactId("contact_id2")
+                    .setRepoId("3-COM")
+                    .build()));
    Key<HistoryEntry> historyEntryKey =
        Key.create(persistResource(new HistoryEntry.Builder().setParent(domainKey).build()));
-    Key<BillingEvent.OneTime> oneTimeBillKey =
-        Key.create(historyEntryKey, BillingEvent.OneTime.class, 1);
-    Key<BillingEvent.Recurring> recurringBillKey =
-        Key.create(historyEntryKey, BillingEvent.Recurring.class, 2);
+    oneTimeBillKey = Key.create(historyEntryKey, BillingEvent.OneTime.class, 1);
+    recurringBillKey = Key.create(historyEntryKey, BillingEvent.Recurring.class, 2);
    Key<PollMessage.Autorenew> autorenewPollKey =
        Key.create(historyEntryKey, PollMessage.Autorenew.class, 3);
    Key<PollMessage.OneTime> onetimePollKey =
@@ -168,21 +175,27 @@ public class DomainBaseTest extends EntityTestCase {

  @Test
  public void testEmptyStringsBecomeNull() {
-    assertThat(newDomainBase("example.com").asBuilder()
-        .setPersistedCurrentSponsorClientId(null)
-        .build()
-            .getCurrentSponsorClientId())
-                .isNull();
-    assertThat(newDomainBase("example.com").asBuilder()
-            .setPersistedCurrentSponsorClientId("")
-            .build()
+    assertThat(
+            newDomainBase("example.com")
+                .asBuilder()
+                .setPersistedCurrentSponsorClientId(null)
+                .build()
                .getCurrentSponsorClientId())
-                    .isNull();
-    assertThat(newDomainBase("example.com").asBuilder()
-        .setPersistedCurrentSponsorClientId(" ")
-        .build()
-            .getCurrentSponsorClientId())
-                .isNotNull();
+        .isNull();
+    assertThat(
+            newDomainBase("example.com")
+                .asBuilder()
+                .setPersistedCurrentSponsorClientId("")
+                .build()
+                .getCurrentSponsorClientId())
+        .isNull();
+    assertThat(
+            newDomainBase("example.com")
+                .asBuilder()
+                .setPersistedCurrentSponsorClientId(" ")
+                .build()
+                .getCurrentSponsorClientId())
+        .isNotNull();
  }

  @Test
@@ -201,29 +214,49 @@ public class DomainBaseTest extends EntityTestCase {
                .build()
                .nsHosts)
        .isNull();
-    assertThat(newDomainBase("example.com").asBuilder()
-        .setNameservers(ImmutableSet.of(Key.create(newHostResource("foo.example.tld"))))
-            .build().nsHosts)
-                .isNotNull();
+    assertThat(
+            newDomainBase("example.com")
+                .asBuilder()
+                .setNameservers(ImmutableSet.of(Key.create(newHostResource("foo.example.tld"))))
+                .build()
+                .nsHosts)
+        .isNotNull();
    // This behavior should also hold true for ImmutableObjects nested in collections.
-    assertThat(newDomainBase("example.com").asBuilder()
-        .setDsData(ImmutableSet.of(DelegationSignerData.create(1, 1, 1, (byte[]) null)))
-        .build().getDsData().asList().get(0).getDigest())
-            .isNull();
-    assertThat(newDomainBase("example.com").asBuilder()
-        .setDsData(ImmutableSet.of(DelegationSignerData.create(1, 1, 1, new byte[]{})))
-        .build().getDsData().asList().get(0).getDigest())
-            .isNull();
-    assertThat(newDomainBase("example.com").asBuilder()
-        .setDsData(ImmutableSet.of(DelegationSignerData.create(1, 1, 1, new byte[]{1})))
-        .build().getDsData().asList().get(0).getDigest())
-            .isNotNull();
+    assertThat(
+            newDomainBase("example.com")
+                .asBuilder()
+                .setDsData(ImmutableSet.of(DelegationSignerData.create(1, 1, 1, (byte[]) null)))
+                .build()
+                .getDsData()
+                .asList()
+                .get(0)
+                .getDigest())
+        .isNull();
+    assertThat(
+            newDomainBase("example.com")
+                .asBuilder()
+                .setDsData(ImmutableSet.of(DelegationSignerData.create(1, 1, 1, new byte[] {})))
+                .build()
+                .getDsData()
+                .asList()
+                .get(0)
+                .getDigest())
+        .isNull();
+    assertThat(
+            newDomainBase("example.com")
+                .asBuilder()
+                .setDsData(ImmutableSet.of(DelegationSignerData.create(1, 1, 1, new byte[] {1})))
+                .build()
+                .getDsData()
+                .asList()
+                .get(0)
+                .getDigest())
+        .isNotNull();
  }

  @Test
  public void testEmptyTransferDataBecomesNull() {
-    DomainBase withNull =
-        newDomainBase("example.com").asBuilder().setTransferData(null).build();
+    DomainBase withNull = newDomainBase("example.com").asBuilder().setTransferData(null).build();
    DomainBase withEmpty = withNull.asBuilder().setTransferData(TransferData.EMPTY).build();
    assertThat(withNull).isEqualTo(withEmpty);
    assertThat(withEmpty.transferData).isNull();
@@ -241,18 +274,22 @@ public class DomainBaseTest extends EntityTestCase {
    StatusValue[] statuses1 = {StatusValue.CLIENT_HOLD};
    // If there are other status values, OK should be suppressed. (Domains can't be LINKED.)
    assertAboutDomains()
-        .that(newDomainBase("example.com").asBuilder()
-            .setNameservers(nameservers)
-            .setStatusValues(ImmutableSet.of(StatusValue.CLIENT_HOLD))
-            .build())
+        .that(
+            newDomainBase("example.com")
+                .asBuilder()
+                .setNameservers(nameservers)
+                .setStatusValues(ImmutableSet.of(StatusValue.CLIENT_HOLD))
+                .build())
        .hasExactlyStatusValues(statuses1);
    StatusValue[] statuses2 = {StatusValue.CLIENT_HOLD};
    // When OK is suppressed, it should be removed even if it was originally there.
    assertAboutDomains()
-        .that(newDomainBase("example.com").asBuilder()
-            .setNameservers(nameservers)
-            .setStatusValues(ImmutableSet.of(StatusValue.OK, StatusValue.CLIENT_HOLD))
-            .build())
+        .that(
+            newDomainBase("example.com")
+                .asBuilder()
+                .setNameservers(nameservers)
+                .setStatusValues(ImmutableSet.of(StatusValue.OK, StatusValue.CLIENT_HOLD))
+                .build())
        .hasExactlyStatusValues(statuses2);
    StatusValue[] statuses3 = {StatusValue.INACTIVE};
    // If there are no nameservers, INACTIVE should be added, which suppresses OK.
@@ -262,17 +299,21 @@ public class DomainBaseTest extends EntityTestCase {
    StatusValue[] statuses4 = {StatusValue.CLIENT_HOLD, StatusValue.INACTIVE};
    // If there are no nameservers but there are status values, INACTIVE should still be added.
    assertAboutDomains()
-        .that(newDomainBase("example.com").asBuilder()
-            .setStatusValues(ImmutableSet.of(StatusValue.CLIENT_HOLD))
-            .build())
+        .that(
+            newDomainBase("example.com")
+                .asBuilder()
+                .setStatusValues(ImmutableSet.of(StatusValue.CLIENT_HOLD))
+                .build())
        .hasExactlyStatusValues(statuses4);
    StatusValue[] statuses5 = {StatusValue.CLIENT_HOLD};
    // If there are nameservers, INACTIVE should be removed even if it was originally there.
    assertAboutDomains()
-        .that(newDomainBase("example.com").asBuilder()
-            .setNameservers(nameservers)
-            .setStatusValues(ImmutableSet.of(StatusValue.INACTIVE, StatusValue.CLIENT_HOLD))
-            .build())
+        .that(
+            newDomainBase("example.com")
+                .asBuilder()
+                .setNameservers(nameservers)
+                .setStatusValues(ImmutableSet.of(StatusValue.INACTIVE, StatusValue.CLIENT_HOLD))
+                .build())
        .hasExactlyStatusValues(statuses5);
  }

@@ -290,18 +331,22 @@ public class DomainBaseTest extends EntityTestCase {

  private void doExpiredTransferTest(DateTime oldExpirationTime) {
    HistoryEntry historyEntry = new HistoryEntry.Builder().setParent(domain).build();
-    BillingEvent.OneTime transferBillingEvent = persistResource(
-        new BillingEvent.OneTime.Builder()
-            .setReason(Reason.TRANSFER)
-            .setClientId("winner")
-            .setTargetId("example.com")
-            .setEventTime(clock.nowUtc())
-            .setBillingTime(
-                clock.nowUtc().plusDays(1).plus(Registry.get("com").getTransferGracePeriodLength()))
-            .setCost(Money.of(USD, 11))
-            .setPeriodYears(1)
-            .setParent(historyEntry)
-            .build());
+    BillingEvent.OneTime transferBillingEvent =
+        persistResource(
+            new BillingEvent.OneTime.Builder()
+                .setReason(Reason.TRANSFER)
+                .setClientId("winner")
+                .setTargetId("example.com")
+                .setEventTime(clock.nowUtc())
+                .setBillingTime(
+                    clock
+                        .nowUtc()
+                        .plusDays(1)
+                        .plus(Registry.get("com").getTransferGracePeriodLength()))
+                .setCost(Money.of(USD, 11))
+                .setPeriodYears(1)
+                .setParent(historyEntry)
+                .build());
    domain =
        domain
            .asBuilder()
@@ -329,14 +374,16 @@ public class DomainBaseTest extends EntityTestCase {
        domain.getTransferData().getServerApproveAutorenewEvent();
    assertTransferred(afterTransfer, newExpirationTime, serverApproveAutorenewEvent);
    assertThat(afterTransfer.getGracePeriods())
-        .containsExactly(GracePeriod.create(
-            GracePeriodStatus.TRANSFER,
-            clock.nowUtc().plusDays(1).plus(Registry.get("com").getTransferGracePeriodLength()),
-            "winner",
-            Key.create(transferBillingEvent)));
+        .containsExactly(
+            GracePeriod.create(
+                GracePeriodStatus.TRANSFER,
+                clock.nowUtc().plusDays(1).plus(Registry.get("com").getTransferGracePeriodLength()),
+                "winner",
+                Key.create(transferBillingEvent)));
    // If we project after the grace period expires all should be the same except the grace period.
-    DomainBase afterGracePeriod = domain.cloneProjectedAtTime(
-        clock.nowUtc().plusDays(2).plus(Registry.get("com").getTransferGracePeriodLength()));
+    DomainBase afterGracePeriod =
+        domain.cloneProjectedAtTime(
+            clock.nowUtc().plusDays(2).plus(Registry.get("com").getTransferGracePeriodLength()));
    assertTransferred(afterGracePeriod, newExpirationTime, serverApproveAutorenewEvent);
    assertThat(afterGracePeriod.getGracePeriods()).isEmpty();
  }
@@ -439,10 +486,11 @@ public class DomainBaseTest extends EntityTestCase {

  @Test
  public void testStackedGracePeriods() {
-    ImmutableList<GracePeriod> gracePeriods = ImmutableList.of(
-        GracePeriod.create(GracePeriodStatus.ADD, clock.nowUtc().plusDays(3), "foo", null),
-        GracePeriod.create(GracePeriodStatus.ADD, clock.nowUtc().plusDays(2), "bar", null),
-        GracePeriod.create(GracePeriodStatus.ADD, clock.nowUtc().plusDays(1), "baz", null));
+    ImmutableList<GracePeriod> gracePeriods =
+        ImmutableList.of(
+            GracePeriod.create(GracePeriodStatus.ADD, clock.nowUtc().plusDays(3), "foo", null),
+            GracePeriod.create(GracePeriodStatus.ADD, clock.nowUtc().plusDays(2), "bar", null),
+            GracePeriod.create(GracePeriodStatus.ADD, clock.nowUtc().plusDays(1), "baz", null));
    domain = domain.asBuilder().setGracePeriods(ImmutableSet.copyOf(gracePeriods)).build();
    for (int i = 1; i < 3; ++i) {
      assertThat(domain.cloneProjectedAtTime(clock.nowUtc().plusDays(i)).getGracePeriods())
@@ -452,12 +500,14 @@ public class DomainBaseTest extends EntityTestCase {

  @Test
  public void testGracePeriodsByType() {
-    ImmutableSet<GracePeriod> addGracePeriods = ImmutableSet.of(
-        GracePeriod.create(GracePeriodStatus.ADD, clock.nowUtc().plusDays(3), "foo", null),
-        GracePeriod.create(GracePeriodStatus.ADD, clock.nowUtc().plusDays(1), "baz", null));
-    ImmutableSet<GracePeriod> renewGracePeriods = ImmutableSet.of(
-        GracePeriod.create(GracePeriodStatus.RENEW, clock.nowUtc().plusDays(3), "foo", null),
-        GracePeriod.create(GracePeriodStatus.RENEW, clock.nowUtc().plusDays(1), "baz", null));
+    ImmutableSet<GracePeriod> addGracePeriods =
+        ImmutableSet.of(
+            GracePeriod.create(GracePeriodStatus.ADD, clock.nowUtc().plusDays(3), "foo", null),
+            GracePeriod.create(GracePeriodStatus.ADD, clock.nowUtc().plusDays(1), "baz", null));
+    ImmutableSet<GracePeriod> renewGracePeriods =
+        ImmutableSet.of(
+            GracePeriod.create(GracePeriodStatus.RENEW, clock.nowUtc().plusDays(3), "foo", null),
+            GracePeriod.create(GracePeriodStatus.RENEW, clock.nowUtc().plusDays(1), "baz", null));
    domain =
        domain
            .asBuilder()
@@ -472,8 +522,7 @@ public class DomainBaseTest extends EntityTestCase {

  @Test
  public void testRenewalsHappenAtExpiration() {
-    DomainBase renewed =
-        domain.cloneProjectedAtTime(domain.getRegistrationExpirationTime());
+    DomainBase renewed = domain.cloneProjectedAtTime(domain.getRegistrationExpirationTime());
    assertThat(renewed.getRegistrationExpirationTime())
        .isEqualTo(domain.getRegistrationExpirationTime().plusYears(1));
    assertThat(renewed.getLastEppUpdateTime()).isEqualTo(domain.getRegistrationExpirationTime());
@@ -490,9 +539,11 @@ public class DomainBaseTest extends EntityTestCase {

  @Test
  public void testRenewalsDontHappenOnFebruary29() {
-    domain = domain.asBuilder()
-        .setRegistrationExpirationTime(DateTime.parse("2004-02-29T22:00:00.0Z"))
-        .build();
+    domain =
+        domain
+            .asBuilder()
+            .setRegistrationExpirationTime(DateTime.parse("2004-02-29T22:00:00.0Z"))
+            .build();
    DomainBase renewed =
        domain.cloneProjectedAtTime(domain.getRegistrationExpirationTime().plusYears(4));
    assertThat(renewed.getRegistrationExpirationTime().getDayOfMonth()).isEqualTo(28);
@@ -517,23 +568,24 @@ public class DomainBaseTest extends EntityTestCase {
                    .put(oldExpirationTime.plusYears(2).plusMillis(1), Money.of(USD, 5))
                    .build())
            .build());
-    DomainBase renewedThreeTimes =
-        domain.cloneProjectedAtTime(oldExpirationTime.plusYears(2));
+    DomainBase renewedThreeTimes = domain.cloneProjectedAtTime(oldExpirationTime.plusYears(2));
    assertThat(renewedThreeTimes.getRegistrationExpirationTime())
        .isEqualTo(oldExpirationTime.plusYears(3));
    assertThat(renewedThreeTimes.getLastEppUpdateTime()).isEqualTo(oldExpirationTime.plusYears(2));
    assertThat(renewedThreeTimes.getGracePeriods())
-        .containsExactly(GracePeriod.createForRecurring(
-            GracePeriodStatus.AUTO_RENEW,
-            oldExpirationTime.plusYears(2).plus(
-                Registry.get("com").getAutoRenewGracePeriodLength()),
-            renewedThreeTimes.getCurrentSponsorClientId(),
-            renewedThreeTimes.autorenewBillingEvent));
+        .containsExactly(
+            GracePeriod.createForRecurring(
+                GracePeriodStatus.AUTO_RENEW,
+                oldExpirationTime
+                    .plusYears(2)
+                    .plus(Registry.get("com").getAutoRenewGracePeriodLength()),
+                renewedThreeTimes.getCurrentSponsorClientId(),
+                renewedThreeTimes.autorenewBillingEvent));
  }

  @Test
  public void testToHydratedString_notCircular() {
-    domain.toHydratedString();  // If there are circular references, this will overflow the stack.
+    domain.toHydratedString(); // If there are circular references, this will overflow the stack.
  }

  @Test
@@ -557,4 +609,154 @@ public class DomainBaseTest extends EntityTestCase {
        .hasMessageThat()
        .contains("Domain name must be in puny-coded, lower-case form");
  }
+
+  @Test
+  public void testClone_doNotExtendExpirationOnDeletedDomain() {
+    DateTime now = DateTime.now(UTC);
+    domain =
+        persistResource(
+            domain
+                .asBuilder()
+                .setRegistrationExpirationTime(now.minusDays(1))
+                .setDeletionTime(now.minusDays(10))
+                .setStatusValues(ImmutableSet.of(StatusValue.PENDING_DELETE, StatusValue.INACTIVE))
+                .build());
+    assertThat(domain.cloneProjectedAtTime(now).getRegistrationExpirationTime())
+        .isEqualTo(now.minusDays(1));
+  }
+
+  @Test
+  public void testClone_doNotExtendExpirationOnFutureDeletedDomain() {
+    // if a domain is in pending deletion (StatusValue.PENDING_DELETE), don't extend expiration
+    DateTime now = DateTime.now(UTC);
+    domain =
+        persistResource(
+            domain
+                .asBuilder()
+                .setRegistrationExpirationTime(now.plusDays(1))
+                .setDeletionTime(now.plusDays(20))
+                .setStatusValues(ImmutableSet.of(StatusValue.PENDING_DELETE, StatusValue.INACTIVE))
+                .build());
+    assertThat(domain.cloneProjectedAtTime(now).getRegistrationExpirationTime())
+        .isEqualTo(now.plusDays(1));
+  }
+
+  @Test
+  public void testClone_extendsExpirationForExpiredTransferredDomain() {
+    // If the transfer implicitly succeeded, the expiration time should be extended
+    DateTime now = DateTime.now(UTC);
+    DateTime transferExpirationTime = now.minusDays(1);
+    DateTime previousExpiration = now.minusDays(2);
+
+    TransferData transferData =
+        new TransferData.Builder()
+            .setPendingTransferExpirationTime(transferExpirationTime)
+            .setTransferStatus(TransferStatus.PENDING)
+            .setGainingClientId("TheRegistrar")
+            .build();
+    Period extensionPeriod = transferData.getTransferPeriod();
+    DateTime newExpiration = previousExpiration.plusYears(extensionPeriod.getValue());
+    domain =
+        persistResource(
+            domain
+                .asBuilder()
+                .setRegistrationExpirationTime(previousExpiration)
+                .setTransferData(transferData)
+                .build());
+
+    assertThat(domain.cloneProjectedAtTime(now).getRegistrationExpirationTime())
+        .isEqualTo(newExpiration);
+  }
+
+  @Test
+  public void testClone_extendsExpirationForNonExpiredTransferredDomain() {
+    // If the transfer implicitly succeeded, the expiration time should be extended even if it
+    // hadn't already expired
+    DateTime now = DateTime.now(UTC);
+    DateTime transferExpirationTime = now.minusDays(1);
+    DateTime previousExpiration = now.plusWeeks(2);
+
+    TransferData transferData =
+        new TransferData.Builder()
+            .setPendingTransferExpirationTime(transferExpirationTime)
+            .setTransferStatus(TransferStatus.PENDING)
+            .setGainingClientId("TheRegistrar")
+            .build();
+    Period extensionPeriod = transferData.getTransferPeriod();
+    DateTime newExpiration = previousExpiration.plusYears(extensionPeriod.getValue());
+    domain =
+        persistResource(
+            domain
+                .asBuilder()
+                .setRegistrationExpirationTime(previousExpiration)
+                .setTransferData(transferData)
+                .build());
+
+    assertThat(domain.cloneProjectedAtTime(now).getRegistrationExpirationTime())
+        .isEqualTo(newExpiration);
+  }
+
+  @Test
+  public void testClone_doesNotExtendExpirationForPendingTransfer() {
+    // Pending transfers shouldn't affect the expiration time
+    DateTime now = DateTime.now(UTC);
+    DateTime transferExpirationTime = now.plusDays(1);
+    DateTime previousExpiration = now.plusWeeks(2);
+
+    TransferData transferData =
+        new TransferData.Builder()
+            .setPendingTransferExpirationTime(transferExpirationTime)
+            .setTransferStatus(TransferStatus.PENDING)
+            .setGainingClientId("TheRegistrar")
+            .build();
+    domain =
+        persistResource(
+            domain
+                .asBuilder()
+                .setRegistrationExpirationTime(previousExpiration)
+                .setTransferData(transferData)
+                .build());
+
+    assertThat(domain.cloneProjectedAtTime(now).getRegistrationExpirationTime())
+        .isEqualTo(previousExpiration);
+  }
+
+  @Test
+  public void testClone_transferDuringAutorenew() {
+    // When the domain is an an autorenew grace period, we should not extend the registration
+    // expiration by a further year--it should just be whatever the autorenew was
+    DateTime now = DateTime.now(UTC);
+    DateTime transferExpirationTime = now.minusDays(1);
+    DateTime previousExpiration = now.minusDays(2);
+
+    TransferData transferData =
+        new TransferData.Builder()
+            .setPendingTransferExpirationTime(transferExpirationTime)
+            .setTransferStatus(TransferStatus.PENDING)
+            .setGainingClientId("TheRegistrar")
+            .setServerApproveAutorenewEvent(recurringBillKey)
+            .setServerApproveBillingEvent(oneTimeBillKey)
+            .build();
+    domain =
+        persistResource(
+            domain
+                .asBuilder()
+                .setRegistrationExpirationTime(previousExpiration)
+                .setGracePeriods(
+                    ImmutableSet.of(
+                        GracePeriod.createForRecurring(
+                            GracePeriodStatus.AUTO_RENEW,
+                            now.plusDays(1),
+                            "NewRegistrar",
+                            recurringBillKey)))
+                .setTransferData(transferData)
+                .setAutorenewBillingEvent(recurringBillKey)
+                .build());
+    DomainBase clone = domain.cloneProjectedAtTime(now);
+    assertThat(clone.getRegistrationExpirationTime())
+        .isEqualTo(domain.getRegistrationExpirationTime().plusYears(1));
+    // Transferring removes the AUTORENEW grace period and adds a TRANSFER grace period
+    assertThat(getOnlyElement(clone.getGracePeriods()).getType())
+        .isEqualTo(GracePeriodStatus.TRANSFER);
+  }
 }
@@ -452,6 +452,14 @@ public class RegistrarTest extends EntityTestCase {
        () -> registrar.asBuilder().setAllowedTldsUncached(ImmutableSet.of("bad")));
  }

+  @Test
+  public void testFailure_driveFolderId_asFullUrl() {
+    IllegalArgumentException thrown =
+        assertThrows(IllegalArgumentException.class, () -> registrar.asBuilder().setDriveFolderId(
+            "https://drive.google.com/drive/folders/1j3v7RZkU25DjbTx2-Q93H04zKOBau89M"));
+    assertThat(thrown).hasMessageThat().isEqualTo("Drive folder ID must not be a full URL");
+  }
+
  @Test
  public void testFailure_nullEmail() {
    NullPointerException thrown =
@@ -16,8 +16,12 @@ package google.registry.rdap;

 import static com.google.common.truth.Truth.assertThat;
 import static com.google.common.truth.Truth8.assertThat;
+import static google.registry.testing.DatastoreHelper.createTld;
+import static google.registry.testing.DatastoreHelper.deleteTld;
 import static google.registry.testing.DatastoreHelper.loadRegistrar;
+import static google.registry.testing.DatastoreHelper.persistResource;
 import static google.registry.testing.DatastoreHelper.persistSimpleResource;
+import static google.registry.testing.JUnitBackports.assertThrows;

 import com.google.api.client.http.LowLevelHttpRequest;
 import com.google.api.client.testing.http.MockHttpTransport;
@@ -27,6 +31,8 @@ import com.google.common.collect.ImmutableList;
 import com.google.common.collect.ImmutableSet;
 import google.registry.model.registrar.Registrar;
 import google.registry.model.registrar.RegistrarAddress;
+import google.registry.model.registry.Registry;
+import google.registry.model.registry.Registry.TldType;
 import google.registry.testing.AppEngineRule;
 import google.registry.testing.ShardableTestCase;
 import java.util.ArrayList;
@@ -95,8 +101,8 @@ public final class UpdateRegistrarRdapBaseUrlsActionTest extends ShardableTestCa
    }
  }

-  TestHttpTransport httpTransport;
-  UpdateRegistrarRdapBaseUrlsAction action;
+  private TestHttpTransport httpTransport;
+  private UpdateRegistrarRdapBaseUrlsAction action;

  @Before
  public void setUp() {
@@ -104,27 +110,10 @@ public final class UpdateRegistrarRdapBaseUrlsActionTest extends ShardableTestCa
    action = new UpdateRegistrarRdapBaseUrlsAction();

    action.password = "myPassword";
-    action.tld = "tld";
    action.httpTransport = httpTransport;
+    addValidResponses(httpTransport);

-    MockLowLevelHttpResponse loginResponse = new MockLowLevelHttpResponse();
-    loginResponse.addHeader(
-        "Set-Cookie",
-        "id=myAuthenticationId; "
-            + "Expires=Tue, 11-Jun-2019 16:34:21 GMT; Path=/mosapi/v1/app; Secure; HttpOnly");
-
-    MockLowLevelHttpResponse listResponse = new MockLowLevelHttpResponse();
-    listResponse.setContent(JSON_LIST_REPLY);
-
-    MockLowLevelHttpResponse logoutResponse = new MockLowLevelHttpResponse();
-    loginResponse.addHeader(
-        "Set-Cookie",
-        "id=id; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/mosapi/v1/app; Secure; HttpOnly");
-
-    httpTransport.addNextResponse(loginResponse);
-    httpTransport.addNextResponse(listResponse);
-    httpTransport.addNextResponse(logoutResponse);
-
+    createTld("tld");
  }

  private void assertCorrectRequestsSent() {
@@ -237,4 +226,72 @@ public final class UpdateRegistrarRdapBaseUrlsActionTest extends ShardableTestCa
    assertThat(loadRegistrar("registrar4001").getRdapBaseUrls())
        .containsExactly("https://rdap.example.com");
  }
+
+  @Test
+  public void testNoTlds() {
+    deleteTld("tld");
+    assertThat(assertThrows(IllegalArgumentException.class, action::run)).hasMessageThat()
+        .isEqualTo("There must exist at least one REAL TLD.");
+  }
+
+  @Test
+  public void testOnlyTestTlds() {
+    persistResource(Registry.get("tld").asBuilder().setTldType(TldType.TEST).build());
+    assertThat(assertThrows(IllegalArgumentException.class, action::run)).hasMessageThat()
+        .isEqualTo("There must exist at least one REAL TLD.");
+  }
+
+  @Test
+  public void testSecondTldSucceeds() {
+    createTld("secondtld");
+    httpTransport = new TestHttpTransport();
+    action.httpTransport = httpTransport;
+
+    // the first TLD request will return a bad cookie but the second will succeed
+    MockLowLevelHttpResponse badLoginResponse = new MockLowLevelHttpResponse();
+    badLoginResponse.addHeader("Set-Cookie",
+        "Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/mosapi/v1/app; Secure; HttpOnly");
+
+    httpTransport.addNextResponse(badLoginResponse);
+    addValidResponses(httpTransport);
+
+    action.run();
+  }
+
+  @Test
+  public void testBothFail() {
+    createTld("secondtld");
+    httpTransport = new TestHttpTransport();
+    action.httpTransport = httpTransport;
+
+    MockLowLevelHttpResponse badLoginResponse = new MockLowLevelHttpResponse();
+    badLoginResponse.addHeader("Set-Cookie",
+        "Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/mosapi/v1/app; Secure; HttpOnly");
+
+    // it should fail for both TLDs
+    httpTransport.addNextResponse(badLoginResponse);
+    httpTransport.addNextResponse(badLoginResponse);
+
+    assertThat(assertThrows(RuntimeException.class, action::run)).hasMessageThat()
+        .isEqualTo("Error contacting MosAPI server. Tried TLDs [secondtld, tld]");
+  }
+
+  private static void addValidResponses(TestHttpTransport httpTransport) {
+    MockLowLevelHttpResponse loginResponse = new MockLowLevelHttpResponse();
+    loginResponse.addHeader(
+        "Set-Cookie",
+        "id=myAuthenticationId; "
+            + "Expires=Tue, 11-Jun-2019 16:34:21 GMT; Path=/mosapi/v1/app; Secure; HttpOnly");
+
+    MockLowLevelHttpResponse listResponse = new MockLowLevelHttpResponse();
+    listResponse.setContent(JSON_LIST_REPLY);
+
+    MockLowLevelHttpResponse logoutResponse = new MockLowLevelHttpResponse();
+    logoutResponse.addHeader(
+        "Set-Cookie",
+        "id=id; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/mosapi/v1/app; Secure; HttpOnly");
+    httpTransport.addNextResponse(loginResponse);
+    httpTransport.addNextResponse(listResponse);
+    httpTransport.addNextResponse(logoutResponse);
+  }
 }
@@ -18,10 +18,10 @@ import static com.google.common.truth.Truth.assertThat;
 import static google.registry.testing.DatastoreHelper.loadRegistrar;
 import static google.registry.testing.DatastoreHelper.persistResource;
 import static google.registry.testing.GcsTestingUtils.writeGcsFile;
-import static google.registry.testing.JUnitBackports.assertThrows;
 import static java.nio.charset.StandardCharsets.UTF_8;
 import static javax.servlet.http.HttpServletResponse.SC_OK;
 import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.eq;
 import static org.mockito.Mockito.mock;
 import static org.mockito.Mockito.times;
 import static org.mockito.Mockito.verify;
@@ -68,6 +68,7 @@ public class CopyDetailReportsActionTest {
  @Before
  public void setUp() {
    persistResource(loadRegistrar("TheRegistrar").asBuilder().setDriveFolderId("0B-12345").build());
+    persistResource(loadRegistrar("NewRegistrar").asBuilder().setDriveFolderId("0B-54321").build());
    response = new FakeResponse();
    driveConnection = mock(DriveConnection.class);
    emailUtils = mock(BillingEmailUtils.class);
@@ -96,21 +97,21 @@ public class CopyDetailReportsActionTest {

    action.run();
    verify(driveConnection)
-        .createFile(
+        .createOrUpdateFile(
            "invoice_details_2017-10_TheRegistrar_test.csv",
            MediaType.CSV_UTF_8,
            "0B-12345",
            "hello,world\n1,2".getBytes(UTF_8));

    verify(driveConnection)
-        .createFile(
+        .createOrUpdateFile(
            "invoice_details_2017-10_TheRegistrar_hello.csv",
            MediaType.CSV_UTF_8,
            "0B-12345",
            "hola,mundo\n3,4".getBytes(UTF_8));
    assertThat(response.getStatus()).isEqualTo(SC_OK);
    assertThat(response.getContentType()).isEqualTo(MediaType.PLAIN_TEXT_UTF_8);
-    assertThat(response.getPayload()).isEqualTo("Copied detail reports.");
+    assertThat(response.getPayload()).isEqualTo("Copied detail reports.\n");
  }

  @Test
@@ -126,7 +127,7 @@ public class CopyDetailReportsActionTest {
        "hello,world\n1,2".getBytes(UTF_8));
    action.run();
    verify(driveConnection)
-        .createFile(
+        .createOrUpdateFile(
            "invoice_details_2017-10_TheRegistrar_hello.csv",
            MediaType.CSV_UTF_8,
            "0B-12345",
@@ -135,7 +136,7 @@ public class CopyDetailReportsActionTest {
    verifyNoMoreInteractions(driveConnection);
    assertThat(response.getStatus()).isEqualTo(SC_OK);
    assertThat(response.getContentType()).isEqualTo(MediaType.PLAIN_TEXT_UTF_8);
-    assertThat(response.getPayload()).isEqualTo("Copied detail reports.");
+    assertThat(response.getPayload()).isEqualTo("Copied detail reports.\n");
  }

  @Test
@@ -144,41 +145,63 @@ public class CopyDetailReportsActionTest {
        gcsService,
        new GcsFilename("test-bucket", "results/invoice_details_2017-10_TheRegistrar_hello.csv"),
        "hola,mundo\n3,4".getBytes(UTF_8));
-    when(driveConnection.createFile(any(), any(), any(), any()))
+    when(driveConnection.createOrUpdateFile(any(), any(), any(), any()))
        .thenThrow(new IOException("expected"))
        .thenReturn("success");

    action.run();
    verify(driveConnection, times(2))
-        .createFile(
+        .createOrUpdateFile(
            "invoice_details_2017-10_TheRegistrar_hello.csv",
            MediaType.CSV_UTF_8,
            "0B-12345",
            "hola,mundo\n3,4".getBytes(UTF_8));
    assertThat(response.getStatus()).isEqualTo(SC_OK);
    assertThat(response.getContentType()).isEqualTo(MediaType.PLAIN_TEXT_UTF_8);
-    assertThat(response.getPayload()).isEqualTo("Copied detail reports.");
+    assertThat(response.getPayload()).isEqualTo("Copied detail reports.\n");
  }

  @Test
-  public void testFail_tooManyFailures_sendsAlertEmail() throws IOException {
+  public void testFail_tooManyFailures_sendsAlertEmail_continues() throws IOException {
    writeGcsFile(
        gcsService,
        new GcsFilename("test-bucket", "results/invoice_details_2017-10_TheRegistrar_hello.csv"),
        "hola,mundo\n3,4".getBytes(UTF_8));
-    when(driveConnection.createFile(any(), any(), any(), any()))
+    writeGcsFile(
+        gcsService,
+        new GcsFilename("test-bucket", "results/invoice_details_2017-10_NewRegistrar_test.csv"),
+        "hello,world\n1,2".getBytes(UTF_8));
+    when(driveConnection.createOrUpdateFile(
+            eq("invoice_details_2017-10_TheRegistrar_hello.csv"), any(), any(), any()))
        .thenThrow(new IOException("expected"));

-    RuntimeException thrown = assertThrows(RuntimeException.class, action::run);
-    assertThat(thrown).hasMessageThat().isEqualTo("java.io.IOException: expected");
+    action.run();
    verify(driveConnection, times(3))
-        .createFile(
+        .createOrUpdateFile(
            "invoice_details_2017-10_TheRegistrar_hello.csv",
            MediaType.CSV_UTF_8,
            "0B-12345",
            "hola,mundo\n3,4".getBytes(UTF_8));
-    verify(emailUtils).sendAlertEmail("Warning: CopyDetailReportsAction failed.\nEncountered: "
-        + "expected on file: invoice_details_2017-10_TheRegistrar_hello.csv");
+    verify(driveConnection)
+        .createOrUpdateFile(
+            "invoice_details_2017-10_NewRegistrar_test.csv",
+            MediaType.CSV_UTF_8,
+            "0B-54321",
+            "hello,world\n1,2".getBytes(UTF_8));
+    verify(emailUtils)
+        .sendAlertEmail(
+            "Copied detail reports.\n"
+                + "The following errors were encountered:\n"
+                + "Registrar: TheRegistrar\n"
+                + "Error: java.io.IOException: expected\n");
+    assertThat(response.getStatus()).isEqualTo(SC_OK);
+    assertThat(response.getContentType()).isEqualTo(MediaType.PLAIN_TEXT_UTF_8);
+    assertThat(response.getPayload())
+        .isEqualTo(
+            "Copied detail reports.\n"
+                + "The following errors were encountered:\n"
+                + "Registrar: TheRegistrar\n"
+                + "Error: java.io.IOException: expected\n");
  }

  @Test
@@ -141,7 +141,15 @@ public class RepeatableRunner extends BlockJUnit4ClassRunner {
      int numSuccess = 0, numFailure = 0;
      Throwable lastException = null;
      for (int attempt = 1; attempt <= MAX_ATTEMPTS; attempt++) {
-        attemptNumber.set(attempt);
+        // attemptNumber would be null if any exception happens during the
+        // instantiation of test class object(including test rules). However,
+        // those exceptions would not be actually thrown until statement.evaluate()
+        // is invoked. So, we should just skip the setter and let the statement
+        // be evaluated. Then we should be able to find the stack trace of
+        // root cause in the test reports.
+        if (attemptNumber != null) {
+          attemptNumber.set(attempt);
+        }
        try {
          statement.evaluate();
          numSuccess++;
@@ -17,6 +17,7 @@ package google.registry.webdriver;
 import google.registry.webdriver.RepeatableRunner.AttemptNumber;
 import org.junit.ClassRule;
 import org.junit.Rule;
+import org.junit.rules.Timeout;

 /** Base class for tests that needs a {@link WebDriverPlusScreenDifferRule}. */
 public class WebDriverTestCase {
@@ -27,4 +28,7 @@ public class WebDriverTestCase {
  @Rule
  public final WebDriverPlusScreenDifferRule driver =
      new WebDriverPlusScreenDifferRule(webDriverProvider::getWebDriver, attemptNumber);
+
+  @Rule
+  public final Timeout timeout = Timeout.seconds(30);
 }
@@ -13,20 +13,17 @@
 // limitations under the License.

 ext {
+    // Direct dependencies and compile-time transitive dependencies (i.e.,
+    // those flagged as undeclared-dependency by Nebula-lint).
    dependencyList = [
      'args4j:args4j:2.0.26',
      'com.beust:jcommander:1.60',
      'com.google.api-client:google-api-client:1.29.2',
-      'com.google.api-client:google-api-client-appengine:1.29.2',
-      'com.google.api-client:google-api-client-jackson2:1.20.0',
      'com.google.api-client:google-api-client-java6:1.27.0',
-      'com.google.api-client:google-api-client-servlet:1.29.2',
      'com.google.apis:google-api-services-admin-directory:directory_v1-rev72-1.22.0',
      'com.google.apis:google-api-services-appengine:v1-rev101-1.25.0',
      'com.google.apis:google-api-services-bigquery:v2-rev325-1.22.0',
-      'com.google.apis:google-api-services-clouddebugger:v2-rev8-1.22.0',
      'com.google.apis:google-api-services-cloudkms:v1-rev12-1.22.0',
-      'com.google.apis:google-api-services-cloudresourcemanager:v1-rev6-1.22.0',
      'com.google.apis:google-api-services-dataflow:v1b3-rev196-1.22.0',
      'com.google.apis:google-api-services-dns:v2beta1-rev6-1.22.0',
      'com.google.apis:google-api-services-drive:v2-rev160-1.19.1',
@@ -35,38 +32,27 @@ ext {
      'com.google.apis:google-api-services-sheets:v4-rev483-1.22.0',
      'com.google.apis:google-api-services-storage:v1-rev150-1.22.0',
      'com.google.appengine:appengine-api-1.0-sdk:1.9.48',
-      'com.google.appengine:appengine-api-labs:1.9.48',
      'com.google.appengine:appengine-api-stubs:1.9.48',
      'com.google.appengine:appengine-remote-api:1.9.48',
      'com.google.appengine:appengine-testing:1.9.58',
-      'com.google.appengine:appengine-tools-sdk:1.9.48',
      'com.google.appengine.tools:appengine-gcs-client:0.6',
      'com.google.appengine.tools:appengine-mapreduce:0.9',
      'com.google.appengine.tools:appengine-pipeline:0.2.13',
      'com.google.auth:google-auth-library-credentials:0.16.1',
      'com.google.auth:google-auth-library-oauth2-http:0.16.1',
-      'com.google.auto:auto-common:0.8',
-      'com.google.auto.factory:auto-factory:1.0-beta3',
      'com.google.auto.value:auto-value:1.6.3',
      'com.google.auto.value:auto-value-annotations:1.6.3',
      'com.google.closure-stylesheets:closure-stylesheets:1.5.0',
      'com.google.cloud:google-cloud-core:1.59.0',
      'com.google.cloud:google-cloud-storage:1.59.0',
-      'com.google.cloud.bigdataoss:gcsio:1.4.5',
-      'com.google.cloud.bigdataoss:util:1.4.5',
-      'com.googlecode.charts4j:charts4j:1.3',
      'com.google.code.findbugs:jsr305:3.0.2',
      'com.google.code.gson:gson:2.8.5',
      'com.googlecode.json-simple:json-simple:1.1.1',
      'com.google.dagger:dagger:2.21',
      'com.google.dagger:dagger-compiler:2.21',
-      'com.google.dagger:dagger-producers:2.21',
      'com.google.errorprone:error_prone_annotations:2.3.3',
-      'com.google.errorprone:javac-shaded:9-dev-r4023-3',
      'com.google.flogger:flogger:0.1',
      'com.google.flogger:flogger-system-backend:0.1',
-      'com.google.gdata:core:1.47.1',
-      'com.google.googlejavaformat:google-java-format:1.4',
      'com.google.guava:guava:27.1-jre',
      'com.google.guava:guava-testlib:25.0-jre',
      'com.google.gwt:gwt-user:2.8.2',
@@ -78,27 +64,18 @@ ext {
      'com.google.monitoring-client:metrics:1.0.4',
      'com.google.monitoring-client:stackdriver:1.0.4',
      'com.google.oauth-client:google-oauth-client:1.29.2',
-      'com.google.oauth-client:google-oauth-client-appengine:1.29.2',
      'com.google.oauth-client:google-oauth-client-java6:1.27.0',
      'com.google.oauth-client:google-oauth-client-jetty:1.28.0',
-      'com.google.oauth-client:google-oauth-client-servlet:1.29.2',
-      'com.google.protobuf:protobuf-java:2.6.0',
      'com.google.re2j:re2j:1.1',
      'com.google.template:soy:2018-03-14',
      'com.google.truth.extensions:truth-java8-extension:0.45',
      'com.google.truth:truth:0.45',
      'com.ibm.icu:icu4j:57.1',
      'com.jcraft:jsch:0.1.55',
-      'com.jcraft:jzlib:1.1.3',
-      'commons-codec:commons-codec:1.10',
-      'commons-logging:commons-logging:1.2',
-      'com.squareup:javapoet:1.8.0',
-      'com.squareup:javawriter:2.5.1',
      'com.sun.activation:javax.activation:1.2.0',
      'com.sun.xml.bind:jaxb-impl:2.2.11',
      'com.sun.xml.bind:jaxb-osgi:2.2.11',
      'com.sun.xml.bind:jaxb-xjc:2.2.11',
-      'com.thoughtworks.paranamer:paranamer:2.7',
      'com.thoughtworks.qdox:qdox:1.12.1',
      'dnsjava:dnsjava:2.1.7',
      'io.netty:netty-buffer:4.1.31.Final',
@@ -106,34 +83,27 @@ ext {
      'io.netty:netty-codec-http:4.1.31.Final',
      'io.netty:netty-common:4.1.31.Final',
      'io.netty:netty-handler:4.1.31.Final',
-      'io.netty:netty-resolver:4.1.28.Final',
-      'io.netty:netty-tcnative:2.0.22.Final',
      'io.netty:netty-tcnative-boringssl-static:2.0.22.Final',
      'io.netty:netty-transport:4.1.31.Final',
-      'it.unimi.dsi:fastutil:6.5.16',
      'javax.annotation:jsr250-api:1.0',
      'javax.inject:javax.inject:1',
      'javax.mail:mail:1.4',
      'javax.servlet:servlet-api:2.5',
      'javax.xml.bind:jaxb-api:2.3.0',
-      'javax.xml.soap:javax.xml.soap-api:1.4.0',
      'jline:jline:1.0',
      'joda-time:joda-time:2.9.2',
      'junit:junit:4.12',
      'org.apache.avro:avro:1.8.2',
      'org.apache.beam:beam-runners-direct-java:2.11.0',
      'org.apache.beam:beam-runners-google-cloud-dataflow-java:2.11.0',
-      'org.apache.beam:beam-sdks-common-runner-api:2.2.0',
      'org.apache.beam:beam-sdks-java-core:2.11.0',
      'org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.11.0',
      'org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.11.0',
-      'org.apache.commons:commons-compress:1.8.1',
      'org.apache.commons:commons-text:1.6',
      'org.apache.ftpserver:ftplet-api:1.0.6',
      'org.apache.ftpserver:ftpserver-core:1.0.6',
      'org.apache.httpcomponents:httpclient:4.5.2',
      'org.apache.httpcomponents:httpcore:4.4.4',
-      'org.apache.mina:mina-core:2.0.4',
      'org.apache.sshd:sshd-core:2.0.0',
      'org.apache.sshd:sshd-scp:2.0.0',
      'org.apache.sshd:sshd-sftp:2.0.0',
@@ -141,26 +111,19 @@ ext {
      'org.bouncycastle:bcpg-jdk15on:1.61',
      'org.bouncycastle:bcpkix-jdk15on:1.61',
      'org.bouncycastle:bcprov-jdk15on:1.61',
-      'org.codehaus.jackson:jackson-core-asl:1.9.13',
-      'org.codehaus.jackson:jackson-mapper-asl:1.9.13',
      'org.glassfish.jaxb:jaxb-runtime:2.3.0',
      'org.hamcrest:hamcrest-all:1.3',
      'org.hamcrest:hamcrest-core:1.3',
      'org.hamcrest:hamcrest-library:1.3',
      'org.joda:joda-money:0.10.0',
      'org.json:json:20160810',
-      'org.khronos:opengl-api:gl1.1-android-2.1_r1',
      'org.mockito:mockito-core:2.25.0',
      'org.mortbay.jetty:jetty:6.1.26',
-      'org.mortbay.jetty:jetty-util:6.1.26',
      'org.seleniumhq.selenium:selenium-api:3.141.59',
      'org.seleniumhq.selenium:selenium-chrome-driver:3.141.59',
      'org.seleniumhq.selenium:selenium-java:3.141.59',
      'org.seleniumhq.selenium:selenium-remote-driver:3.141.59',
-      'org.slf4j:slf4j-api:1.7.16',
      'org.testcontainers:selenium:1.10.7',
-      'org.tukaani:xz:1.8',
-      'org.xerial.snappy:snappy-java:1.1.4-M3',
      'org.yaml:snakeyaml:1.17',
      'xerces:xmlParserAPIs:2.6.2',
      'xpp3:xpp3:1.1.4c'
@@ -0,0 +1,87 @@
+// Copyright 2019 The Nomulus Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// Legacy-style declaration of the gradle-license-report plugin, which is necessary
+// in shared scripts.
+// Note that the 'apply plugin' line does not use string literal. It references directly the
+// main class of the plugin.
+buildscript {
+  repositories {
+    maven {
+      url 'https://plugins.gradle.org/m2/'
+    }
+  }
+  dependencies {
+    classpath 'com.github.jk1:gradle-license-report:1.8'
+  }
+}
+apply plugin: com.github.jk1.license.LicenseReportPlugin
+
+// Configures the checkLicense task.
+//
+// This task discovers license information for each dependency including a
+// name phrase and possibly additional details, such as an http link or relative
+// path of a file in the jar. This information is saved in json format under
+// build/reports/dependency-license. It is not used right now, but may be useful
+// if we want to check the content of the licenses.
+//
+// The required 'allowedLicensesFile' contains the acceptable license name phrases.
+// If a phrase not in this file is found, the checkLicense task will fail. Exact
+// string match is used, so they are case-sensitive and preceding/trailing whitespaces are important.
+//
+// This plugin also generates an html report under build/reports/dependency-license.
+// The optional 'filters' parameter accepts custom name-normalization rules. This
+// report is not currently in use.
+//
+// Note that this task needs to be explicitly invoked on buildSrc.
+//
+// Also note that dependencies of buildscript classpath and 'official' Gradle plugins are
+// not checked.
+//
+// Due to a bug/feature of this plugin, the Nomulus project can not lock all dependencies.
+// The 'dependencyLicenseReport' configuration must be excluded from locking.
+// TODO(weiminyu): add github issue number after reporting to author.
+licenseReport {
+  ext.toFullFilePath = { relativePath ->
+    if (rootDir.path.endsWith('buildSrc')) {
+      return "${rootDir.path}/../${relativePath}"
+    }
+    return "${rootDir.path}/${relativePath}"
+  }
+
+  // Check licenses dependencies in all configurations. This does not include
+  // buildscript-classpath.
+  configurations = ALL
+
+  def configFolder = 'config/dependency-license'
+
+  allowedLicensesFile = file(toFullFilePath("${configFolder}/allowed_licenses.json"))
+
+  filters = new com.github.jk1.license.filter.LicenseBundleNormalizer(
+      bundlePath: toFullFilePath("${configFolder}/license_normalizer_bundle.json"),
+      createDefaultTransformationRules: false)
+
+  // The following dependency jars do not contain license info.
+  // Investigation results are documented inline.
+  excludes = [
+      // BSD: https://github.com/javacc/javacc/blob/master/LICENSE
+      'javacc:javacc',
+      // CDDL+GPL2.0 according to Maven Central:
+      // https://mvnrepository.com/artifact/javax.servlet/servlet-api/2.5
+      'javax.servlet:servlet-api',
+      // Presumed free since this is from Apache:
+      // http://www.apache.org/licenses/
+      'xerces:xmlParserAPIs'
+  ]
+}
@@ -11,10 +11,15 @@ com.diffplug.spotless:spotless-lib:1.18.0
 com.diffplug.spotless:spotless-plugin-gradle:3.18.0
 com.github.jengelman.gradle.plugins:shadow:5.0.0
 com.github.johnrengelman.shadow:com.github.johnrengelman.shadow.gradle.plugin:5.0.0
-com.google.cloud.tools:appengine-gradle-plugin:1.3.3
-com.google.cloud.tools:appengine-plugins-core:0.3.2
-com.google.code.gson:gson:2.7
-com.google.guava:guava:20.0
+com.google.cloud.tools:appengine-gradle-plugin:2.0.1
+com.google.cloud.tools:appengine-plugins-core:0.7.5
+com.google.code.findbugs:jsr305:3.0.2
+com.google.code.gson:gson:2.8.5
+com.google.errorprone:error_prone_annotations:2.2.0
+com.google.guava:failureaccess:1.0
+com.google.guava:guava:27.0-jre
+com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
+com.google.j2objc:j2objc-annotations:1.1
 com.googlecode.concurrent-trees:concurrent-trees:2.6.1
 com.googlecode.javaewah:JavaEWAH:1.1.6
 com.jcraft:jsch:0.1.54
@@ -39,6 +44,7 @@ org.apache.ant:ant-antlr:1.8.4
 org.apache.ant:ant-junit:1.8.4
 org.apache.ant:ant-launcher:1.9.7
 org.apache.ant:ant:1.9.7
+org.apache.commons:commons-compress:1.18
 org.apache.commons:commons-lang3:3.8.1
 org.apache.httpcomponents:httpclient:4.5.2
 org.apache.httpcomponents:httpcore:4.4.4
@@ -48,6 +54,7 @@ org.apache.maven:maven-artifact:3.6.1
 org.apache.maven:maven-builder-support:3.6.1
 org.apache.maven:maven-model-builder:3.6.1
 org.apache.maven:maven-model:3.6.1
+org.checkerframework:checker-qual:2.5.2
 org.codehaus.gpars:gpars:1.2.1
 org.codehaus.groovy:groovy-all:2.4.9
 org.codehaus.groovy:groovy-ant:2.1.8
@@ -56,12 +63,14 @@ org.codehaus.groovy:groovy-templates:2.1.8
 org.codehaus.groovy:groovy-xml:2.4.7
 org.codehaus.groovy:groovy:2.4.7
 org.codehaus.jsr166-mirror:jsr166y:1.7.0
+org.codehaus.mojo:animal-sniffer-annotations:1.17
 org.codehaus.plexus:plexus-component-annotations:1.7.1
 org.codehaus.plexus:plexus-interpolation:1.25
 org.codehaus.plexus:plexus-utils:3.2.0
 org.codenarc:CodeNarc:0.25.2
 org.eclipse.jdt:core:3.1.1
 org.eclipse.jgit:org.eclipse.jgit:5.0.1.201806211838-r
+org.glassfish:javax.json:1.0.4
 org.gmetrics:GMetrics:0.7
 org.hamcrest:hamcrest-core:1.3
 org.jdom:jdom2:2.0.6
@@ -82,4 +91,4 @@ org.sonatype.aether:aether-spi:1.13.1
 org.sonatype.aether:aether-util:1.13.1
 org.spockframework:spock-core:1.1-groovy-2.4-rc-4
 org.vafer:jdependency:2.1.1
-org.yaml:snakeyaml:1.17
+org.yaml:snakeyaml:1.21
@@ -1,6 +1,5 @@
-#Tue Jun 18 15:57:55 EDT 2019
 distributionBase=GRADLE_USER_HOME
 distributionPath=wrapper/dists
+distributionUrl=https\://services.gradle.org/distributions/gradle-5.5.1-all.zip
 zipStoreBase=GRADLE_USER_HOME
 zipStorePath=wrapper/dists
-distributionUrl=https\://services.gradle.org/distributions/gradle-5.4.1-all.zip
@@ -7,7 +7,7 @@
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
 #
-#      http://www.apache.org/licenses/LICENSE-2.0
+#      https://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
@@ -5,7 +5,7 @@
@rem you may not use this file except in compliance with the License.
@rem You may obtain a copy of the License at
@rem
-@rem      http://www.apache.org/licenses/LICENSE-2.0
+@rem      https://www.apache.org/licenses/LICENSE-2.0
@rem
@rem Unless required by applicable law or agreed to in writing, software
@rem distributed under the License is distributed on an "AS IS" BASIS,
@@ -28,13 +28,13 @@ module.exports = function(config) {
    autoWatch: false,
    files: [
      'node_modules/google-closure-library/closure/goog/base.js',
-      'core/build/resources/test/**/*_test.js',
+      'core/src/test/javascript/**/*_test.js',
      {
-        pattern: 'core/build/resources/test/**/!(*_test).js',
+        pattern: 'core/src/test/javascript/**/!(*_test).js',
        included: false
      },
      {
-        pattern: 'core/build/resources/main/**/*.js',
+        pattern: 'core/src/main/javascript/**/*.js',
        included: false
      },
      {
@@ -63,9 +63,7 @@ module.exports = function(config) {
      'node_modules/google-closure-library/closure/goog/deps.js': ['closure', 'closure-deps'],
      'node_modules/google-closure-library/closure/goog/base.js': ['closure'],
      'node_modules/google-closure-library/closure/**/*.js': ['closure'],
-      'core/build/resources/test/**/*_test.js': ['closure'],
-      'core/build/resources/test/**/!(*_test).js': ['closure'],
-      'core/build/resources/main/**/*.js': ['closure'],
+      'core/src/*/javascript/**/*.js': ['closure'],
      'core/build/generated/source/custom/main/**/*.soy.js': ['closure'],
      'node_modules/soyutils_usegoog.js': ['closure']
    },
@@ -1007,7 +1007,8 @@
        "ansi-regex": {
          "version": "2.1.1",
          "bundled": true,
-          "dev": true
+          "dev": true,
+          "optional": true
        },
        "aproba": {
          "version": "1.2.0",
@@ -1028,12 +1029,14 @@
        "balanced-match": {
          "version": "1.0.0",
          "bundled": true,
-          "dev": true
+          "dev": true,
+          "optional": true
        },
        "brace-expansion": {
          "version": "1.1.11",
          "bundled": true,
          "dev": true,
+          "optional": true,
          "requires": {
            "balanced-match": "^1.0.0",
            "concat-map": "0.0.1"
@@ -1048,17 +1051,20 @@
        "code-point-at": {
          "version": "1.1.0",
          "bundled": true,
-          "dev": true
+          "dev": true,
+          "optional": true
        },
        "concat-map": {
          "version": "0.0.1",
          "bundled": true,
-          "dev": true
+          "dev": true,
+          "optional": true
        },
        "console-control-strings": {
          "version": "1.1.0",
          "bundled": true,
-          "dev": true
+          "dev": true,
+          "optional": true
        },
        "core-util-is": {
          "version": "1.0.2",
@@ -1175,7 +1181,8 @@
        "inherits": {
          "version": "2.0.3",
          "bundled": true,
-          "dev": true
+          "dev": true,
+          "optional": true
        },
        "ini": {
          "version": "1.3.5",
@@ -1187,6 +1194,7 @@
          "version": "1.0.0",
          "bundled": true,
          "dev": true,
+          "optional": true,
          "requires": {
            "number-is-nan": "^1.0.0"
          }
@@ -1201,6 +1209,7 @@
          "version": "3.0.4",
          "bundled": true,
          "dev": true,
+          "optional": true,
          "requires": {
            "brace-expansion": "^1.1.7"
          }
@@ -1208,12 +1217,14 @@
        "minimist": {
          "version": "0.0.8",
          "bundled": true,
-          "dev": true
+          "dev": true,
+          "optional": true
        },
        "minipass": {
          "version": "2.3.5",
          "bundled": true,
          "dev": true,
+          "optional": true,
          "requires": {
            "safe-buffer": "^5.1.2",
            "yallist": "^3.0.0"
@@ -1232,6 +1243,7 @@
          "version": "0.5.1",
          "bundled": true,
          "dev": true,
+          "optional": true,
          "requires": {
            "minimist": "0.0.8"
          }
@@ -1312,7 +1324,8 @@
        "number-is-nan": {
          "version": "1.0.1",
          "bundled": true,
-          "dev": true
+          "dev": true,
+          "optional": true
        },
        "object-assign": {
          "version": "4.1.1",
@@ -1324,6 +1337,7 @@
          "version": "1.4.0",
          "bundled": true,
          "dev": true,
+          "optional": true,
          "requires": {
            "wrappy": "1"
          }
@@ -1409,7 +1423,8 @@
        "safe-buffer": {
          "version": "5.1.2",
          "bundled": true,
-          "dev": true
+          "dev": true,
+          "optional": true
        },
        "safer-buffer": {
          "version": "2.1.2",
@@ -1445,6 +1460,7 @@
          "version": "1.0.2",
          "bundled": true,
          "dev": true,
+          "optional": true,
          "requires": {
            "code-point-at": "^1.0.0",
            "is-fullwidth-code-point": "^1.0.0",
@@ -1464,6 +1480,7 @@
          "version": "3.0.1",
          "bundled": true,
          "dev": true,
+          "optional": true,
          "requires": {
            "ansi-regex": "^2.0.0"
          }
@@ -1507,12 +1524,14 @@
        "wrappy": {
          "version": "1.0.2",
          "bundled": true,
-          "dev": true
+          "dev": true,
+          "optional": true
        },
        "yallist": {
          "version": "3.0.3",
          "bundled": true,
-          "dev": true
+          "dev": true,
+          "optional": true
        }
      }
    },
@@ -1935,9 +1954,9 @@
      "dev": true
    },
    "lodash": {
-      "version": "4.17.11",
-      "resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.11.tgz",
-      "integrity": "sha512-cQKh8igo5QUhZ7lg38DYWAxMvjSAKG0A8wGSVimP07SIUEK2UO+arSRKbRZWtelMtN5V0Hkwh5ryOto/SshYIg==",
+      "version": "4.17.14",
+      "resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.14.tgz",
+      "integrity": "sha512-mmKYbW3GLuJeX+iGP+Y7Gp1AiGHGbXHCOh/jZmrawMmsE7MS4znI3RL2FsjbqOyMayHInjOeykW7PEajUk1/xw==",
      "dev": true
    },
    "log4js": {
@@ -0,0 +1 @@
+out/
@@ -0,0 +1,49 @@
+// Copyright 2019 The Nomulus Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+apply plugin: 'java'
+
+createUberJar('deployJar', 'prober', 'google.registry.monitoring.blackbox.Prober')
+
+dependencies {
+    def deps = rootProject.dependencyMap
+
+    compile deps['com.google.auto.value:auto-value-annotations']
+    compile deps['com.google.dagger:dagger']
+    compile deps['com.google.flogger:flogger']
+    compile deps['com.google.guava:guava']
+    compile deps['io.netty:netty-buffer']
+    compile deps['io.netty:netty-codec-http']
+    compile deps['io.netty:netty-codec']
+    compile deps['io.netty:netty-common']
+    compile deps['io.netty:netty-handler']
+    compile deps['io.netty:netty-transport']
+    compile deps['javax.inject:javax.inject']
+
+    runtime deps['com.google.flogger:flogger-system-backend']
+    runtime deps['com.google.auto.value:auto-value']
+    runtime deps['io.netty:netty-tcnative-boringssl-static']
+
+    testCompile deps['com.google.truth:truth']
+    testCompile deps['junit:junit']
+    testCompile deps['org.mockito:mockito-core']
+    testCompile project(':third_party')
+
+    // Include auto-value in compile until nebula-lint understands
+    // annotationProcessor
+    annotationProcessor deps['com.google.auto.value:auto-value']
+    testAnnotationProcessor deps['com.google.auto.value:auto-value']
+    annotationProcessor deps['com.google.dagger:dagger-compiler']
+    testAnnotationProcessor deps['com.google.dagger:dagger-compiler']
+}
@@ -0,0 +1,58 @@
+// Copyright 2019 The Nomulus Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package google.registry.monitoring.blackbox;
+
+import com.google.auto.value.AutoValue;
+import com.google.common.collect.ImmutableList;
+import io.netty.channel.ChannelHandler;
+import javax.inject.Provider;
+
+/**
+ * {@link AutoValue} class that stores all unchanged variables necessary for type of connection
+ */
+@AutoValue
+public abstract class Protocol {
+
+  abstract String name();
+
+  public abstract int port();
+
+  /** The {@link ChannelHandler} providers to use for the protocol, in order. */
+  abstract ImmutableList<Provider<? extends ChannelHandler>> handlerProviders();
+
+  /** Boolean that notes if connection associated with Protocol is persistent.*/
+  abstract boolean persistentConnection();
+
+  public abstract Builder toBuilder();
+
+  public static Builder builder() {
+    return new AutoValue_Protocol.Builder();
+  }
+
+  @AutoValue.Builder
+  public static abstract class Builder {
+
+    public abstract Builder name(String value);
+
+    public abstract Builder port(int num);
+
+    public abstract Builder handlerProviders(ImmutableList<Provider<? extends ChannelHandler>> providers);
+
+    public abstract Builder persistentConnection(boolean value);
+
+    public abstract Protocol build();
+  }
+}
+
@@ -0,0 +1,74 @@
+// Copyright 2019 The Nomulus Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package google.registry.monitoring.blackbox.handlers;
+
+import com.google.common.flogger.FluentLogger;
+import google.registry.monitoring.blackbox.messages.InboundMessageType;
+import google.registry.monitoring.blackbox.messages.OutboundMessageType;
+import io.netty.channel.SimpleChannelInboundHandler;
+import io.netty.channel.ChannelFuture;
+import io.netty.channel.ChannelHandlerContext;
+import io.netty.channel.ChannelPromise;
+
+/**
+ *Superclass of all {@link ChannelHandler}s placed at end of channel pipeline
+ *
+ * <p> {@code ActionHandler} inherits from {@link SimpleChannelInboundHandler< InboundMessageType >}, as it should only be passed in
+ * messages that implement the {@link InboundMessageType} interface. </p>
+ *
+ * <p> The {@code ActionHandler} skeleton exists for a few main purposes. First, it returns a {@link ChannelPromise},
+ * which informs the {@link ProbingAction} in charge that a response has been read. Second, it stores the {@link OutboundMessageType}
+ * passed down the pipeline, so that subclasses can use that information for their own processes. Lastly, with any exception
+ * thrown, the connection is closed, and the ProbingAction governing this channel is informed of the error. Subclasses
+ * specify further work to be done for specific kinds of channel pipelines. </p>
+ */
+public abstract class ActionHandler extends SimpleChannelInboundHandler<InboundMessageType> {
+
+  private static final FluentLogger logger = FluentLogger.forEnclosingClass();
+
+  protected ChannelPromise finished;
+
+  /** Takes in {@link OutboundMessageType} type and saves for subclasses. Then returns initialized {@link ChannelPromise}*/
+  public ChannelFuture getFuture() {
+    return finished;
+  }
+
+  /** Initializes new {@link ChannelPromise} */
+  @Override
+  public void handlerAdded(ChannelHandlerContext ctx) {
+    //Once handler is added to channel pipeline, initialize channel and future for this handler
+    finished = ctx.newPromise();
+  }
+
+  @Override
+  public void channelRead0(ChannelHandlerContext ctx, InboundMessageType inboundMessage) throws Exception {
+    //simply marks finished as success
+    finished.setSuccess();
+  }
+
+  /** Logs the channel and pipeline that caused error, closes channel, then informs {@link ProbingAction} listeners of error */
+  @Override
+  public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
+    logger.atSevere().withCause(cause).log(String.format(
+        "Attempted Action was unsuccessful with channel: %s, having pipeline: %s",
+        ctx.channel().toString(),
+        ctx.channel().pipeline().toString()));
+
+    finished.setFailure(cause);
+    ChannelFuture closedFuture = ctx.channel().close();
+    closedFuture.addListener(f -> logger.atInfo().log("Unsuccessful channel connection closed"));
+  }
+}
+
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
gbrodman	bf29d159f9	Fix a few deprecations (#186 )	2019-07-22 14:12:55 -04:00
Lai Jiang	e17cb52bf7	Fail gracefully when copying detailed reports (#181 ) * Fail gracefully when copying detailed reports When the detailed reports are copied from GCS to registrars' Drive folders, do not fail the entire copy operation when a single registrar fails. Instead, send an alert email about the failure, and continue to copy the rest of the reports. Also, instead of creating duplicates, overwrite the existing files on Drive. BUG=127690361	2019-07-22 14:09:49 -04:00
Lai Jiang	7352f9b4a6	Remove unused local variable (#185 )	2019-07-22 10:04:16 -04:00
Lai Jiang	5da48184f9	Merge beam and GAE configs deployment to one GCB job (#182 ) * Merge beam and GAE configs deployment to one GCB job Deployment of GAE configs requires that the credential used by gcloud to have GAE admin role of the project to be managed. We do not want to grant the GCB service account that role, because it would all any GCB job to deploy anything to GAE. Instead we use a dedicated credential originally created to deploy beam pipelines. This credential is encrypted by KMS and stored on GCS. Since the beam pipeline deployment GCB job already does the decryption, it make sense to add the config deployment step there as well. The beam deployment steps are tweaked to use the nomulus tool docker image instead of the jar file. Also moved the content of deploy_configs_to_env.sh to the GCB yaml file itself because the shell script is not uploaded to GC Bat the same time as the yaml file when the job is triggered by Spinnaker. Lastly, due to b/137891685, using GCB to deploy cron jobs does not work as we cannot use service account credential to deploy to projects under google.com.	2019-07-19 16:54:56 -04:00
gbrodman	5bd2ccd210	Add a Cloud Build task to update YAML configs (#177 ) * Add a Cloud Build task to update YAML configs * CR responses * Move config deployment to a script * Pin builder version * Create different beam and deploy-config files per environment * Update comments and make a for loop	2019-07-18 12:15:15 -04:00
Lai Jiang	8fd5ab2bec	Build proxy image in Gradle (#179 )	2019-07-17 20:38:03 -04:00
Lai Jiang	30f6113b05	Upgrade to Gradle 5.5.1 (#178 ) Also make the default wrapper type "all" instead of "binary". This is helpful for IDEs to understand the gradle script.	2019-07-17 17:37:44 -04:00
Michael Muller	bd48041961	Build docker image of nomulus tool (#142 ) * Build docker image of nomulus tool In the course of "gradle build", build a docker image of nomulus tool so that users can run this to allow us to bundle the java version with the image.	2019-07-16 20:18:44 -04:00
gbrodman	39ceda628c	Don't extend expiration times for deleted domains (#160 ) * Don't extend expiration times for deleted domains * Flip order and add a comment * oops forgot a period * Use END_OF_TIME * Add tests for expiration times of domains with pending transfers * Add test for transfer during autorenew and clean up other tests * Clarify tests * Add domain expiration check in EppLifecycleDomainTest * Add a comment and format test files	2019-07-16 18:34:21 -04:00
Aman Sanger	0e9b75e5e9	Created Prober subproject and setup basic ActionHandler and its unit tests (#133 ) * Initial Commit. * Deleted unfinished features. Added ActionHandler and its Unit Tests. * Included prober subproject in settings.gradle * Added Protocol Class and its Basic Unit Tests * Added Changes Suggested by jianglai * Fixed Gitignore to take out AutoValue generated code * Removed AutoValue java files * Added gitignore within prober * Removed all generated java * Final Changes in .gitignore * Added Ssl and WebWhois Action Handlers and their unit tests in addition to the ProbingAction class * Fixed build.gradle changes requested * Removed Files irrelevant to current pull request * Minor fixes to ActionHandler, as responded in comments, removed package-info, and updated settings.gradle * Fully Updated ActionHandler (missing updated JavaDoc) * Added changed Protocol and both Inbound and Outbound Markers * Removed AutoVaue ignore clause from .gitignore * removed unneccessary dependencies in build.gradle * Fixed Javadoc and comments for ActionHandler * Fixed comments and JavaDoc on other files * EOL added * Removed Unnecessary Files * fixed .gradle files styles * Merge remote-tracking branch 'upstream/master' * Removed outbound message from ActionHandler's fields and renamed Marker Interfaces * Fixed javadoc for Marker Interfaced * Modified Comments on ActionHandler * Removed LocalAddress from Protocol * Fixed Travis Build Issues	2019-07-16 10:35:14 -04:00
Lai Jiang	c1207464d8	Fix a typo (#174 )	2019-07-15 17:49:22 -04:00
Weimin Yu	62eab98921	Unused deps check (#171 ) * Save for later * Check for unused dependencies Add a task to check for unused dependencies in dependencies.gradle. This file should only contain dependencies explicitly added to a configuration in root project or some subprojects. * Check for unused dependencies Add a task to check for unused dependencies in dependencies.gradle. This file should only contain dependencies explicitly added to a configuration in root project or some subprojects. * Check for unused dependencies Add a task to check for unused dependencies in dependencies.gradle. This file should only contain dependencies explicitly added to a configuration in root project or some subprojects. * Check for unused dependencies Add a task to check for unused dependencies in dependencies.gradle. This file should only contain dependencies explicitly added to a configuration in root project or some subprojects. * Check for unused dependencies Add a task to check for unused dependencies in dependencies.gradle. This file should only contain dependencies explicitly added to a configuration in root project or some subprojects. * Check for unused dependencies Add a task to check for unused dependencies in dependencies.gradle. This file should only contain dependencies explicitly added to a configuration in root project or some subprojects. * Check for unused dependencies Add a task to check for unused dependencies in dependencies.gradle. This file should only contain dependencies explicitly added to a configuration in root project or some subprojects. * Check for unused dependencies Add a task to check for unused dependencies in dependencies.gradle. This file should only contain dependencies explicitly added to a configuration in root project or some subprojects. * Check for unused dependencies Add a task to check for unused dependencies in dependencies.gradle. This file should only contain dependencies explicitly added to a configuration in root project or some subprojects.	2019-07-15 16:19:34 -04:00
Shicong Huang	633dd887f4	Fix permission issue in Beam pipeline deployment (#170 )	2019-07-15 16:13:42 -04:00
Lai Jiang	650f1fdd52	Bump lodash from 4.17.11 to 4.17.14 (#173 )	2019-07-15 11:26:12 -04:00
gbrodman	b0f6a498fd	Bump the version of the AppEngine Gradle plugin (#169 ) * Bump the version of the AppEngine Gradle plugin * deploy should depend on deployAll	2019-07-15 11:25:23 -04:00
gbrodman	77590dcd8e	Add a metric for EPP processing time regardless of ID/TLD (#163 ) * Add a metric for EPP processing time regardless of ID/TLD * Change name to request_time * Record EPP processing time by traffic type * grammar * request type * semicolon	2019-07-11 14:28:37 -04:00
Weimin Yu	29e4d8de05	Check dependencies' open-source licenses (#165 ) * Check dependencies' open-source licenses Use jk1/Gradle-License-Report to verify that all dependencies have open-source licenses. Note that the following dependencies are not checked: Dependencies of official Gradle plugins Dependencies declared in buildscript block Dependencies of jk1/Gradle-License-Report itself	2019-07-11 12:08:08 -04:00
Weimin Yu	242a560f20	Remove the maybeRuntime configuration (#164 ) * Remove the maybeRuntime configuration It contains dependencies present in the bazel build but not needed for compile. We now know they are not needed in runtime either.	2019-07-10 16:56:53 -04:00
gbrodman	3903abd9de	Move and/or delete a bunch of random resources (#152 )	2019-07-09 17:38:24 -04:00
Shicong Huang	8371cb838c	Add a 30s timeout for all webdriver tests (#161 ) Sometimes, the webdriver tests get stuck forever for no reason. It could be some issue in the test container but it is hard to root cause it. So, adding a 30s timeout can either trigger the retry earlier or let the test just fail.	2019-07-09 14:42:32 -04:00
Shicong Huang	8dd6797614	Read golden images from src directly (#159 ) This PR prevents Gradle from copying the golden images to build/resources/test, so the screenshot test would read golden images from src/test/resources directly and display the path in test log if the test fails. Because the path pointing to the actual file in src/ folder, the engineer can easily find it.	2019-07-09 10:49:20 -04:00
Lai Jiang	730f108e13	Upgrade to Gradle 5.5 (#158 )	2019-07-08 12:56:43 -04:00
gbrodman	e5bafddd2f	Move JS and CSS files to a Javascript source dir (#156 )	2019-07-05 12:01:16 -04:00
Lai Jiang	82f51accbd	Set beam deployment environment in GCB trigger (#157 )	2019-07-03 16:28:28 -04:00
Shicong Huang	6536631857	Remove injected credentials from invoice pipeline (#155 ) We got non-serialization object error when deploying the invoicing pipeline. It turns out that Beam requires every field in the pipeline object is serilizable. However, it is non-trivial to make GoogleCredentialsBundle serilizable because almost all of its dependency are not serilizable and not contraled by us. Also, it is non-necessary to inject the credential as the spec11 pipeline also writes output to GCS without having injected credential. So, removing the injected variable can solve the problem. TESTED=First reproduced the problem locally by deploying the invoicing pipeline with the previous code; applied this change and successfully deploy the pipeline without having any issue.	2019-07-03 15:12:48 -04:00
gbrodman	1be92968bf	Attempt login to MosAPI via all available TLDs (#141 ) * Attempt login to MosAPI via all available TLDs There's no reason why we should need a TLD as input here because it doesn't actually matter which one we use (they all have the same password). * Refactor the TLD loop and change cron jobs * Re-throw the last exception if one exists * Fix tests and exception * Remove alpha cron job	2019-07-03 14:25:39 -04:00
Ben McIlwain	0564b207f2	Prevent accidentally using full Drive URL as folder ID (#144 ) * Prevent accidentally using full Drive URL as folder ID	2019-07-03 14:22:15 -04:00
gbrodman	8afc4f4d3d	Enforce that source files end in a newline (#153 )	2019-07-03 12:16:11 -04:00