Compare commits

..
Author SHA1 Message Date
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
722bf3fcb8 Bump engine.io from 6.2.0 to 6.2.1 in /console-webapp (#1895)
Bumps [engine.io](https://github.com/socketio/engine.io) from 6.2.0 to 6.2.1.
- [Release notes](https://github.com/socketio/engine.io/releases)
- [Changelog](https://github.com/socketio/engine.io/blob/main/CHANGELOG.md)
- [Commits](https://github.com/socketio/engine.io/compare/6.2.0...6.2.1)

---
updated-dependencies:
- dependency-name: engine.io
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-05 21:47:11 -05:00
Pavlo TkachandGitHub 274ae57385 Fix billing pipeline first month scheduling (#1891)
* Fix billing pipeline first month scheduling

* compare to expansion next month

* use yoda date comparison

* update cursor time to be mid of day
2023-01-05 21:45:56 -05:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
ecd1dd81a2 Bump loader-utils from 2.0.2 to 2.0.4 in /console-webapp (#1894)
Bumps [loader-utils](https://github.com/webpack/loader-utils) from 2.0.2 to 2.0.4.
- [Release notes](https://github.com/webpack/loader-utils/releases)
- [Changelog](https://github.com/webpack/loader-utils/blob/v2.0.4/CHANGELOG.md)
- [Commits](https://github.com/webpack/loader-utils/compare/v2.0.2...v2.0.4)

---
updated-dependencies:
- dependency-name: loader-utils
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-05 21:41:03 -05:00
Pavlo TkachandGitHub 8f844cb437 Add new console backbone (#1876)
* Create console webapp, add material ui, initialize tlds and home pages

* Add servlet for serving console static files

* Add console tasks to nomulus tasks routine

* Fix for base console GCP base usr

* Add jetty dep and update_dependency.sh

* Update console servlet url

* verified fix for static url handler

* Another deps update

* Add Copyright

* Remove unused variable

* Update titles to Nomulus Console
2023-01-05 16:23:40 -05:00
Weimin YuandGitHub e1864bee4e Disable id preassignment when writing to sql (#1893)
* Disable id preassignment when writing to sql

See b/264416932 for details.
2023-01-05 11:04:38 -05:00
sarahcaseybotandGitHub 18641327de Add default tokens to TLD using nomulus tool (#1888)
* Add defualt tokens to TLD using nomulus tool

* add test
2023-01-04 13:25:25 -05:00
gbrodmanandGitHub db9525903d Add an optional IAP-enabled ID token when using the Nomulus tool (#1887)
We can use the saved refresh token associated with the nomulus tool to
request an ID token with an audience of the IAP client in order to
satisfy IAP with with the Nomulus tool.

Note: this requires that the user of the Nomulus tool, e.g.
"gbrodman@google.com" has a User object stored in SQL.

Tested on QA
2023-01-04 11:43:31 -05:00
Lai JiangandGitHub 9b2431807c Drop GAE user ID from RegistryPoc table (#1861) 2022-12-28 09:42:46 -05:00
Weimin YuandGitHub f9659af3b2 Remove aggressive check in RegistryJpaIO.Write (#1889) 2022-12-22 17:12:09 -05:00
Ben McIlwainandGitHub 0aeb92ee16 Standardize hostname handling in URS command (#1886) 2022-12-19 16:22:52 -05:00
sarahcaseybotandGitHub 4ede5f0c8a Prevent saving duplicate rows in spec11 pipeline (#1810)
* Prevent saving duplicate rows in spec11 pipeline

* Chain applies together
2022-12-15 15:51:28 -05:00
Lai JiangandGitHub 2292bfcaed Remove pipline servlet endpoint mapping (#1885) 2022-12-14 16:28:57 -05:00
sarahcaseybotandGitHub b056d2945f Add package monitoring for active domain limits (#1867)
* Add monitoring for package active domain limits

* Reformat action class

* Fix a bunch of nits

* Change native query to HQL
2022-12-14 12:10:10 -05:00
Lai JiangandGitHub b8b1dce40a Remove TransactionManagerFactory.jpaTm() (#1883)
Since JPA is the only TM now, there's no point distinguishing tm() from
jpaTm().
2022-12-13 14:37:46 -05:00
Lai JiangandGitHub d7e008a4af Move SQL files to resources from java (#1879)
This is similar to where we store the SQL files for beam pipelines, and
frankly makes more sense. Also streamlined the use of the API to read
SQL files from a jar.
2022-12-12 16:32:27 -05:00
Lai JiangandGitHub d943ebd423 Add CodeQL workflow (#1884)
This is based on #1864. Also removes the LGTM setup as it is turning
down.
2022-12-12 15:52:47 -05:00
gbrodmanandGitHub 0ff9543efa Use standard Java thread creation in Concurrent (#1880)
The AppEngine thread factory is only useful if we can't create our own
(this is no longer the case) or if we need access to AppEngine APIs
(this is no longer the case).

The Concurrent class is only used by the DNS writer and the
CreateGroupsAction.
2022-12-12 15:42:02 -05:00
gbrodmanandGitHub bb54ace0c0 Change the cookie auth mechanism to use IAP-provided JWTs (#1877) 2022-12-12 13:51:33 -05:00
Lai JiangandGitHub cfee4713ed Remove sharding parameter from RegistryJpaIO (#1856)
This parameter is misleading and does not do what it purports to do.
Namely, it does not impact the level of parallelism. Given the input n for this
parameter, and m for the batch size, the elements are divided (keyed) into n
groups, each of which are then spread evenly across all threads, which
are eventually in turn batched into batches with size m:

https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/GroupIntoBatches.java#L227

This is also evident in the implementation itself, where the ShardedKey
is determined by the unique number for a worker/thread combo and the
original key:

https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/GroupIntoBatches.java#L268

Using a more concrete example, suppose we have 100 elements and 10
worker threads, with a target batch size of 5. If the "shard" number is set to
1, we first spread the 100 elements across 10 threads, resulting in 10
elements per thread, each thread then batches the elements into 2
batches of size 5.

If the "shard" number is set to 2, the 100 elements are first divided into 2
"shards" of 50 each. Each "shard" is then distributed within the 10
threads, resulting in 5 elements per "shard" per thread. They then get
turned into 1 batch per "shard" per thread. In the end, each thread
still processes 2 batches, even though they are from 2 different "shards".

Therefore this "shard" number does not perform horizontal partitioning
that one normally associates with sharding, and provides no
performance benefits but rather confuses the user.

It is also suggested that using withShardedKey() alone is already
sufficient to achieve auto-sharding within the keyed group. There is no
need to manually divide the input by keying them differently based on
the "shard" number specified:

https://youtu.be/jses0W4Zalc?t=967
2022-12-12 11:55:24 -05:00
Lai JiangandGitHub dc7d123f6d Remove @EmptySetToNull (#1878)
This annotation was only used in cross-database comparison.

<!-- Reviewable:start -->
- - -
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1878)
<!-- Reviewable:end -->
2022-12-09 15:07:00 -05:00
235 changed files with 27960 additions and 5649 deletions
+63
View File
@@ -0,0 +1,63 @@
name: "CodeQL"
on:
push:
branches: [ 'master' ]
pull_request:
# The branches below must be a subset of the branches above
branches: [ 'master' ]
schedule:
- cron: '24 4 * * 2'
jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
security-events: write
strategy:
fail-fast: false
matrix:
language: [ 'java', 'javascript', 'python' ]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support
steps:
- name: Checkout repository
uses: actions/checkout@v3
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.
# Details on CodeQL's query packs refer to : https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs
queries: security-and-quality
# Autobuild attempts to build any compiled languages (C/C++, C#, Go, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v2
# ️ Command-line programs to run using the OS shell.
# 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
# If the Autobuild fails above, remove it and uncomment the following three lines.
# modify them (or add more) to build your code if your project, please refer to the EXAMPLE below for guidance.
# - run: |
# echo "Run, Build Application using script"
# ./location_of_script_within_repo/buildscript.sh
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
with:
category: "/language:${{matrix.language}}"
-6
View File
@@ -1,6 +0,0 @@
extraction:
java:
prepare:
packages: "npm"
index:
java_version: "11"
+3 -3
View File
@@ -1,8 +1,8 @@
# Nomulus
| Internal Build | FOSS Build | LGTM | License | Code Search |
|:--------------:|:----------:|:----:|:-------:|:-----------:|
|[![Build Status for Google Registry internal build](https://storage.googleapis.com/domain-registry-kokoro/internal/build.svg)](https://storage.googleapis.com/domain-registry-kokoro/internal/index.html)|[![Build Status for the open source build](https://storage.googleapis.com/domain-registry-kokoro/foss/build.svg)](https://storage.googleapis.com/domain-registry-kokoro/foss/index.html)|[![Total alerts](https://img.shields.io/lgtm/alerts/g/google/nomulus.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/google/nomulus/alerts/)|[![License for this repo](https://img.shields.io/github/license/google/nomulus.svg)](https://github.com/google/nomulus/blob/master/LICENSE)|[![Link to Code Search](https://www.gstatic.com/devopsconsole/images/oss/favicons/oss-32x32.png)](https://cs.opensource.google/nomulus/nomulus)|
| Internal Build | FOSS Build | License | Code Search |
|:--------------:|:----------:|:-------:|:-----------:|
|[![Build Status for Google Registry internal build](https://storage.googleapis.com/domain-registry-kokoro/internal/build.svg)](https://storage.googleapis.com/domain-registry-kokoro/internal/index.html)|[![Build Status for the open source build](https://storage.googleapis.com/domain-registry-kokoro/foss/build.svg)](https://storage.googleapis.com/domain-registry-kokoro/foss/index.html)|[![License for this repo](https://img.shields.io/github/license/google/nomulus.svg)](https://github.com/google/nomulus/blob/master/LICENSE)|[![Link to Code Search](https://www.gstatic.com/devopsconsole/images/oss/favicons/oss-32x32.png)](https://cs.opensource.google/nomulus/nomulus)|
![Nomulus logo](./nomulus-logo.png)
+5
View File
@@ -47,6 +47,10 @@ war {
if (project.path == ":services:default") {
war {
from("${rootDir}/console-webapp/dist/console-webapp") {
include "**/*"
into("console")
}
from("${coreResourcesDir}/google/registry/ui") {
include "registrar_bin.js"
if (environment != "production") {
@@ -101,6 +105,7 @@ explodeWar.doLast {
rootProject.deploy.dependsOn appengineDeployAll
rootProject.stage.dependsOn appengineStage
tasks['war'].dependsOn ':console-webapp:buildConsoleWebappProd'
tasks['war'].dependsOn ':core:compileProdJS'
tasks['war'].dependsOn ':core:processResources'
tasks['war'].dependsOn ':core:jar'
+1
View File
@@ -551,6 +551,7 @@ task coreDev {
dependsOn 'javadoc'
dependsOn 'checkDependenciesDotGradle'
dependsOn 'checkLicense'
dependsOn ':console-webapp:runConsoleWebappUnitTests'
dependsOn ':core:check'
dependsOn 'assemble'
}
+3 -3
View File
@@ -25,7 +25,7 @@ import textwrap
import re
# We should never analyze any generated files
UNIVERSALLY_SKIPPED_PATTERNS = {"/build/", "cloudbuild-caches", "/out/", ".git/", ".gradle/"}
UNIVERSALLY_SKIPPED_PATTERNS = {"/build/", "cloudbuild-caches", "/out/", ".git/", ".gradle/", "/dist/", "karma.conf.js", "polyfills.ts", "test.ts"}
# We can't rely on CI to have the Enum package installed so we do this instead.
FORBIDDEN = 1
REQUIRED = 2
@@ -86,7 +86,7 @@ PRESUBMITS = {
# License check
PresubmitCheck(
r".*Copyright 20\d{2} The Nomulus Authors\. All Rights Reserved\.",
("java", "js", "soy", "sql", "py", "sh", "gradle"), {
("java", "js", "soy", "sql", "py", "sh", "gradle", "ts"), {
".git", "/build/", "/generated/", "/generated_tests/",
"node_modules/", "LoggerConfig.java", "registrar_bin.",
"registrar_dbg.", "google-java-format-diff.py",
@@ -95,7 +95,7 @@ PRESUBMITS = {
"File did not include the license header.",
# Files must end in a newline
PresubmitCheck(r".*\n$", ("java", "js", "soy", "sql", "py", "sh", "gradle"),
PresubmitCheck(r".*\n$", ("java", "js", "soy", "sql", "py", "sh", "gradle", "ts"),
{"node_modules/"}, REQUIRED):
"Source files must end in a newline.",
+16
View File
@@ -0,0 +1,16 @@
# This file is used by the build system to adjust CSS and JS output to support the specified browsers below.
# For additional information regarding the format and rule options, please see:
# https://github.com/browserslist/browserslist#queries
# For the full list of supported browsers by the Angular framework, please see:
# https://angular.io/guide/browser-support
# You can see what browsers were selected by your queries by running:
# npx browserslist
last 1 Chrome version
last 1 Firefox version
last 2 Edge major versions
last 2 Safari major versions
last 2 iOS major versions
Firefox ESR
+16
View File
@@ -0,0 +1,16 @@
# Editor configuration, see https://editorconfig.org
root = true
[*]
charset = utf-8
indent_style = space
indent_size = 2
insert_final_newline = true
trim_trailing_whitespace = true
[*.ts]
quote_type = single
[*.md]
max_line_length = off
trim_trailing_whitespace = false
+42
View File
@@ -0,0 +1,42 @@
# See http://help.github.com/ignore-files/ for more about ignoring files.
# Compiled output
/dist
/tmp
/out-tsc
/bazel-out
# Node
/node_modules
npm-debug.log
yarn-error.log
# IDEs and editors
.idea/
.project
.classpath
.c9/
*.launch
.settings/
*.sublime-workspace
# Visual Studio Code
.vscode/*
!.vscode/settings.json
!.vscode/tasks.json
!.vscode/launch.json
!.vscode/extensions.json
.history/*
# Miscellaneous
/.angular/cache
.sass-cache/
/connect.lock
/coverage
/libpeerconnection.log
testem.log
/typings
# System files
.DS_Store
Thumbs.db
+27
View File
@@ -0,0 +1,27 @@
# ConsoleWebapp
This project was generated with [Angular CLI](https://github.com/angular/angular-cli) version 14.2.3.
## Development server
Run `ng serve` for a dev server. Navigate to `http://localhost:4200/`. The application will automatically reload if you change any of the source files.
## Code scaffolding
Run `ng generate component component-name` to generate a new component. You can also use `ng generate directive|pipe|service|class|guard|interface|enum|module`.
## Build
Run `ng build` to build the project. The build artifacts will be stored in the `dist/` directory.
## Running unit tests
Run `ng test` to execute the unit tests via [Karma](https://karma-runner.github.io).
## Running end-to-end tests
Run `ng e2e` to execute the end-to-end tests via a platform of your choice. To use this command, you need to first add a package that implements end-to-end testing capabilities.
## Further help
To get more help on the Angular CLI use `ng help` or go check out the [Angular CLI Overview and Command Reference](https://angular.io/cli) page.
+109
View File
@@ -0,0 +1,109 @@
{
"$schema": "./node_modules/@angular/cli/lib/config/schema.json",
"version": 1,
"newProjectRoot": "projects",
"projects": {
"console-webapp": {
"projectType": "application",
"schematics": {
"@schematics/angular:component": {
"style": "less"
}
},
"root": "",
"sourceRoot": "src",
"prefix": "app",
"architect": {
"build": {
"builder": "@angular-devkit/build-angular:browser",
"options": {
"outputPath": "dist/console-webapp",
"index": "src/index.html",
"main": "src/main.ts",
"polyfills": "src/polyfills.ts",
"tsConfig": "tsconfig.app.json",
"inlineStyleLanguage": "less",
"assets": [
"src/favicon.ico",
"src/assets"
],
"styles": [
"./node_modules/@angular/material/prebuilt-themes/indigo-pink.css",
"src/styles.less"
],
"scripts": []
},
"configurations": {
"production": {
"budgets": [
{
"type": "initial",
"maximumWarning": "500kb",
"maximumError": "1mb"
},
{
"type": "anyComponentStyle",
"maximumWarning": "2kb",
"maximumError": "4kb"
}
],
"fileReplacements": [
{
"replace": "src/environments/environment.ts",
"with": "src/environments/environment.prod.ts"
}
],
"outputHashing": "all"
},
"development": {
"buildOptimizer": false,
"optimization": false,
"vendorChunk": true,
"extractLicenses": false,
"sourceMap": true,
"namedChunks": true
}
},
"defaultConfiguration": "production"
},
"serve": {
"builder": "@angular-devkit/build-angular:dev-server",
"configurations": {
"production": {
"browserTarget": "console-webapp:build:production"
},
"development": {
"browserTarget": "console-webapp:build:development"
}
},
"defaultConfiguration": "development"
},
"extract-i18n": {
"builder": "@angular-devkit/build-angular:extract-i18n",
"options": {
"browserTarget": "console-webapp:build"
}
},
"test": {
"builder": "@angular-devkit/build-angular:karma",
"options": {
"main": "src/test.ts",
"polyfills": "src/polyfills.ts",
"tsConfig": "tsconfig.spec.json",
"karmaConfig": "karma.conf.js",
"inlineStyleLanguage": "less",
"assets": [
"src/favicon.ico",
"src/assets"
],
"styles": [
"./node_modules/@angular/material/prebuilt-themes/indigo-pink.css",
"src/styles.less"
],
"scripts": []
}
}
}
}
}
}
+54
View File
@@ -0,0 +1,54 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
def consoleDir = "${rootDir}/console-webapp"
clean {
delete "${consoleDir}/node_modules"
delete "${consoleDir}/dist"
}
task npmInstallDeps(type: Exec) {
workingDir "${consoleDir}/"
executable 'npm'
args 'i', '--no-audit', '--no-fund', '--loglevel=error'
}
task runConsoleWebappLocally(type: Exec) {
workingDir "${consoleDir}/"
executable 'npm'
args 'run', 'start:dev'
}
task runConsoleWebappUnitTests(type: Exec) {
workingDir "${consoleDir}/"
executable 'npm'
args 'run', 'test'
}
task buildConsoleWebappNonProd(type: Exec) {
workingDir "${consoleDir}/"
executable 'npm'
args 'run', 'build'
}
// Keeping the same as non prod for now before we figure out optimization we want to include
task buildConsoleWebappProd(type: Exec) {
workingDir "${consoleDir}/"
executable 'npm'
args 'run', 'build'
}
tasks.runConsoleWebappUnitTests.dependsOn(tasks.npmInstallDeps)
tasks.buildConsoleWebappProd.dependsOn(tasks.npmInstallDeps)
@@ -0,0 +1,4 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
empty=classpath
+7
View File
@@ -0,0 +1,7 @@
{
"/registrar":
{
"target": "http://localhost:8080/registrar",
"secure": false
}
}
+48
View File
@@ -0,0 +1,48 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
antlr:antlr:2.7.7=checkstyle
com.github.ben-manes.caffeine:caffeine:2.7.0=annotationProcessor,errorprone,testAnnotationProcessor
com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,errorprone,testAnnotationProcessor
com.google.auto:auto-common:0.10=annotationProcessor,errorprone,testAnnotationProcessor
com.google.code.findbugs:jFormatString:3.0.0=annotationProcessor,errorprone,testAnnotationProcessor
com.google.code.findbugs:jsr305:3.0.2=annotationProcessor,checkstyle,errorprone,testAnnotationProcessor
com.google.errorprone:error_prone_annotation:2.3.4=annotationProcessor,errorprone,testAnnotationProcessor
com.google.errorprone:error_prone_annotations:2.3.4=annotationProcessor,checkstyle,errorprone,testAnnotationProcessor
com.google.errorprone:error_prone_check_api:2.3.4=annotationProcessor,errorprone,testAnnotationProcessor
com.google.errorprone:error_prone_core:2.3.4=annotationProcessor,errorprone,testAnnotationProcessor
com.google.errorprone:error_prone_type_annotations:2.3.4=annotationProcessor,errorprone,testAnnotationProcessor
com.google.guava:failureaccess:1.0.1=annotationProcessor,checkstyle,errorprone,testAnnotationProcessor
com.google.guava:guava:27.0.1-jre=annotationProcessor,errorprone,testAnnotationProcessor
com.google.guava:guava:29.0-jre=checkstyle
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,errorprone,testAnnotationProcessor
com.google.j2objc:j2objc-annotations:1.1=annotationProcessor,errorprone,testAnnotationProcessor
com.google.j2objc:j2objc-annotations:1.3=checkstyle
com.google.protobuf:protobuf-java:3.4.0=annotationProcessor,errorprone,testAnnotationProcessor
com.googlecode.java-diff-utils:diffutils:1.3.0=annotationProcessor,errorprone,testAnnotationProcessor
com.puppycrawl.tools:checkstyle:8.37=checkstyle
commons-beanutils:commons-beanutils:1.9.4=checkstyle
commons-collections:commons-collections:3.2.2=checkstyle
info.picocli:picocli:4.5.2=checkstyle
net.sf.saxon:Saxon-HE:10.3=checkstyle
org.antlr:antlr4-runtime:4.8-1=checkstyle
org.checkerframework:checker-qual:2.11.1=checkstyle
org.checkerframework:checker-qual:3.0.0=annotationProcessor,errorprone,testAnnotationProcessor
org.checkerframework:dataflow:3.0.0=annotationProcessor,errorprone,testAnnotationProcessor
org.checkerframework:javacutil:3.0.0=annotationProcessor,errorprone,testAnnotationProcessor
org.codehaus.mojo:animal-sniffer-annotations:1.17=annotationProcessor,errorprone,testAnnotationProcessor
org.jacoco:org.jacoco.agent:0.8.6=jacocoAgent,jacocoAnt
org.jacoco:org.jacoco.ant:0.8.6=jacocoAnt
org.jacoco:org.jacoco.core:0.8.6=jacocoAnt
org.jacoco:org.jacoco.report:0.8.6=jacocoAnt
org.javassist:javassist:3.26.0-GA=checkstyle
org.ow2.asm:asm-analysis:8.0.1=jacocoAnt
org.ow2.asm:asm-commons:8.0.1=jacocoAnt
org.ow2.asm:asm-tree:8.0.1=jacocoAnt
org.ow2.asm:asm:8.0.1=jacocoAnt
org.pcollections:pcollections:2.1.2=annotationProcessor,errorprone,testAnnotationProcessor
org.plumelib:plume-util:1.0.6=annotationProcessor,errorprone,testAnnotationProcessor
org.plumelib:reflection-util:0.0.2=annotationProcessor,errorprone,testAnnotationProcessor
org.plumelib:require-javadoc:0.1.0=annotationProcessor,errorprone,testAnnotationProcessor
org.reflections:reflections:0.9.12=checkstyle
empty=archives,compileClasspath,default,deploy_jar,errorproneJavac,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
+44
View File
@@ -0,0 +1,44 @@
// Karma configuration file, see link for more information
// https://karma-runner.github.io/1.0/config/configuration-file.html
module.exports = function (config) {
config.set({
basePath: '',
frameworks: ['jasmine', '@angular-devkit/build-angular'],
plugins: [
require('karma-jasmine'),
require('karma-chrome-launcher'),
require('karma-jasmine-html-reporter'),
require('karma-coverage'),
require('@angular-devkit/build-angular/plugins/karma')
],
client: {
jasmine: {
// you can add configuration options for Jasmine here
// the possible options are listed at https://jasmine.github.io/api/edge/Configuration.html
// for example, you can disable the random execution with `random: false`
// or set a specific seed with `seed: 4321`
},
clearContext: false // leave Jasmine Spec Runner output visible in browser
},
jasmineHtmlReporter: {
suppressAll: true // removes the duplicated traces
},
coverageReporter: {
dir: require('path').join(__dirname, './coverage/console-webapp'),
subdir: '.',
reporters: [
{ type: 'html' },
{ type: 'text-summary' }
]
},
reporters: ['progress', 'kjhtml'],
port: 9876,
colors: true,
logLevel: config.LOG_INFO,
autoWatch: true,
browsers: ['Chrome'],
singleRun: false,
restartOnFileChange: true
});
};
+20701
View File
File diff suppressed because it is too large Load Diff
+44
View File
@@ -0,0 +1,44 @@
{
"name": "console-webapp",
"version": "0.0.0",
"scripts": {
"ng": "ng",
"start": "ng serve",
"build": "ng build --base-href=/console/",
"build:local": "ng build --base-href=/default/console/",
"watch": "ng build --watch --configuration development",
"test": "ng test --browsers=ChromeHeadless --watch=false",
"run:dev": "",
"start:dev": "concurrently \"./../gradlew :core:runTestServer\" \"ng serve --proxy-config dev-proxy.config.json\""
},
"private": true,
"dependencies": {
"@angular/animations": "^14.2.0",
"@angular/cdk": "^14.2.2",
"@angular/common": "^14.2.0",
"@angular/compiler": "^14.2.0",
"@angular/core": "^14.2.0",
"@angular/forms": "^14.2.0",
"@angular/material": "^14.2.2",
"@angular/platform-browser": "^14.2.0",
"@angular/platform-browser-dynamic": "^14.2.0",
"@angular/router": "^14.2.0",
"rxjs": "~7.5.0",
"tslib": "^2.3.0",
"zone.js": "~0.11.4"
},
"devDependencies": {
"@angular-devkit/build-angular": "^14.2.3",
"@angular/cli": "~14.2.3",
"@angular/compiler-cli": "^14.2.0",
"@types/jasmine": "~4.0.0",
"concurrently": "^7.6.0",
"jasmine-core": "~4.3.0",
"karma": "~6.4.0",
"karma-chrome-launcher": "~3.1.0",
"karma-coverage": "~2.2.0",
"karma-jasmine": "~5.1.0",
"karma-jasmine-html-reporter": "~2.0.0",
"typescript": "~4.7.2"
}
}
@@ -0,0 +1,29 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { NgModule } from '@angular/core';
import { RouterModule, Routes } from '@angular/router';
import {TldsComponent} from './tlds/tlds.component';
import {HomeComponent} from './home/home.component';
const routes: Routes = [
{ path: 'home', component: HomeComponent },
{ path: 'tlds', component: TldsComponent },
];
@NgModule({
imports: [RouterModule.forRoot(routes)],
exports: [RouterModule]
})
export class AppRoutingModule { }
+14
View File
@@ -0,0 +1,14 @@
<div class="toolbar" role="banner">
Nomulus Console
</div>
<div class="content" role="main">
<nav>
<ul>
<li><a routerLink="/home" routerLinkActive="active" ariaCurrentWhenActive="page">Home page</a></li>
<li><a routerLink="/tlds" routerLinkActive="active" ariaCurrentWhenActive="page">TLDs</a></li>
</ul>
</nav>
</div>
<router-outlet></router-outlet>
+35
View File
@@ -0,0 +1,35 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
:host {
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol";
font-size: 14px;
color: #333;
box-sizing: border-box;
-webkit-font-smoothing: antialiased;
-moz-osx-font-smoothing: grayscale;
}
h1,
h2,
h3,
h4,
h5,
h6 {
margin: 8px 0;
}
p {
margin: 0;
}
@@ -0,0 +1,37 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { TestBed } from '@angular/core/testing';
import { RouterTestingModule } from '@angular/router/testing';
import { AppComponent } from './app.component';
describe('AppComponent', () => {
beforeEach(async () => {
await TestBed.configureTestingModule({
imports: [
RouterTestingModule
],
declarations: [
AppComponent
],
}).compileComponents();
});
it('should create the app', () => {
const fixture = TestBed.createComponent(AppComponent);
const app = fixture.componentInstance;
expect(app).toBeTruthy();
});
});
+24
View File
@@ -0,0 +1,24 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { Component } from '@angular/core';
@Component({
selector: 'app-root',
templateUrl: './app.component.html',
styleUrls: ['./app.component.less']
})
export class AppComponent {
}
+41
View File
@@ -0,0 +1,41 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { NgModule } from '@angular/core';
import { BrowserModule } from '@angular/platform-browser';
import { AppRoutingModule } from './app-routing.module';
import { AppComponent } from './app.component';
import { BrowserAnimationsModule } from '@angular/platform-browser/animations';
import {MaterialModule} from './material.module';
import { HomeComponent } from './home/home.component';
import { TldsComponent } from './tlds/tlds.component';
@NgModule({
declarations: [
AppComponent,
HomeComponent,
TldsComponent,
],
imports: [
MaterialModule,
BrowserModule,
AppRoutingModule,
BrowserAnimationsModule
],
providers: [],
bootstrap: [AppComponent]
})
export class AppModule { }
@@ -0,0 +1,14 @@
<h3>Recent Activity</h3>
<table mat-table [dataSource]="dataSource" class="mat-elevation-z8 console-home__activity">
<ng-container *ngFor="let column of columns" [matColumnDef]="column.columnDef">
<th mat-header-cell *matHeaderCellDef>
{{column.header}}
</th>
<td mat-cell *matCellDef="let row">
{{column.cell(row)}}
</td>
</ng-container>
<tr mat-header-row *matHeaderRowDef="displayedColumns"></tr>
<tr mat-row *matRowDef="let row; columns: displayedColumns;"></tr>
</table>
@@ -0,0 +1,14 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
@@ -0,0 +1,39 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { ComponentFixture, TestBed } from '@angular/core/testing';
import { HomeComponent } from './home.component';
import {MaterialModule} from '../material.module';
describe('HomeComponent', () => {
let component: HomeComponent;
let fixture: ComponentFixture<HomeComponent>;
beforeEach(async () => {
await TestBed.configureTestingModule({
imports: [MaterialModule],
declarations: [ HomeComponent ]
})
.compileComponents();
fixture = TestBed.createComponent(HomeComponent);
component = fixture.componentInstance;
fixture.detectChanges();
});
it('should create', () => {
expect(component).toBeTruthy();
});
});
@@ -0,0 +1,94 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { Component } from '@angular/core';
export interface ActivityRecord {
eventType: string;
userName: string;
registrarName: string;
timestamp: string;
details: string
}
const MOCK_DATA: ActivityRecord[] = [
{eventType: "Export DUMS", userName:"user3", registrarName: "registrar1", timestamp: "2022-03-15T19:46:39.007", details: "All Domains under management exported as .csv file" },
{eventType: "Update Contact", userName:"user3", registrarName: "registrar1", timestamp: "2022-03-15T19:46:39.007", details: "All Domains under management exported as .csv file" },
{eventType: "Delete Domain", userName:"user3", registrarName: "registrar1", timestamp: "2022-03-15T19:46:39.007", details: "All Domains under management exported as .csv file" },
{eventType: "Export DUMS", userName:"user3", registrarName: "registrar1", timestamp: "2022-03-15T19:46:39.007", details: "All Domains under management exported as .csv file" },
{eventType: "Update Contact", userName:"user3", registrarName: "registrar1", timestamp: "2022-03-15T19:46:39.007", details: "All Domains under management exported as .csv file" },
{eventType: "Delete Domain", userName:"user3", registrarName: "registrar1", timestamp: "2022-03-15T19:46:39.007", details: "All Domains under management exported as .csv file" },
{eventType: "Export DUMS", userName:"user3", registrarName: "registrar1", timestamp: "2022-03-15T19:46:39.007", details: "All Domains under management exported as .csv file" },
{eventType: "Update Contact", userName:"user3", registrarName: "registrar1", timestamp: "2022-03-15T19:46:39.007", details: "All Domains under management exported as .csv file" },
{eventType: "Delete Domain", userName:"user3", registrarName: "registrar1", timestamp: "2022-03-15T19:46:39.007", details: "All Domains under management exported as .csv file" },
{eventType: "Export DUMS", userName:"user3", registrarName: "registrar1", timestamp: "2022-03-15T19:46:39.007", details: "All Domains under management exported as .csv file" },
{eventType: "Update Contact", userName:"user3", registrarName: "registrar1", timestamp: "2022-03-15T19:46:39.007", details: "All Domains under management exported as .csv file" },
{eventType: "Delete Domain", userName:"user3", registrarName: "registrar1", timestamp: "2022-03-15T19:46:39.007", details: "All Domains under management exported as .csv file" },
{eventType: "Export DUMS", userName:"user3", registrarName: "registrar1", timestamp: "2022-03-15T19:46:39.007", details: "All Domains under management exported as .csv file" },
{eventType: "Update Contact", userName:"user3", registrarName: "registrar1", timestamp: "2022-03-15T19:46:39.007", details: "All Domains under management exported as .csv file" },
{eventType: "Delete Domain", userName:"user3", registrarName: "registrar1", timestamp: "2022-03-15T19:46:39.007", details: "All Domains under management exported as .csv file" },
{eventType: "Export DUMS", userName:"user3", registrarName: "registrar1", timestamp: "2022-03-15T19:46:39.007", details: "All Domains under management exported as .csv file" },
{eventType: "Update Contact", userName:"user3", registrarName: "registrar1", timestamp: "2022-03-15T19:46:39.007", details: "All Domains under management exported as .csv file" },
{eventType: "Delete Domain", userName:"user3", registrarName: "registrar1", timestamp: "2022-03-15T19:46:39.007", details: "All Domains under management exported as .csv file" },
{eventType: "Export DUMS", userName:"user3", registrarName: "registrar1", timestamp: "2022-03-15T19:46:39.007", details: "All Domains under management exported as .csv file" },
{eventType: "Update Contact", userName:"user3", registrarName: "registrar1", timestamp: "2022-03-15T19:46:39.007", details: "All Domains under management exported as .csv file" },
{eventType: "Delete Domain", userName:"user3", registrarName: "registrar1", timestamp: "2022-03-15T19:46:39.007", details: "All Domains under management exported as .csv file" },
{eventType: "Export DUMS", userName:"user3", registrarName: "registrar1", timestamp: "2022-03-15T19:46:39.007", details: "All Domains under management exported as .csv file" },
{eventType: "Update Contact", userName:"user3", registrarName: "registrar1", timestamp: "2022-03-15T19:46:39.007", details: "All Domains under management exported as .csv file" },
{eventType: "Delete Domain", userName:"user3", registrarName: "registrar1", timestamp: "2022-03-15T19:46:39.007", details: "All Domains under management exported as .csv file" },
];
@Component({
selector: 'app-home',
templateUrl: './home.component.html',
styleUrls: ['./home.component.less']
})
export class HomeComponent {
columns = [
{
columnDef: 'eventType',
header: 'Event Type',
cell:(record: ActivityRecord) => `${record.eventType}`,
},
{
columnDef: 'userName',
header: 'User',
cell: (record: ActivityRecord) => `${record.userName}`,
},
{
columnDef: 'registrarName',
header: 'Registrar',
cell: (record: ActivityRecord) => `${record.registrarName}`,
},
{
columnDef: 'timestamp',
header: 'Timestamp',
cell: (record: ActivityRecord) => `${record.timestamp}`,
},
{
columnDef: 'details',
header: 'Details',
cell: (record: ActivityRecord) => `${record.details}`,
},
];
dataSource = MOCK_DATA;
displayedColumns = this.columns.map(c => c.columnDef);
constructor() {
}
}
+26
View File
@@ -0,0 +1,26 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import {NgModule} from '@angular/core';
import {MatCardModule} from '@angular/material/card';
import {MatTableModule} from '@angular/material/table';
const MATERIAL_MODULES = [
MatCardModule,
MatTableModule,
];
@NgModule({imports: MATERIAL_MODULES, exports: MATERIAL_MODULES})
export class MaterialModule {
}
@@ -0,0 +1,24 @@
<div class="console-tlds__cards">
<mat-card class="console-tlds__card">
<mat-card-title>.how</mat-card-title>
<mat-card-subtitle>A place for thinkers, tinkerers, and knowledge seekers</mat-card-subtitle>
<mat-card-actions class="console-tlds__card-links">
<a title="Onboarding Now" href="#" target="_blank" rel="noopener">Onboarding Now</a>
<a title="Marketing Materials" href="#" target="_blank" rel="noopener">Marketing Materials</a>
<a title="Visit get.how for more information" href="#" target="_blank" rel="noopener">Visit get.how for more information</a>
</mat-card-actions>
</mat-card>
</div>
<div class="console-tlds__cards">
<mat-card class="console-tlds__card">
<mat-card-title>.soy</mat-card-title>
<mat-card-subtitle>A place for thinkers, tinkerers, and knowledge seekers</mat-card-subtitle>
<mat-card-actions class="console-tlds__card-links">
<a title="Onboarding Now" href="#" target="_blank" rel="noopener">Onboarding Now</a>
<a title="Marketing Materials" href="#" target="_blank" rel="noopener">Marketing Materials</a>
<a title="Visit get.how for more information" href="#" target="_blank" rel="noopener">Visit iam.soy for more information</a>
</mat-card-actions>
</mat-card>
</div>
@@ -0,0 +1,28 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
.console-tlds {
&__cards {
display: flex;
border-top: 1px solid #ddd;
padding: 1rem;
}
&__card {
max-width: 300px;
}
&__card-links {
display: flex;
flex-direction: column;
}
}
@@ -0,0 +1,39 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { ComponentFixture, TestBed } from '@angular/core/testing';
import { TldsComponent } from './tlds.component';
import {MaterialModule} from '../material.module';
describe('TldsComponent', () => {
let component: TldsComponent;
let fixture: ComponentFixture<TldsComponent>;
beforeEach(async () => {
await TestBed.configureTestingModule({
imports: [MaterialModule],
declarations: [ TldsComponent ]
})
.compileComponents();
fixture = TestBed.createComponent(TldsComponent);
component = fixture.componentInstance;
fixture.detectChanges();
});
it('should create', () => {
expect(component).toBeTruthy();
});
});
@@ -0,0 +1,29 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { Component, OnInit } from '@angular/core';
@Component({
selector: 'app-tlds',
templateUrl: './tlds.component.html',
styleUrls: ['./tlds.component.less']
})
export class TldsComponent implements OnInit {
constructor() { }
ngOnInit(): void {
}
}
View File
@@ -0,0 +1,17 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
export const environment = {
production: true
};
@@ -0,0 +1,30 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// This file can be replaced during build by using the `fileReplacements` array.
// `ng build` replaces `environment.ts` with `environment.prod.ts`.
// The list of file replacements can be found in `angular.json`.
export const environment = {
production: false
};
/*
* For easier debugging in development mode, you can import the following file
* to ignore zone related error stack frames such as `zone.run`, `zoneDelegate.invokeTask`.
*
* This import should be commented out in production mode because it will have a negative impact
* on performance if an error is thrown.
*/
// import 'zone.js/plugins/zone-error'; // Included with Angular CLI.
Binary file not shown.

After

Width:  |  Height:  |  Size: 948 B

+16
View File
@@ -0,0 +1,16 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Nomulus Console</title>
<base href="/">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="icon" type="image/x-icon" href="favicon.ico">
<link rel="preconnect" href="https://fonts.gstatic.com">
<link href="https://fonts.googleapis.com/css2?family=Roboto:wght@300;400;500&display=swap" rel="stylesheet">
<link href="https://fonts.googleapis.com/icon?family=Material+Icons" rel="stylesheet">
</head>
<body class="mat-typography">
<app-root></app-root>
</body>
</html>
+26
View File
@@ -0,0 +1,26 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { enableProdMode } from '@angular/core';
import { platformBrowserDynamic } from '@angular/platform-browser-dynamic';
import { AppModule } from './app/app.module';
import { environment } from './environments/environment';
if (environment.production) {
enableProdMode();
}
platformBrowserDynamic().bootstrapModule(AppModule)
.catch(err => console.error(err));
+53
View File
@@ -0,0 +1,53 @@
/**
* This file includes polyfills needed by Angular and is loaded before the app.
* You can add your own extra polyfills to this file.
*
* This file is divided into 2 sections:
* 1. Browser polyfills. These are applied before loading ZoneJS and are sorted by browsers.
* 2. Application imports. Files imported after ZoneJS that should be loaded before your main
* file.
*
* The current setup is for so-called "evergreen" browsers; the last versions of browsers that
* automatically update themselves. This includes recent versions of Safari, Chrome (including
* Opera), Edge on the desktop, and iOS and Chrome on mobile.
*
* Learn more in https://angular.io/guide/browser-support
*/
/***************************************************************************************************
* BROWSER POLYFILLS
*/
/**
* By default, zone.js will patch all possible macroTask and DomEvents
* user can disable parts of macroTask/DomEvents patch by setting following flags
* because those flags need to be set before `zone.js` being loaded, and webpack
* will put import in the top of bundle, so user need to create a separate file
* in this directory (for example: zone-flags.ts), and put the following flags
* into that file, and then add the following code before importing zone.js.
* import './zone-flags';
*
* The flags allowed in zone-flags.ts are listed here.
*
* The following flags will work for all browsers.
*
* (window as any).__Zone_disable_requestAnimationFrame = true; // disable patch requestAnimationFrame
* (window as any).__Zone_disable_on_property = true; // disable patch onProperty such as onclick
* (window as any).__zone_symbol__UNPATCHED_EVENTS = ['scroll', 'mousemove']; // disable patch specified eventNames
*
* in IE/Edge developer tools, the addEventListener will also be wrapped by zone.js
* with the following flag, it will bypass `zone.js` patch for IE/Edge
*
* (window as any).__Zone_enable_cross_context_check = true;
*
*/
/***************************************************************************************************
* Zone JS is required by default for Angular itself.
*/
import 'zone.js'; // Included with Angular CLI.
/***************************************************************************************************
* APPLICATION IMPORTS
*/
+18
View File
@@ -0,0 +1,18 @@
// Copyright 2022 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
html, body { height: 100%; }
body { margin: 0; font-family: Roboto, "Helvetica Neue", sans-serif; }
+26
View File
@@ -0,0 +1,26 @@
// This file is required by karma.conf.js and loads recursively all the .spec and framework files
import 'zone.js/testing';
import { getTestBed } from '@angular/core/testing';
import {
BrowserDynamicTestingModule,
platformBrowserDynamicTesting
} from '@angular/platform-browser-dynamic/testing';
declare const require: {
context(path: string, deep?: boolean, filter?: RegExp): {
<T>(id: string): T;
keys(): string[];
};
};
// First, initialize the Angular testing environment.
getTestBed().initTestEnvironment(
BrowserDynamicTestingModule,
platformBrowserDynamicTesting(),
);
// Then we find all the tests.
const context = require.context('./', true, /\.spec\.ts$/);
// And load the modules.
context.keys().forEach(context);
+15
View File
@@ -0,0 +1,15 @@
/* To learn more about this file see: https://angular.io/config/tsconfig. */
{
"extends": "./tsconfig.json",
"compilerOptions": {
"outDir": "./out-tsc/app",
"types": []
},
"files": [
"src/main.ts",
"src/polyfills.ts"
],
"include": [
"src/**/*.d.ts"
]
}
+32
View File
@@ -0,0 +1,32 @@
/* To learn more about this file see: https://angular.io/config/tsconfig. */
{
"compileOnSave": false,
"compilerOptions": {
"baseUrl": "./",
"outDir": "./dist/out-tsc",
"forceConsistentCasingInFileNames": true,
"strict": true,
"noImplicitOverride": true,
"noPropertyAccessFromIndexSignature": true,
"noImplicitReturns": true,
"noFallthroughCasesInSwitch": true,
"sourceMap": true,
"declaration": false,
"downlevelIteration": true,
"experimentalDecorators": true,
"moduleResolution": "node",
"importHelpers": true,
"target": "es2020",
"module": "es2020",
"lib": [
"es2020",
"dom"
]
},
"angularCompilerOptions": {
"enableI18nLegacyMessageIdFormat": false,
"strictInjectionParameters": true,
"strictInputAccessModifiers": true,
"strictTemplates": true
}
}
+18
View File
@@ -0,0 +1,18 @@
/* To learn more about this file see: https://angular.io/config/tsconfig. */
{
"extends": "./tsconfig.json",
"compilerOptions": {
"outDir": "./out-tsc/spec",
"types": [
"jasmine"
]
},
"files": [
"src/test.ts",
"src/polyfills.ts"
],
"include": [
"src/**/*.spec.ts",
"src/**/*.d.ts"
]
}
+2
View File
@@ -270,6 +270,8 @@ dependencies {
implementation deps['org.jsoup:jsoup']
testImplementation deps['org.mortbay.jetty:jetty']
implementation deps['org.postgresql:postgresql']
implementation "org.eclipse.jetty:jetty-server:9.4.49.v20220914"
implementation "org.eclipse.jetty:jetty-servlet:9.4.49.v20220914"
testImplementation deps['org.seleniumhq.selenium:selenium-api']
testImplementation deps['org.seleniumhq.selenium:selenium-chrome-driver']
testImplementation deps['org.seleniumhq.selenium:selenium-java']
+7
View File
@@ -367,6 +367,13 @@ org.codehaus.mojo:animal-sniffer-annotations:1.22=default,deploy_jar,nonprodRunt
org.conscrypt:conscrypt-openjdk-uber:2.5.2=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.easymock:easymock:3.0=css
org.eclipse.angus:angus-activation:1.0.0=nonprodRuntime,runtime
org.eclipse.jetty:jetty-http:9.4.49.v20220914=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.eclipse.jetty:jetty-io:9.4.49.v20220914=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.eclipse.jetty:jetty-security:9.4.49.v20220914=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.eclipse.jetty:jetty-server:9.4.49.v20220914=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.eclipse.jetty:jetty-servlet:9.4.49.v20220914=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.eclipse.jetty:jetty-util-ajax:9.4.49.v20220914=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.eclipse.jetty:jetty-util:9.4.49.v20220914=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.flywaydb:flyway-core:9.10.0=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.glassfish.jaxb:jaxb-core:4.0.1=nonprodRuntime,runtime
org.glassfish.jaxb:jaxb-runtime:2.3.1=compileClasspath,default,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
@@ -14,10 +14,11 @@
package google.registry.batch;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.Iterables;
import com.google.common.flogger.FluentLogger;
import com.google.common.primitives.Ints;
import google.registry.config.RegistryConfig.Config;
@@ -29,8 +30,10 @@ import google.registry.request.Action;
import google.registry.request.Action.Service;
import google.registry.request.auth.Auth;
import google.registry.ui.server.SendEmailUtils;
import google.registry.util.Clock;
import java.util.Optional;
import javax.inject.Inject;
import org.joda.time.Days;
/**
* An action that checks all {@link PackagePromotion} objects for compliance with their max create
@@ -45,78 +48,164 @@ public class CheckPackagesComplianceAction implements Runnable {
public static final String PATH = "/_dr/task/checkPackagesCompliance";
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private final SendEmailUtils sendEmailUtils;
private final String packageCreateLimitEmailSubjectText;
private final String packageCreateLimitEmailBodyText;
private final Clock clock;
private final String packageCreateLimitEmailSubject;
private final String packageDomainLimitWarningEmailSubject;
private final String packageDomainLimitUpgradeEmailSubject;
private final String packageCreateLimitEmailBody;
private final String packageDomainLimitWarningEmailBody;
private final String packageDomainLimitUpgradeEmailBody;
private final String registrySupportEmail;
private static final int THIRTY_DAYS = 30;
private static final int FORTY_DAYS = 40;
@Inject
public CheckPackagesComplianceAction(
SendEmailUtils sendEmailUtils,
@Config("packageCreateLimitEmailSubjectText") String packageCreateLimitEmailSubjectText,
@Config("packageCreateLimitEmailBodyText") String packageCreateLimitEmailBodyText,
Clock clock,
@Config("packageCreateLimitEmailSubject") String packageCreateLimitEmailSubject,
@Config("packageDomainLimitWarningEmailSubject") String packageDomainLimitWarningEmailSubject,
@Config("packageDomainLimitUpgradeEmailSubject") String packageDomainLimitUpgradeEmailSubject,
@Config("packageCreateLimitEmailBody") String packageCreateLimitEmailBody,
@Config("packageDomainLimitWarningEmailBody") String packageDomainLimitWarningEmailBody,
@Config("packageDomainLimitUpgradeEmailBody") String packageDomainLimitUpgradeEmailBody,
@Config("registrySupportEmail") String registrySupportEmail) {
this.sendEmailUtils = sendEmailUtils;
this.packageCreateLimitEmailSubjectText = packageCreateLimitEmailSubjectText;
this.packageCreateLimitEmailBodyText = packageCreateLimitEmailBodyText;
this.clock = clock;
this.packageCreateLimitEmailSubject = packageCreateLimitEmailSubject;
this.packageDomainLimitWarningEmailSubject = packageDomainLimitWarningEmailSubject;
this.packageDomainLimitUpgradeEmailSubject = packageDomainLimitUpgradeEmailSubject;
this.packageCreateLimitEmailBody = packageCreateLimitEmailBody;
this.packageDomainLimitWarningEmailBody = packageDomainLimitWarningEmailBody;
this.packageDomainLimitUpgradeEmailBody = packageDomainLimitUpgradeEmailBody;
this.registrySupportEmail = registrySupportEmail;
}
@Override
public void run() {
tm().transact(
() -> {
ImmutableList<PackagePromotion> packages = tm().loadAllOf(PackagePromotion.class);
ImmutableList.Builder<PackagePromotion> packagesOverCreateLimit =
new ImmutableList.Builder<>();
for (PackagePromotion packagePromo : packages) {
Long creates =
(Long)
jpaTm()
.query(
"SELECT COUNT(*) FROM DomainHistory WHERE current_package_token ="
+ " :token AND modificationTime >= :lastBilling AND type ="
+ " 'DOMAIN_CREATE'")
.setParameter("token", packagePromo.getToken().getKey().toString())
.setParameter(
"lastBilling", packagePromo.getNextBillingDate().minusYears(1))
.getSingleResult();
if (creates > packagePromo.getMaxCreates()) {
int overage = Ints.saturatedCast(creates) - packagePromo.getMaxCreates();
logger.atInfo().log(
"Package with package token %s has exceeded their max domain creation limit"
+ " by %d name(s).",
packagePromo.getToken().getKey(), overage);
packagesOverCreateLimit.add(packagePromo);
}
}
if (packagesOverCreateLimit.build().isEmpty()) {
logger.atInfo().log("Found no packages over their create limit.");
} else {
logger.atInfo().log(
"Found %d packages over their create limit.",
packagesOverCreateLimit.build().size());
for (PackagePromotion packagePromotion : packagesOverCreateLimit.build()) {
AllocationToken packageToken = tm().loadByKey(packagePromotion.getToken());
Optional<Registrar> registrar =
Registrar.loadByRegistrarIdCached(
packageToken.getAllowedRegistrarIds().iterator().next());
if (registrar.isPresent()) {
String body =
String.format(
packageCreateLimitEmailBodyText,
registrar.get().getRegistrarName(),
packageToken.getToken(),
registrySupportEmail);
sendNotification(
packageToken, packageCreateLimitEmailSubjectText, body, registrar.get());
} else {
logger.atSevere().log(
String.format(
"Could not find registrar for package token %s", packageToken));
}
}
}
});
tm().transact(this::checkPackages);
}
private void checkPackages() {
ImmutableList<PackagePromotion> packages = tm().loadAllOf(PackagePromotion.class);
ImmutableList.Builder<PackagePromotion> packagesOverCreateLimitBuilder =
new ImmutableList.Builder<>();
ImmutableList.Builder<PackagePromotion> packagesOverActiveDomainsLimitBuilder =
new ImmutableList.Builder<>();
for (PackagePromotion packagePromo : packages) {
Long creates =
(Long)
tm().query(
"SELECT COUNT(*) FROM DomainHistory WHERE current_package_token ="
+ " :token AND modificationTime >= :lastBilling AND type ="
+ " 'DOMAIN_CREATE'")
.setParameter("token", packagePromo.getToken().getKey().toString())
.setParameter("lastBilling", packagePromo.getNextBillingDate().minusYears(1))
.getSingleResult();
if (creates > packagePromo.getMaxCreates()) {
int overage = Ints.saturatedCast(creates) - packagePromo.getMaxCreates();
logger.atInfo().log(
"Package with package token %s has exceeded their max domain creation limit"
+ " by %d name(s).",
packagePromo.getToken().getKey(), overage);
packagesOverCreateLimitBuilder.add(packagePromo);
}
Long activeDomains =
tm().query(
"SELECT COUNT(*) FROM Domain WHERE currentPackageToken = :token"
+ " AND deletionTime = :endOfTime",
Long.class)
.setParameter("token", packagePromo.getToken())
.setParameter("endOfTime", END_OF_TIME)
.getSingleResult();
if (activeDomains > packagePromo.getMaxDomains()) {
int overage = Ints.saturatedCast(activeDomains) - packagePromo.getMaxDomains();
logger.atInfo().log(
"Package with package token %s has exceed their max active domains limit by"
+ " %d name(s).",
packagePromo.getToken().getKey(), overage);
packagesOverActiveDomainsLimitBuilder.add(packagePromo);
}
}
handlePackageCreationOverage(packagesOverCreateLimitBuilder.build());
handleActiveDomainOverage(packagesOverActiveDomainsLimitBuilder.build());
}
private void handlePackageCreationOverage(ImmutableList<PackagePromotion> overageList) {
if (overageList.isEmpty()) {
logger.atInfo().log("Found no packages over their create limit.");
return;
}
logger.atInfo().log("Found %d packages over their create limit.", overageList.size());
for (PackagePromotion packagePromotion : overageList) {
AllocationToken packageToken = tm().loadByKey(packagePromotion.getToken());
Optional<Registrar> registrar =
Registrar.loadByRegistrarIdCached(
Iterables.getOnlyElement(packageToken.getAllowedRegistrarIds()));
if (registrar.isPresent()) {
String body =
String.format(
packageCreateLimitEmailBody,
registrar.get().getRegistrarName(),
packageToken.getToken(),
registrySupportEmail);
sendNotification(packageToken, packageCreateLimitEmailSubject, body, registrar.get());
} else {
throw new IllegalStateException(
String.format("Could not find registrar for package token %s", packageToken));
}
}
}
private void handleActiveDomainOverage(ImmutableList<PackagePromotion> overageList) {
if (overageList.isEmpty()) {
logger.atInfo().log("Found no packages over their active domains limit.");
return;
}
logger.atInfo().log("Found %d packages over their active domains limit.", overageList.size());
for (PackagePromotion packagePromotion : overageList) {
int daysSinceLastNotification =
packagePromotion
.getLastNotificationSent()
.map(sentDate -> Days.daysBetween(sentDate, clock.nowUtc()).getDays())
.orElse(Integer.MAX_VALUE);
if (daysSinceLastNotification < THIRTY_DAYS) {
// Don't send an email if notification was already sent within the last 30
// days
continue;
} else if (daysSinceLastNotification < FORTY_DAYS) {
// Send an upgrade email if last email was between 30 and 40 days ago
sendActiveDomainOverageEmail(/* warning= */ false, packagePromotion);
} else {
// Send a warning email
sendActiveDomainOverageEmail(/* warning= */ true, packagePromotion);
}
}
}
private void sendActiveDomainOverageEmail(boolean warning, PackagePromotion packagePromotion) {
String emailSubject =
warning ? packageDomainLimitWarningEmailSubject : packageDomainLimitUpgradeEmailSubject;
String emailTemplate =
warning ? packageDomainLimitWarningEmailBody : packageDomainLimitUpgradeEmailBody;
AllocationToken packageToken = tm().loadByKey(packagePromotion.getToken());
Optional<Registrar> registrar =
Registrar.loadByRegistrarIdCached(
Iterables.getOnlyElement(packageToken.getAllowedRegistrarIds()));
if (registrar.isPresent()) {
String body =
String.format(
emailTemplate,
registrar.get().getRegistrarName(),
packageToken.getToken(),
registrySupportEmail);
sendNotification(packageToken, emailSubject, body, registrar.get());
tm().put(packagePromotion.asBuilder().setLastNotificationSent(clock.nowUtc()).build());
} else {
throw new IllegalStateException(
String.format("Could not find registrar for package token %s", packageToken));
}
}
private void sendNotification(
@@ -21,7 +21,6 @@ import static google.registry.batch.BatchModule.PARAM_DRY_RUN;
import static google.registry.config.RegistryEnvironment.PRODUCTION;
import static google.registry.model.reporting.HistoryEntry.Type.DOMAIN_DELETE;
import static google.registry.model.tld.Registries.getTldsOfType;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.request.Action.Method.POST;
import static google.registry.request.RequestParameters.PARAM_TLDS;
@@ -140,7 +139,7 @@ public class DeleteProberDataAction implements Runnable {
private void runSqlJob(ImmutableSet<String> deletableTlds) {
AtomicInteger softDeletedDomains = new AtomicInteger();
AtomicInteger hardDeletedDomains = new AtomicInteger();
jpaTm().transact(() -> processDomains(deletableTlds, softDeletedDomains, hardDeletedDomains));
tm().transact(() -> processDomains(deletableTlds, softDeletedDomains, hardDeletedDomains));
logger.atInfo().log(
"%s %d domains.",
isDryRun ? "Would have soft-deleted" : "Soft-deleted", softDeletedDomains.get());
@@ -157,8 +156,7 @@ public class DeleteProberDataAction implements Runnable {
// Scroll through domains, soft-deleting as necessary (very few will be soft-deleted) and
// keeping track of which domains to hard-delete (there can be many, so we batch them up)
try (ScrollableResults scrollableResult =
jpaTm()
.query(DOMAIN_QUERY_STRING, Domain.class)
tm().query(DOMAIN_QUERY_STRING, Domain.class)
.setParameter("tlds", deletableTlds)
.setParameter(
"creationTimeCutoff", CreateAutoTimestamp.create(now.minus(DOMAIN_USED_DURATION)))
@@ -183,8 +181,8 @@ public class DeleteProberDataAction implements Runnable {
domainRepoIdsToHardDelete.build(), hostNamesToHardDelete.build());
domainRepoIdsToHardDelete = new ImmutableList.Builder<>();
hostNamesToHardDelete = new ImmutableList.Builder<>();
jpaTm().getEntityManager().flush();
jpaTm().getEntityManager().clear();
tm().getEntityManager().flush();
tm().getEntityManager().clear();
}
}
// process the remainder
@@ -226,32 +224,25 @@ public class DeleteProberDataAction implements Runnable {
private void hardDeleteDomainsAndHosts(
ImmutableList<String> domainRepoIds, ImmutableList<String> hostNames) {
jpaTm()
.query("DELETE FROM Host WHERE hostName IN :hostNames")
tm().query("DELETE FROM Host WHERE hostName IN :hostNames")
.setParameter("hostNames", hostNames)
.executeUpdate();
jpaTm()
.query("DELETE FROM BillingEvent WHERE domainRepoId IN :repoIds")
tm().query("DELETE FROM BillingEvent WHERE domainRepoId IN :repoIds")
.setParameter("repoIds", domainRepoIds)
.executeUpdate();
jpaTm()
.query("DELETE FROM BillingRecurrence WHERE domainRepoId IN :repoIds")
tm().query("DELETE FROM BillingRecurrence WHERE domainRepoId IN :repoIds")
.setParameter("repoIds", domainRepoIds)
.executeUpdate();
jpaTm()
.query("DELETE FROM BillingCancellation WHERE domainRepoId IN :repoIds")
tm().query("DELETE FROM BillingCancellation WHERE domainRepoId IN :repoIds")
.setParameter("repoIds", domainRepoIds)
.executeUpdate();
jpaTm()
.query("DELETE FROM DomainHistory WHERE repoId IN :repoIds")
tm().query("DELETE FROM DomainHistory WHERE repoId IN :repoIds")
.setParameter("repoIds", domainRepoIds)
.executeUpdate();
jpaTm()
.query("DELETE FROM PollMessage WHERE domainRepoId IN :repoIds")
tm().query("DELETE FROM PollMessage WHERE domainRepoId IN :repoIds")
.setParameter("repoIds", domainRepoIds)
.executeUpdate();
jpaTm()
.query("DELETE FROM Domain WHERE repoId IN :repoIds")
tm().query("DELETE FROM Domain WHERE repoId IN :repoIds")
.setParameter("repoIds", domainRepoIds)
.executeUpdate();
}
@@ -23,7 +23,6 @@ import static google.registry.model.common.Cursor.CursorType.RECURRING_BILLING;
import static google.registry.model.domain.Period.Unit.YEARS;
import static google.registry.model.reporting.HistoryEntry.Type.DOMAIN_AUTORENEW;
import static google.registry.persistence.transaction.QueryComposer.Comparator.EQ;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.CollectionUtils.union;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
@@ -118,15 +117,13 @@ public class ExpandRecurringBillingEventsAction implements Runnable {
do {
final long prevMaxProcessedRecurrenceId = maxProcessedRecurrenceId;
sqlBatchResults =
jpaTm()
.transact(
tm().transact(
() -> {
Set<String> expandedDomains = newHashSet();
int batchBillingEventsSaved = 0;
long maxRecurrenceId = prevMaxProcessedRecurrenceId;
List<Recurring> recurrings =
jpaTm()
.query(
tm().query(
"FROM BillingRecurrence "
+ "WHERE eventTime <= :executeTime "
+ "AND eventTime < recurrenceEndTime "
@@ -16,7 +16,6 @@ package google.registry.batch;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.request.Action.Method.POST;
import static google.registry.tools.LockOrUnlockDomainCommand.REGISTRY_LOCK_STATUSES;
@@ -189,7 +188,7 @@ public class RelockDomainAction implements Runnable {
"Domain %s has a pending delete.",
domainName);
checkArgument(
!DateTimeUtils.isAtOrAfter(jpaTm().getTransactionTime(), domain.getDeletionTime()),
!DateTimeUtils.isAtOrAfter(tm().getTransactionTime(), domain.getDeletionTime()),
"Domain %s has been deleted.",
domainName);
checkArgument(
@@ -14,7 +14,7 @@
package google.registry.batch;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static org.apache.http.HttpStatus.SC_INTERNAL_SERVER_ERROR;
import static org.apache.http.HttpStatus.SC_OK;
@@ -77,8 +77,7 @@ public class WipeOutContactHistoryPiiAction implements Runnable {
int numOfWipedEntities = 0;
do {
numOfWipedEntities =
jpaTm()
.transact(
tm().transact(
() ->
wipeOutContactHistoryData(
getNextContactHistoryEntitiesWithPiiBatch(wipeOutTime)));
@@ -113,8 +112,7 @@ public class WipeOutContactHistoryPiiAction implements Runnable {
// email is one of the required fields in EPP, meaning it's initially not null.
// Therefore, checking if it's null is one way to avoid processing contact history entities
// that have been processed previously. Refer to RFC 5733 for more information.
return jpaTm()
.query(
return tm().query(
"FROM ContactHistory WHERE modificationTime < :wipeOutTime " + "AND email IS NOT NULL",
ContactHistory.class)
.setParameter("wipeOutTime", wipeOutTime)
@@ -128,7 +126,7 @@ public class WipeOutContactHistoryPiiAction implements Runnable {
AtomicInteger numOfEntities = new AtomicInteger(0);
contactHistoryEntities.forEach(
contactHistoryEntity -> {
jpaTm().update(contactHistoryEntity.asBuilder().wipeOutPii().build());
tm().update(contactHistoryEntity.asBuilder().wipeOutPii().build());
numOfEntities.incrementAndGet();
});
logger.atInfo().log(
@@ -18,9 +18,7 @@ import static com.google.common.base.Preconditions.checkArgument;
import com.google.common.base.Joiner;
import com.google.common.collect.ImmutableList;
import com.google.common.io.Resources;
import google.registry.util.Clock;
import google.registry.util.ResourceUtils;
import java.util.regex.Pattern;
import org.apache.avro.generic.GenericRecord;
import org.apache.beam.sdk.io.gcp.bigquery.SchemaAndRecord;
@@ -57,14 +55,6 @@ public class BeamUtils {
}
}
/**
* Returns the {@link String} contents for a file in the {@code sql/} directory relative to a
* class.
*/
public static String getQueryFromFile(Class<?> clazz, String filename) {
return ResourceUtils.readResourceUtf8(Resources.getResource(clazz, "sql/" + filename));
}
/** Creates a beam job name and validates that it conforms to the requirements. */
public static String createJobName(String prefix, Clock clock) {
// Flex template job name must be unique and consists of only characters [-a-z0-9], starting
@@ -15,7 +15,7 @@
package google.registry.beam.common;
import static com.google.common.base.Preconditions.checkState;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.flogger.FluentLogger;
import java.util.List;
@@ -52,7 +52,7 @@ public class DatabaseSnapshot implements AutoCloseable {
}
private DatabaseSnapshot open() {
entityManager = jpaTm().getStandaloneEntityManager();
entityManager = tm().getStandaloneEntityManager();
transaction = entityManager.getTransaction();
transaction.setRollbackOnly();
transaction.begin();
@@ -14,8 +14,7 @@
package google.registry.beam.common;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static org.apache.beam.sdk.values.TypeDescriptors.integers;
import com.google.auto.value.AutoValue;
@@ -26,7 +25,6 @@ import google.registry.persistence.transaction.JpaTransactionManager;
import google.registry.persistence.transaction.TransactionManagerFactory;
import java.util.Map;
import java.util.Objects;
import java.util.concurrent.ThreadLocalRandom;
import javax.annotation.Nullable;
import javax.persistence.criteria.CriteriaQuery;
import org.apache.beam.sdk.coders.Coder;
@@ -51,8 +49,8 @@ import org.apache.beam.sdk.values.PCollection;
*
* <p>The {@code JpaTransactionManager} is instantiated once on each pipeline worker VM (through
* {@link RegistryPipelineWorkerInitializer}), made available through the static method {@link
* TransactionManagerFactory#jpaTm()}, and is shared by all threads on the VM. Configuration is
* through {@link RegistryPipelineOptions}.
* TransactionManagerFactory#tm()}, and is shared by all threads on the VM. Configuration is through
* {@link RegistryPipelineOptions}.
*/
public final class RegistryJpaIO {
@@ -232,11 +230,10 @@ public final class RegistryJpaIO {
@ProcessElement
public void processElement(OutputReceiver<T> outputReceiver) {
jpaTm()
.transactNoRetry(
tm().transactNoRetry(
() -> {
if (snapshotId != null) {
jpaTm().setDatabaseSnapshot(snapshotId);
tm().setDatabaseSnapshot(snapshotId);
}
query.stream().map(resultMapper::apply).forEach(outputReceiver::output);
});
@@ -263,38 +260,11 @@ public final class RegistryJpaIO {
public static final int DEFAULT_BATCH_SIZE = 1;
/** The default number of write shard. Please refer to {@link #shards} for more information. */
public static final int DEFAULT_SHARDS = 1;
public abstract String name();
/** Number of elements to be written in one call. */
public abstract int batchSize();
/**
* The number of shards the output should be split into.
*
* <p>This value is a hint to the pipeline runner on the level of parallelism, and should be
* significantly greater than the number of threads working on this transformation (see next
* paragraph for more information). On the other hand, it should not be too large to the point
* that the number of elements per shard is lower than {@link #batchSize()}. As a rule of thumb,
* the following constraint should hold: {@code shards * batchSize * nThreads <=
* inputElementCount}. Although it is not always possible to determine the number of threads
* working on this transform, when the pipeline run is IO-bound, it most likely is close to the
* total number of threads in the pipeline, which is explained below.
*
* <p>With Cloud Dataflow runner, the total number of worker threads in a batch pipeline (which
* includes all existing Registry pipelines) is the number of vCPUs used by the pipeline, and
* can be set by the {@code --maxNumWorkers} and {@code --workerMachineType} parameters. The
* number of worker threads in a streaming pipeline can be set by the {@code --maxNumWorkers}
* and {@code --numberOfWorkerHarnessThreads} parameters.
*
* <p>Note that connections on the database server are a limited resource, therefore the number
* of threads that interact with the database should be set to an appropriate limit. Again, we
* cannot control this number, but can influence it by controlling the total number of threads.
*/
public abstract int shards();
public abstract SerializableFunction<T, Object> jpaConverter();
public Write<T> withName(String name) {
@@ -305,10 +275,6 @@ public final class RegistryJpaIO {
return toBuilder().batchSize(batchSize).build();
}
public Write<T> withShards(int shards) {
return toBuilder().shards(shards).build();
}
/**
* An optional function that converts the input entities to a form that can be written into the
* database.
@@ -322,10 +288,7 @@ public final class RegistryJpaIO {
@Override
public PCollection<Void> expand(PCollection<T> input) {
return input
.apply(
"Shard data " + name(),
WithKeys.<Integer, T>of(e -> ThreadLocalRandom.current().nextInt(shards()))
.withKeyType(integers()))
.apply("Add key to data " + name(), WithKeys.<Integer, T>of(0).withKeyType(integers()))
// The call to withShardedKey() is performance critical. The resulting transform ensures
// that data is spread evenly across all worker threads.
.apply(
@@ -340,7 +303,6 @@ public final class RegistryJpaIO {
return new AutoValue_RegistryJpaIO_Write.Builder<T>()
.name(DEFAULT_NAME)
.batchSize(DEFAULT_BATCH_SIZE)
.shards(DEFAULT_SHARDS)
.jpaConverter(x -> x);
}
@@ -351,8 +313,6 @@ public final class RegistryJpaIO {
abstract Builder<T> batchSize(int batchSize);
abstract Builder<T> shards(int jdbcNumConnsHint);
abstract Builder<T> jpaConverter(SerializableFunction<T, Object> jpaConverter);
abstract Write<T> build();
@@ -382,12 +342,10 @@ public final class RegistryJpaIO {
.filter(Objects::nonNull)
.collect(ImmutableList.toImmutableList());
try {
jpaTm()
.transact(
tm().transact(
() -> {
// Don't modify existing objects as it could lead to race conditions
entities.forEach(this::verifyObjectNonexistence);
jpaTm().putAll(entities);
// TODO(b/263502442): properly handle creations and blind-writes.
tm().putAll(entities);
});
counter.inc(entities.size());
} catch (RuntimeException e) {
@@ -402,12 +360,10 @@ public final class RegistryJpaIO {
private void processSingly(ImmutableList<Object> entities) {
for (Object entity : entities) {
try {
jpaTm()
.transact(
tm().transact(
() -> {
// Don't modify existing objects as it could lead to race conditions
verifyObjectNonexistence(entity);
jpaTm().put(entity);
// TODO(b/263502442): properly handle creations and blind-writes.
tm().put(entity);
});
counter.inc();
} catch (RuntimeException e) {
@@ -419,14 +375,12 @@ public final class RegistryJpaIO {
/** Returns this entity's primary key field(s) in a string. */
private String toEntityKeyString(Object entity) {
try {
return jpaTm()
.transact(
return tm().transact(
() ->
String.format(
"%s_%s",
entity.getClass().getSimpleName(),
jpaTm()
.getEntityManager()
tm().getEntityManager()
.getEntityManagerFactory()
.getPersistenceUnitUtil()
.getIdentifier(entity)));
@@ -434,16 +388,5 @@ public final class RegistryJpaIO {
return "Non-SqlEntity: " + entity;
}
}
/** SqlBatchWriter should not re-write existing entities due to potential race conditions. */
private void verifyObjectNonexistence(Object obj) {
// We cannot rely on calling "insert" on the objects because the underlying JPA persist call
// adds the input object to the persistence context, meaning that any modifications (e.g.
// updateTimestamp) are reflected in the input object. Beam doesn't allow modification of
// input objects, so this throws an exception.
// TODO(go/non-datastore-allocateid): also check that all the objects have IDs
checkArgument(
!jpaTm().exists(obj), "Entities created in SqlBatchWriter must not already exist");
}
}
}
@@ -14,7 +14,6 @@
package google.registry.beam.common;
import google.registry.beam.common.RegistryJpaIO.Write;
import google.registry.config.RegistryEnvironment;
import google.registry.model.annotations.DeleteAfterMigration;
import google.registry.persistence.PersistenceModule.JpaTransactionManagerType;
@@ -29,9 +28,9 @@ import org.apache.beam.sdk.options.Description;
* Defines Nomulus-specific pipeline options, e.g. JPA configurations.
*
* <p>When using the Cloud Dataflow runner, users are recommended to set an upper bound on active
* database connections by setting the pipeline worker options including {@code --maxNumWorkers},
* {@code workerMachineType}, and {@code numberOfWorkerHarnessThreads}. Please refer to {@link
* Write#shards()} for more information.
* database connections by setting the max number of pipeline worker threads using {@code
* --maxNumWorkers} and {@code workerMachineType} for batch pipelines, or {@code --maxNumWorkers}
* and {@code --numberOfWorkerHarnessThreads} for streaming pipelines.
*/
public interface RegistryPipelineOptions extends GcpOptions {
@@ -58,14 +57,6 @@ public interface RegistryPipelineOptions extends GcpOptions {
void setSqlWriteBatchSize(int sqlWriteBatchSize);
@Description(
"Number of shards to create out of the data before writing to the SQL database. Please refer "
+ "to the Javadoc of RegistryJpaIO.Write.shards() for how to choose this value.")
@Default.Integer(100)
int getSqlWriteShards();
void setSqlWriteShards(int maxConcurrentSqlWriters);
@DeleteAfterMigration
@Description(
"Whether to use self allocated primary IDs when building entities. This should only be used"
@@ -14,7 +14,7 @@
package google.registry.beam.common;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import google.registry.persistence.transaction.JpaTransactionManager;
import java.io.Serializable;
@@ -53,7 +53,7 @@ public interface RegistryQuery<T> extends Serializable {
static <T> RegistryQuery<T> createQuery(
String sql, @Nullable Map<String, Object> parameters, boolean nativeQuery) {
return () -> {
EntityManager entityManager = jpaTm().getEntityManager();
EntityManager entityManager = tm().getEntityManager();
Query query =
nativeQuery ? entityManager.createNativeQuery(sql) : entityManager.createQuery(sql);
if (parameters != null) {
@@ -76,7 +76,7 @@ public interface RegistryQuery<T> extends Serializable {
String jpql, @Nullable Map<String, Object> parameters, Class<T> clazz) {
return () -> {
// TODO(b/193662898): switch to jpaTm().query() when it can properly detach loaded entities.
EntityManager entityManager = jpaTm().getEntityManager();
EntityManager entityManager = tm().getEntityManager();
TypedQuery<T> query = entityManager.createQuery(jpql, clazz);
if (parameters != null) {
parameters.forEach(query::setParameter);
@@ -98,7 +98,7 @@ public interface RegistryQuery<T> extends Serializable {
static <T> RegistryQuery<T> createQuery(CriteriaQuerySupplier<T> criteriaQuery) {
return () -> {
// TODO(b/193662898): switch to jpaTm().query() when it can properly detach loaded entities.
EntityManager entityManager = jpaTm().getEntityManager();
EntityManager entityManager = tm().getEntityManager();
TypedQuery<T> query = entityManager.createQuery(criteriaQuery.get());
JpaTransactionManager.setQueryFetchSize(query, QUERY_FETCH_SIZE);
return query.getResultStream().map(e -> detach(entityManager, e));
@@ -15,7 +15,6 @@
package google.registry.beam.invoicing;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.beam.BeamUtils.getQueryFromFile;
import static org.apache.beam.sdk.values.TypeDescriptors.strings;
import com.google.common.flogger.FluentLogger;
@@ -29,6 +28,7 @@ import google.registry.model.registrar.Registrar;
import google.registry.persistence.PersistenceModule.TransactionIsolationLevel;
import google.registry.reporting.billing.BillingModule;
import google.registry.util.DomainNameUtils;
import google.registry.util.ResourceUtils;
import google.registry.util.SqlTemplate;
import java.io.Serializable;
import java.time.YearMonth;
@@ -209,7 +209,8 @@ public class InvoicingPipeline implements Serializable {
YearMonth endMonth = YearMonth.parse(yearMonth).plusMonths(1);
String queryWithComments =
SqlTemplate.create(
getQueryFromFile(InvoicingPipeline.class, "cloud_sql_billing_events.sql"))
ResourceUtils.readResourceUtf8(
InvoicingPipeline.class, "sql/cloud_sql_billing_events.sql"))
.put("FIRST_TIMESTAMP_OF_MONTH", yearMonth + "-01")
.put(
"LAST_TIMESTAMP_OF_MONTH",
@@ -26,7 +26,7 @@ import static google.registry.beam.rde.RdePipeline.TupleTags.REFERENCED_HOSTS;
import static google.registry.beam.rde.RdePipeline.TupleTags.REVISION_ID;
import static google.registry.beam.rde.RdePipeline.TupleTags.SUPERORDINATE_DOMAINS;
import static google.registry.model.reporting.HistoryEntryDao.RESOURCE_TYPES_TO_HISTORY_TYPES;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static org.apache.beam.sdk.values.TypeDescriptors.kvs;
import com.google.common.collect.ImmutableList;
@@ -305,7 +305,7 @@ public class RdePipeline implements Serializable {
(String registrarRepoId) -> {
VKey<Registrar> key = VKey.create(Registrar.class, registrarRepoId);
includedRegistrarCounter.inc();
Registrar registrar = jpaTm().transact(() -> jpaTm().loadByKey(key));
Registrar registrar = tm().transact(() -> tm().loadByKey(key));
DepositFragment fragment = marshaller.marshalRegistrar(registrar);
ImmutableSet<KV<PendingDeposit, DepositFragment>> fragments =
pendingDeposits.stream()
@@ -376,11 +376,9 @@ public class RdePipeline implements Serializable {
private <T extends HistoryEntry> EppResource loadResourceByHistoryEntryId(
Class<T> historyEntryClazz, String repoId, long revisionId) {
return jpaTm()
.transact(
return tm().transact(
() ->
jpaTm()
.loadByKey(
tm().loadByKey(
VKey.create(historyEntryClazz, new HistoryEntryId(repoId, revisionId))))
.getResourceAtPointInTime()
.map(resource -> resource.cloneProjectedAtTime(watermark))
@@ -15,7 +15,7 @@
package google.registry.beam.resave;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static org.apache.beam.sdk.values.TypeDescriptors.integers;
import com.google.common.collect.ImmutableList;
@@ -33,7 +33,6 @@ import google.registry.persistence.PersistenceModule.TransactionIsolationLevel;
import google.registry.persistence.VKey;
import google.registry.util.DateTimeUtils;
import java.io.Serializable;
import java.util.concurrent.ThreadLocalRandom;
import org.apache.beam.sdk.Pipeline;
import org.apache.beam.sdk.PipelineResult;
import org.apache.beam.sdk.coders.StringUtf8Coder;
@@ -143,15 +142,13 @@ public class ResaveAllEppResourcesPipeline implements Serializable {
/** Projects and re-saves all resources with repo IDs provided by the {@link Read}. */
private <T extends EppResource> void projectAndResaveResources(
Pipeline pipeline, Class<T> clazz, Read<?, String> repoIdRead) {
int numShards = options.getSqlWriteShards();
int batchSize = options.getSqlWriteBatchSize();
String className = clazz.getSimpleName();
pipeline
.apply("Read " + className, repoIdRead)
.apply(
"Shard data for class" + className,
WithKeys.<Integer, String>of(e -> ThreadLocalRandom.current().nextInt(numShards))
.withKeyType(integers()))
WithKeys.<Integer, String>of(0).withKeyType(integers()))
.apply(
"Group into batches for class" + className,
GroupIntoBatches.<Integer, String>ofSize(batchSize).withShardedKey())
@@ -174,19 +171,18 @@ public class ResaveAllEppResourcesPipeline implements Serializable {
public void processElement(
@Element KV<ShardedKey<Integer>, Iterable<String>> element,
OutputReceiver<Void> outputReceiver) {
jpaTm()
.transact(
tm().transact(
() -> {
DateTime now = jpaTm().getTransactionTime();
DateTime now = tm().getTransactionTime();
ImmutableList<VKey<? extends EppResource>> keys =
Streams.stream(element.getValue())
.map(repoId -> VKey.create(clazz, repoId))
.collect(toImmutableList());
ImmutableList<EppResource> mappedResources =
jpaTm().loadByKeys(keys).values().stream()
tm().loadByKeys(keys).values().stream()
.map(r -> r.cloneProjectedAtTime(now))
.collect(toImmutableList());
jpaTm().putAll(mappedResources);
tm().putAll(mappedResources);
});
}
}
@@ -15,7 +15,7 @@
package google.registry.beam.spec11;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.auto.value.AutoValue;
import com.google.common.collect.ImmutableSet;
@@ -45,6 +45,7 @@ import org.apache.beam.sdk.transforms.DoFn;
import org.apache.beam.sdk.transforms.GroupByKey;
import org.apache.beam.sdk.transforms.MapElements;
import org.apache.beam.sdk.transforms.ParDo;
import org.apache.beam.sdk.transforms.Reshuffle;
import org.apache.beam.sdk.values.KV;
import org.apache.beam.sdk.values.PCollection;
import org.apache.beam.sdk.values.TypeDescriptor;
@@ -131,9 +132,8 @@ public class Spec11Pipeline implements Serializable {
public void processElement(
@Element KV<String, String> input, OutputReceiver<DomainNameInfo> output) {
Domain domain =
jpaTm()
.transact(
() -> jpaTm().loadByKey(VKey.create(Domain.class, input.getKey())));
tm().transact(
() -> tm().loadByKey(VKey.create(Domain.class, input.getKey())));
String emailAddress = input.getValue();
if (emailAddress == null) {
emailAddress = "";
@@ -155,26 +155,36 @@ public class Spec11Pipeline implements Serializable {
static void saveToSql(
PCollection<KV<DomainNameInfo, ThreatMatch>> threatMatches, Spec11PipelineOptions options) {
String transformId = "Spec11 Threat Matches";
LocalDate date = LocalDate.parse(options.getDate(), ISODateTimeFormat.date());
threatMatches.apply(
"Write to Sql: " + transformId,
RegistryJpaIO.<KV<DomainNameInfo, ThreatMatch>>write()
.withName(transformId)
.withBatchSize(options.getSqlWriteBatchSize())
.withShards(options.getSqlWriteShards())
.withJpaConverter(
(kv) -> {
DomainNameInfo domainNameInfo = kv.getKey();
return new Spec11ThreatMatch.Builder()
.setThreatTypes(
ImmutableSet.of(ThreatType.valueOf(kv.getValue().threatType())))
.setCheckDate(date)
.setDomainName(domainNameInfo.domainName())
.setDomainRepoId(domainNameInfo.domainRepoId())
.setRegistrarId(domainNameInfo.registrarId())
.build();
}));
String transformId = "Spec11 Threat Matches";
threatMatches
.apply(
"Construct objects",
ParDo.of(
new DoFn<KV<DomainNameInfo, ThreatMatch>, Spec11ThreatMatch>() {
@ProcessElement
public void processElement(
@Element KV<DomainNameInfo, ThreatMatch> input,
OutputReceiver<Spec11ThreatMatch> output) {
Spec11ThreatMatch spec11ThreatMatch =
new Spec11ThreatMatch.Builder()
.setThreatTypes(
ImmutableSet.of(ThreatType.valueOf(input.getValue().threatType())))
.setCheckDate(date)
.setDomainName(input.getKey().domainName())
.setDomainRepoId(input.getKey().domainRepoId())
.setRegistrarId(input.getKey().registrarId())
// TODO(b/264416932) Assign id to prevent duplicate inserts.
.build();
output.output(spec11ThreatMatch);
}
}))
.apply("Prevent Fusing", Reshuffle.viaRandomKey())
.apply(
"Write to Sql: " + transformId,
RegistryJpaIO.<Spec11ThreatMatch>write()
.withName(transformId)
.withBatchSize(options.getSqlWriteBatchSize()));
}
static void saveToGcs(
@@ -106,6 +106,12 @@ public final class RegistryConfig {
return config.gcpProject.projectId;
}
@Provides
@Config("projectIdNumber")
public static long provideProjectIdNumber(RegistryConfigSettings config) {
return config.gcpProject.projectIdNumber;
}
@Provides
@Config("locationId")
public static String provideLocationId(RegistryConfigSettings config) {
@@ -1147,6 +1153,12 @@ public final class RegistryConfig {
return ImmutableSet.copyOf(config.oAuth.allowedOauthClientIds);
}
@Provides
@Config("iapClientId")
public static Optional<String> provideIapClientId(RegistryConfigSettings config) {
return Optional.ofNullable(config.oAuth.iapClientId);
}
/**
* Provides the OAuth scopes required for accessing Google APIs using the default credential.
*/
@@ -1311,15 +1323,41 @@ public final class RegistryConfig {
}
@Provides
@Config("packageCreateLimitEmailSubjectText")
public static String providePackageCreateLimitEmailSubjectText(RegistryConfigSettings config) {
return config.packageMonitoring.packageCreateLimitEmailSubjectText;
@Config("packageCreateLimitEmailSubject")
public static String providePackageCreateLimitEmailSubject(RegistryConfigSettings config) {
return config.packageMonitoring.packageCreateLimitEmailSubject;
}
@Provides
@Config("packageCreateLimitEmailBodyText")
public static String providePackageCreateLimitEmailBodyText(RegistryConfigSettings config) {
return config.packageMonitoring.packageCreateLimitEmailBodyText;
@Config("packageCreateLimitEmailBody")
public static String providePackageCreateLimitEmailBody(RegistryConfigSettings config) {
return config.packageMonitoring.packageCreateLimitEmailBody;
}
@Provides
@Config("packageDomainLimitWarningEmailSubject")
public static String providePackageDomainLimitWarningEmailSubject(
RegistryConfigSettings config) {
return config.packageMonitoring.packageDomainLimitWarningEmailSubject;
}
@Provides
@Config("packageDomainLimitWarningEmailBody")
public static String providePackageDomainLimitWarningEmailBody(RegistryConfigSettings config) {
return config.packageMonitoring.packageDomainLimitWarningEmailBody;
}
@Provides
@Config("packageDomainLimitUpgradeEmailSubject")
public static String providePackageDomainLimitUpgradeEmailSubject(
RegistryConfigSettings config) {
return config.packageMonitoring.packageDomainLimitUpgradeEmailSubject;
}
@Provides
@Config("packageDomainLimitUpgradeEmailBody")
public static String providePackageDomainLimitUpgradeEmailBody(RegistryConfigSettings config) {
return config.packageMonitoring.packageDomainLimitUpgradeEmailBody;
}
}
@@ -47,6 +47,7 @@ public class RegistryConfigSettings {
/** Configuration options that apply to the entire GCP project. */
public static class GcpProject {
public String projectId;
public long projectIdNumber;
public String locationId;
public boolean isLocal;
public String defaultServiceUrl;
@@ -60,6 +61,7 @@ public class RegistryConfigSettings {
public List<String> availableOauthScopes;
public List<String> requiredOauthScopes;
public List<String> allowedOauthClientIds;
public String iapClientId;
}
/** Configuration options for accessing Google APIs. */
@@ -254,7 +256,11 @@ public class RegistryConfigSettings {
/** Configuration for package compliance monitoring. */
public static class PackageMonitoring {
public String packageCreateLimitEmailSubjectText;
public String packageCreateLimitEmailBodyText;
public String packageCreateLimitEmailSubject;
public String packageCreateLimitEmailBody;
public String packageDomainLimitWarningEmailSubject;
public String packageDomainLimitWarningEmailBody;
public String packageDomainLimitUpgradeEmailSubject;
public String packageDomainLimitUpgradeEmailBody;
}
}
@@ -8,6 +8,8 @@
gcpProject:
# Globally unique GCP project ID
projectId: registry-project-id
# Corresponding project ID number
projectIdNumber: 123456789012
# Location of the GCP project, note that us-central1 and europe-west1 are special in that
# they are used without the trailing number in GCP commands and Google Cloud Console.
# See: https://cloud.google.com/appengine/docs/locations as an example
@@ -304,6 +306,9 @@ oAuth:
# in this list. Client IDs are typically of the format
# numbers-alphanumerics.apps.googleusercontent.com
allowedOauthClientIds: []
# GCP Identity-Aware Proxy client ID, if set up (note: this requires manual setup
# of User objects in the database for Nomulus tool users)
iapClientId: null
credentialOAuth:
# OAuth scopes required for accessing Google APIs using the default
@@ -534,9 +539,9 @@ sslCertificateValidation:
# Configuration options for the package compliance monitoring
packageMonitoring:
# Email subject text to notify partners their package has exceeded the limit for domain creates
packageCreateLimitEmailSubjectText: "NOTICE: Your Package Is Being Upgraded"
packageCreateLimitEmailSubject: "NOTICE: Your package is being upgraded"
# Email body text template notify partners their package has exceeded the limit for domain creates
packageCreateLimitEmailBodyText: >
packageCreateLimitEmailBody: >
Dear %1$s,
We are contacting you to inform you that your package with the package token
@@ -548,3 +553,37 @@ packageMonitoring:
Regards,
Example Registry
# Email subject text to notify partners their package has exceeded the limit for current active domains and warn them their package will be upgraded in 30 days
packageDomainLimitWarningEmailSubject: "WARNING: Your package has exceeded the domain limit"
# Email body text template to warn partners their package has exceeded the limit for active domains and will be upgraded in 30 days
packageDomainLimitWarningEmailBody: >
Dear %1$s,
We are contacting you to inform you that your package with the package token
%2$s has exceeded its limit for active domains.
Your package will be upgraded to the next tier in 30 days if the number of active domains does not return below the limit.
If you have any questions or require additional support, please contact us
at %3$s.
Regards,
Example Registry
# Email subject text to notify partners their package has exceeded the limit
# for current active domains for more than 30 days and will be upgraded
packageDomainLimitUpgradeEmailSubject: "NOTICE: Your package is being upgraded"
# Email body text template to warn partners their package has exceeded the
# limit for active domains for more than 30 days and will be upgraded
packageDomainLimitUpgradeEmailBody: >
Dear %1$s,
We are contacting you to inform you that your package with the package token
%2$s has exceeded its limit for active domains.
Your package will now be upgraded to the next tier.
If you have any questions or require additional support, please contact us
at %3$s.
Regards,
Example Registry
@@ -175,16 +175,6 @@
<url-pattern>/_dr/cron/fanout</url-pattern>
</servlet-mapping>
<!-- Pipeline GUI servlets. -->
<servlet>
<servlet-name>pipeline</servlet-name>
<servlet-class>com.google.appengine.tools.pipeline.impl.servlets.PipelineServlet</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>pipeline</servlet-name>
<url-pattern>/_ah/pipeline/*</url-pattern>
</servlet-mapping>
<!-- Syncs registrars to the registrar spreadsheet. -->
<servlet-mapping>
<servlet-name>backend-servlet</servlet-name>
@@ -13,6 +13,29 @@
<load-on-startup>1</load-on-startup>
</servlet>
<!-- Dedicated servlet for static content with cache control -->
<servlet>
<servlet-name>default</servlet-name>
<servlet-class>org.eclipse.jetty.servlet.DefaultServlet</servlet-class>
<init-param>
<param-name>cacheControl</param-name>
<param-value>max-age=18000,public</param-value>
</init-param>
<init-param>
<param-name>precompressed</param-name>
<param-value>true</param-value>
</init-param>
<init-param>
<param-name>dirAllowed</param-name>
<param-value>false</param-value>
</init-param>
<load-on-startup>1</load-on-startup>
</servlet>
<servlet-mapping>
<servlet-name>default</servlet-name>
<url-pattern>/console/*</url-pattern>
</servlet-mapping>
<!-- The primary EPP endpoint for the Registry, which accepts EPP requests from our TLS proxy. -->
<servlet-mapping>
<servlet-name>frontend-servlet</servlet-name>
@@ -16,7 +16,6 @@ package google.registry.export;
import static com.google.common.base.Verify.verifyNotNull;
import static google.registry.model.tld.Registries.getTldsOfType;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.request.Action.Method.POST;
import static java.nio.charset.StandardCharsets.UTF_8;
@@ -89,8 +88,7 @@ public class ExportDomainListsAction implements Runnable {
// DateTime are persisted as timestamp_z in SQL. It is only the
// validation that compares the Java types, and only with the first
// field that compares with the substituted value.
jpaTm()
.query(
tm().query(
"SELECT domainName FROM Domain "
+ "WHERE tld = :tld "
+ "AND deletionTime > :now "
@@ -37,7 +37,6 @@ import static google.registry.model.tld.label.ReservationType.FULLY_BLOCKED;
import static google.registry.model.tld.label.ReservationType.NAME_COLLISION;
import static google.registry.model.tld.label.ReservationType.RESERVED_FOR_ANCHOR_TENANT;
import static google.registry.model.tld.label.ReservationType.RESERVED_FOR_SPECIFIC_USE;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.pricing.PricingEngineProxy.isDomainPremium;
import static google.registry.util.CollectionUtils.nullToEmpty;
@@ -1179,8 +1178,7 @@ public class DomainFlowUtils {
private static List<DomainHistory> findRecentHistoryEntries(
Domain domain, DateTime now, Duration maxSearchPeriod) {
return jpaTm()
.query(
return tm().query(
"FROM DomainHistory WHERE modificationTime >= :beginning AND repoId = "
+ ":repoId ORDER BY modificationTime ASC",
DomainHistory.class)
@@ -15,7 +15,6 @@
package google.registry.flows.domain.token;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.base.Strings;
@@ -229,8 +228,8 @@ public class AllocationTokenFlowUtils {
// the Recurring billing event is reloaded later in the renew flow, so we synchronize changed
// RecurringBillingEvent with storage manually
tm().put(newRecurringBillingEvent);
jpaTm().getEntityManager().flush();
jpaTm().getEntityManager().clear();
tm().getEntityManager().flush();
tm().getEntityManager().clear();
// Remove current package token
return domain
@@ -14,7 +14,7 @@
package google.registry.model;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import javax.annotation.Nullable;
import javax.persistence.Column;
@@ -34,7 +34,7 @@ public class CreateAutoTimestamp extends ImmutableObject implements UnsafeSerial
@PreUpdate
void setTimestamp() {
if (creationTime == null) {
creationTime = jpaTm().getTransactionTime();
creationTime = tm().getTransactionTime();
}
}
@@ -16,7 +16,6 @@ package google.registry.model;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import static google.registry.util.DateTimeUtils.isAtOrAfter;
@@ -41,8 +40,8 @@ import google.registry.model.transfer.DomainTransferData;
import google.registry.model.transfer.TransferData;
import google.registry.model.transfer.TransferStatus;
import google.registry.persistence.VKey;
import java.util.Collection;
import java.util.Comparator;
import java.util.List;
import java.util.Optional;
import java.util.function.Function;
import java.util.function.Supplier;
@@ -184,7 +183,7 @@ public final class EppResourceUtils {
* @param now the logical time of the check
*/
public static <T extends EppResource> ImmutableSet<String> checkResourcesExist(
Class<T> clazz, List<String> uniqueIds, final DateTime now) {
Class<T> clazz, Collection<String> uniqueIds, final DateTime now) {
return ForeignKeyUtils.load(clazz, uniqueIds, now).keySet();
}
@@ -349,14 +348,12 @@ public final class EppResourceUtils {
Query query;
if (isContactKey) {
query =
jpaTm()
.query(CONTACT_LINKED_DOMAIN_QUERY, String.class)
tm().query(CONTACT_LINKED_DOMAIN_QUERY, String.class)
.setParameter("fkRepoId", key)
.setParameter("now", now);
} else {
query =
jpaTm()
.getEntityManager()
tm().getEntityManager()
.createNativeQuery(HOST_LINKED_DOMAIN_QUERY)
.setParameter("fkRepoId", key.getKey())
.setParameter("now", now.toDate());
@@ -18,8 +18,8 @@ import static com.google.common.collect.ImmutableList.toImmutableList;
import static com.google.common.collect.ImmutableMap.toImmutableMap;
import static google.registry.config.RegistryConfig.getEppResourceCachingDuration;
import static google.registry.config.RegistryConfig.getEppResourceMaxCachedEntries;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.replicaJpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.replicaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.github.benmanes.caffeine.cache.CacheLoader;
import com.github.benmanes.caffeine.cache.LoadingCache;
@@ -106,12 +106,12 @@ public final class ForeignKeyUtils {
* to duplicate keys.
*/
private static <E extends EppResource> ImmutableMap<String, MostRecentResource> load(
Class<E> clazz, Collection<String> foreignKeys, boolean useReplicaJpaTm) {
Class<E> clazz, Collection<String> foreignKeys, boolean useReplicaTm) {
String fkProperty = RESOURCE_TYPE_TO_FK_PROPERTY.get(clazz);
JpaTransactionManager jpaTmToUse = useReplicaJpaTm ? replicaJpaTm() : jpaTm();
return jpaTmToUse.transact(
JpaTransactionManager tmToUse = useReplicaTm ? replicaTm() : tm();
return tmToUse.transact(
() ->
jpaTmToUse
tmToUse
.query(
("SELECT %fkProperty%, repoId, deletionTime FROM %entity% WHERE (%fkProperty%,"
+ " deletionTime) IN (SELECT %fkProperty%, MAX(deletionTime) FROM"
@@ -15,7 +15,7 @@
package google.registry.model;
import static com.google.common.base.Preconditions.checkState;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static org.joda.time.DateTimeZone.UTC;
import com.google.appengine.api.datastore.DatastoreServiceFactory;
@@ -70,12 +70,10 @@ public final class IdService {
* <p>The generated IDs are project-wide unique
*/
private static Long getSequenceBasedId() {
return jpaTm()
.transact(
return tm().transact(
() ->
(BigInteger)
jpaTm()
.getEntityManager()
tm().getEntityManager()
.createNativeQuery("SELECT nextval('project_wide_unique_id_seq')")
.getSingleResult())
.longValue();
@@ -51,15 +51,6 @@ public abstract class ImmutableObject implements Cloneable {
@Target(FIELD)
public @interface DoNotHydrate {}
/**
* Indicates that the field stores a null value to indicate an empty set. This is also used in
* object comparison.
*/
@Documented
@Retention(RUNTIME)
@Target(FIELD)
public @interface EmptySetToNull {}
/**
* Indicates that the field does not take part in the immutability contract.
*
@@ -14,7 +14,7 @@
package google.registry.model;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import java.util.Optional;
@@ -38,7 +38,7 @@ public class UpdateAutoTimestamp extends ImmutableObject implements UnsafeSerial
@PrePersist
@PreUpdate
void setTimestamp() {
lastUpdateTime = jpaTm().getTransactionTime();
lastUpdateTime = tm().getTransactionTime();
}
/** Returns the timestamp, or {@code START_OF_TIME} if it's null. */
@@ -15,7 +15,7 @@
package google.registry.model.common;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import com.github.benmanes.caffeine.cache.LoadingCache;
@@ -192,7 +192,7 @@ public class DatabaseMigrationStateSchedule extends CrossTldSingleton {
/** Sets and persists to SQL the provided migration transition schedule. */
public static void set(ImmutableSortedMap<DateTime, MigrationState> migrationTransitionMap) {
jpaTm().assertInTransaction();
tm().assertInTransaction();
TimedTransitionProperty<MigrationState> transitions =
TimedTransitionProperty.make(
migrationTransitionMap,
@@ -201,7 +201,7 @@ public class DatabaseMigrationStateSchedule extends CrossTldSingleton {
MigrationState.DATASTORE_ONLY,
"migrationTransitionMap must start with DATASTORE_ONLY");
validateTransitionAtCurrentTime(transitions);
jpaTm().put(new DatabaseMigrationStateSchedule(transitions));
tm().put(new DatabaseMigrationStateSchedule(transitions));
CACHE.invalidateAll();
}
@@ -218,12 +218,10 @@ public class DatabaseMigrationStateSchedule extends CrossTldSingleton {
/** Loads the currently-set migration schedule from SQL, or the default if none exists. */
@VisibleForTesting
static TimedTransitionProperty<MigrationState> getUncached() {
return jpaTm()
.transactWithoutBackup(
return tm().transactWithoutBackup(
() -> {
try {
return jpaTm()
.loadSingleton(DatabaseMigrationStateSchedule.class)
return tm().loadSingleton(DatabaseMigrationStateSchedule.class)
.map(s -> s.migrationTransitions)
.orElse(DEFAULT_TRANSITION_MAP);
} catch (PersistenceException e) {
@@ -245,8 +243,8 @@ public class DatabaseMigrationStateSchedule extends CrossTldSingleton {
*/
private static void validateTransitionAtCurrentTime(
TimedTransitionProperty<MigrationState> newTransitions) {
MigrationState currentValue = getUncached().getValueAtTime(jpaTm().getTransactionTime());
MigrationState nextCurrentValue = newTransitions.getValueAtTime(jpaTm().getTransactionTime());
MigrationState currentValue = getUncached().getValueAtTime(tm().getTransactionTime());
MigrationState nextCurrentValue = newTransitions.getValueAtTime(tm().getTransactionTime());
checkArgument(
VALID_STATE_TRANSITIONS.get(currentValue).contains(nextCurrentValue),
"Cannot transition from current state-as-of-now %s to new state-as-of-now %s",
@@ -15,7 +15,7 @@
package google.registry.model.console;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import java.util.Optional;
@@ -24,11 +24,9 @@ public class UserDao {
/** Retrieves the one user with this email address if it exists. */
public static Optional<User> loadUser(String emailAddress) {
return jpaTm()
.transact(
return tm().transact(
() ->
jpaTm()
.query("FROM User WHERE emailAddress = :emailAddress", User.class)
tm().query("FROM User WHERE emailAddress = :emailAddress", User.class)
.setParameter("emailAddress", emailAddress)
.getResultStream()
.findFirst());
@@ -36,8 +34,7 @@ public class UserDao {
/** Saves the given user, checking that no existing user already exists with this email. */
public static void saveUser(User user) {
jpaTm()
.transact(
tm().transact(
() -> {
// Check for an existing user (the unique constraint protects us, but this gives a
// nicer exception)
@@ -51,7 +48,7 @@ public class UserDao {
+ " email already exists with ID %s",
user.getEmailAddress(), user.getId(), savedUser.getId()));
}
jpaTm().put(user);
tm().put(user);
});
}
}
@@ -124,7 +124,7 @@ public class DomainBase extends EppResource
String tld;
/** References to hosts that are the nameservers for the domain. */
@EmptySetToNull @Transient Set<VKey<Host>> nsHosts;
@Transient Set<VKey<Host>> nsHosts;
/** Contacts. */
VKey<Contact> adminContact;
@@ -96,7 +96,6 @@ public class DomainHistory extends HistoryEntry {
columnList =
"domain_history_history_revision_id,domain_history_domain_repo_id,host_repo_id",
unique = true))
@EmptySetToNull
@Column(name = "host_repo_id")
Set<VKey<Host>> nsHosts;
@@ -160,7 +159,6 @@ public class DomainHistory extends HistoryEntry {
@OneToMany(cascade = CascadeType.ALL, fetch = FetchType.EAGER, orphanRemoval = true)
@JoinColumn(name = "historyRevisionId", referencedColumnName = "historyRevisionId")
@JoinColumn(name = "domainRepoId", referencedColumnName = "domainRepoId")
@EmptySetToNull
Set<DomainTransactionRecord> domainTransactionRecords;
public Set<DomainTransactionRecord> getDomainTransactionRecords() {
@@ -15,7 +15,6 @@
package google.registry.model.domain.token;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
@@ -101,9 +100,8 @@ public class PackagePromotion extends ImmutableObject implements Buildable {
/** Loads and returns a PackagePromotion entity by its token string directly from Cloud SQL. */
public static Optional<PackagePromotion> loadByTokenString(String tokenString) {
jpaTm().assertInTransaction();
return jpaTm()
.query("FROM PackagePromotion WHERE token = :token", PackagePromotion.class)
tm().assertInTransaction();
return tm().query("FROM PackagePromotion WHERE token = :token", PackagePromotion.class)
.setParameter("token", VKey.create(AllocationToken.class, tokenString))
.getResultStream()
.findFirst();
@@ -27,7 +27,6 @@ import static com.google.common.io.BaseEncoding.base64;
import static google.registry.config.RegistryConfig.getDefaultRegistrarWhoisServer;
import static google.registry.model.CacheUtils.memoizeWithShortExpiration;
import static google.registry.model.tld.Registries.assertTldsExist;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.CollectionUtils.nullToEmptyImmutableCopy;
import static google.registry.util.CollectionUtils.nullToEmptyImmutableSortedCopy;
@@ -580,8 +579,7 @@ public class Registrar extends UpdateAutoTimestampEntity implements Buildable, J
private Iterable<RegistrarPoc> getContactsIterable() {
return tm().transact(
() ->
jpaTm()
.query("FROM RegistrarPoc WHERE registrarId = :registrarId", RegistrarPoc.class)
tm().query("FROM RegistrarPoc WHERE registrarId = :registrarId", RegistrarPoc.class)
.setParameter("registrarId", registrarId)
.getResultStream()
.collect(toImmutableList()));
@@ -20,7 +20,6 @@ import static com.google.common.base.Strings.isNullOrEmpty;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static com.google.common.io.BaseEncoding.base64;
import static google.registry.model.registrar.Registrar.checkValidEmail;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.CollectionUtils.nullToEmptyImmutableSortedCopy;
import static google.registry.util.PasswordUtils.SALT_SUPPLIER;
@@ -185,8 +184,7 @@ public class RegistrarPoc extends ImmutableObject implements Jsonifiable, Unsafe
() -> {
ImmutableSet<String> emailAddressesToKeep =
contacts.stream().map(RegistrarPoc::getEmailAddress).collect(toImmutableSet());
jpaTm()
.query(
tm().query(
"DELETE FROM RegistrarPoc WHERE registrarId = :registrarId AND "
+ "emailAddress NOT IN :emailAddressesToKeep")
.setParameter("registrarId", registrar.getRegistrarId())
@@ -16,7 +16,7 @@ package google.registry.model.reporting;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
@@ -56,8 +56,7 @@ public class HistoryEntryDao {
/** Loads all history objects in the times specified, including all types. */
public static ImmutableList<HistoryEntry> loadAllHistoryObjects(
DateTime afterTime, DateTime beforeTime) {
return jpaTm()
.transact(
return tm().transact(
() ->
new ImmutableList.Builder<HistoryEntry>()
.addAll(loadAllHistoryObjects(ContactHistory.class, afterTime, beforeTime))
@@ -84,8 +83,8 @@ public class HistoryEntryDao {
/** Loads all history objects in the time period specified for the given {@link EppResource}. */
public static ImmutableList<HistoryEntry> loadHistoryObjectsForResource(
VKey<? extends EppResource> resourceKey, DateTime afterTime, DateTime beforeTime) {
return jpaTm()
.transact(() -> loadHistoryObjectsForResourceInternal(resourceKey, afterTime, beforeTime));
return tm().transact(
() -> loadHistoryObjectsForResourceInternal(resourceKey, afterTime, beforeTime));
}
/**
@@ -119,8 +118,7 @@ public class HistoryEntryDao {
/** Loads all history objects from all time from the given registrars. */
public static Iterable<HistoryEntry> loadHistoryObjectsByRegistrars(
ImmutableCollection<String> registrarIds) {
return jpaTm()
.transact(
return tm().transact(
() ->
Streams.concat(
loadHistoryObjectByRegistrarsInternal(ContactHistory.class, registrarIds),
@@ -132,8 +130,7 @@ public class HistoryEntryDao {
private static <T extends HistoryEntry> Stream<T> loadHistoryObjectByRegistrarsInternal(
Class<T> historyClass, ImmutableCollection<String> registrarIds) {
return jpaTm()
.criteriaQuery(
return tm().criteriaQuery(
CriteriaQueryBuilder.create(historyClass)
.whereFieldIsIn("clientId", registrarIds)
.build())
@@ -144,7 +141,7 @@ public class HistoryEntryDao {
VKey<? extends EppResource> resourceKey, DateTime afterTime, DateTime beforeTime) {
// The class we're searching from is based on which resource type (e.g. Domain) we have
Class<? extends HistoryEntry> historyClass = getHistoryClassFromParent(resourceKey.getKind());
CriteriaBuilder criteriaBuilder = jpaTm().getEntityManager().getCriteriaBuilder();
CriteriaBuilder criteriaBuilder = tm().getEntityManager().getCriteriaBuilder();
CriteriaQuery<? extends HistoryEntry> criteriaQuery =
CriteriaQueryBuilder.create(historyClass)
.where("modificationTime", criteriaBuilder::greaterThanOrEqualTo, afterTime)
@@ -154,7 +151,7 @@ public class HistoryEntryDao {
.orderByAsc("modificationTime")
.build();
return ImmutableList.copyOf(jpaTm().criteriaQuery(criteriaQuery).getResultList());
return ImmutableList.copyOf(tm().criteriaQuery(criteriaQuery).getResultList());
}
public static Class<? extends HistoryEntry> getHistoryClassFromParent(
@@ -168,9 +165,8 @@ public class HistoryEntryDao {
private static <T extends HistoryEntry> List<T> loadAllHistoryObjects(
Class<T> historyClass, DateTime afterTime, DateTime beforeTime) {
CriteriaBuilder criteriaBuilder = jpaTm().getEntityManager().getCriteriaBuilder();
return jpaTm()
.criteriaQuery(
CriteriaBuilder criteriaBuilder = tm().getEntityManager().getCriteriaBuilder();
return tm().criteriaQuery(
CriteriaQueryBuilder.create(historyClass)
.where("modificationTime", criteriaBuilder::greaterThanOrEqualTo, afterTime)
.where("modificationTime", criteriaBuilder::lessThanOrEqualTo, beforeTime)
@@ -22,6 +22,7 @@ import com.google.common.collect.ImmutableSet;
import google.registry.model.Buildable;
import google.registry.model.ImmutableObject;
import google.registry.util.DomainNameUtils;
import java.io.Serializable;
import java.util.Set;
import javax.persistence.Column;
import javax.persistence.Entity;
@@ -39,7 +40,7 @@ import org.joda.time.LocalDate;
@Index(name = "spec11threatmatch_tld_idx", columnList = "tld"),
@Index(name = "spec11threatmatch_check_date_idx", columnList = "checkDate")
})
public class Spec11ThreatMatch extends ImmutableObject implements Buildable {
public class Spec11ThreatMatch extends ImmutableObject implements Buildable, Serializable {
/** The type of threat detected. */
public enum ThreatType {
@@ -132,16 +133,6 @@ public class Spec11ThreatMatch extends ImmutableObject implements Buildable {
return super.build();
}
/**
* Manually set the ID for testing or other special circumstances.
*
* <p>In general the ID is generated by SQL and there should be no need to set it manually.
*/
public Builder setId(Long id) {
getInstance().id = id;
return this;
}
public Builder setDomainName(String domainName) {
getInstance().domainName = domainName;
getInstance().tld = DomainNameUtils.getTldFromDomainName(domainName);
@@ -15,7 +15,7 @@
package google.registry.model.server;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.isAtOrAfter;
import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
@@ -212,12 +212,11 @@ public class Lock extends ImmutableObject implements Serializable {
String scope = tld != null ? tld : GLOBAL;
Supplier<AcquireResult> lockAcquirer =
() -> {
DateTime now = jpaTm().getTransactionTime();
DateTime now = tm().getTransactionTime();
// Checking if an unexpired lock still exists - if so, the lock can't be acquired.
Lock lock =
jpaTm()
.loadByKeyIfPresent(VKey.create(Lock.class, new LockId(resourceName, scope)))
tm().loadByKeyIfPresent(VKey.create(Lock.class, new LockId(resourceName, scope)))
.orElse(null);
if (lock != null) {
logger.atInfo().log(
@@ -237,11 +236,11 @@ public class Lock extends ImmutableObject implements Serializable {
Lock newLock =
create(resourceName, scope, requestStatusChecker.getLogId(), now, leaseLength);
jpaTm().put(newLock);
tm().put(newLock);
return AcquireResult.create(now, lock, newLock, lockState);
};
AcquireResult acquireResult = jpaTm().transactWithoutBackup(lockAcquirer);
AcquireResult acquireResult = tm().transactWithoutBackup(lockAcquirer);
logAcquireResult(acquireResult);
lockMetrics.recordAcquire(resourceName, scope, acquireResult.lockState());
@@ -258,15 +257,15 @@ public class Lock extends ImmutableObject implements Serializable {
// this can happen if release() is called around the expiration time and the lock
// expires underneath us.
VKey<Lock> key = VKey.create(Lock.class, new LockId(resourceName, scope));
Lock loadedLock = jpaTm().loadByKeyIfPresent(key).orElse(null);
Lock loadedLock = tm().loadByKeyIfPresent(key).orElse(null);
if (equals(loadedLock)) {
// Use deleteIgnoringReadOnly() so that we don't create a commit log entry for deleting
// the lock.
logger.atInfo().log("Deleting lock: %s", lockId);
jpaTm().delete(key);
tm().delete(key);
lockMetrics.recordRelease(
resourceName, scope, new Duration(acquiredTime, jpaTm().getTransactionTime()));
resourceName, scope, new Duration(acquiredTime, tm().getTransactionTime()));
} else {
logger.atSevere().log(
"The lock we acquired was transferred to someone else before we"
@@ -278,7 +277,7 @@ public class Lock extends ImmutableObject implements Serializable {
}
return null;
};
jpaTm().transactWithoutBackup(lockReleaser);
tm().transactWithoutBackup(lockReleaser);
}
static class LockId extends ImmutableObject implements Serializable {
@@ -14,7 +14,7 @@
package google.registry.model.smd;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import com.google.common.collect.ImmutableMap;
@@ -28,15 +28,12 @@ public class SignedMarkRevocationListDao {
/** Loads the {@link SignedMarkRevocationList}. */
static SignedMarkRevocationList load() {
Optional<SignedMarkRevocationList> smdrl =
jpaTm()
.transact(
tm().transact(
() -> {
Long revisionId =
jpaTm()
.query("SELECT MAX(revisionId) FROM SignedMarkRevocationList", Long.class)
tm().query("SELECT MAX(revisionId) FROM SignedMarkRevocationList", Long.class)
.getSingleResult();
return jpaTm()
.query(
return tm().query(
"FROM SignedMarkRevocationList smrl LEFT JOIN FETCH smrl.revokes "
+ "WHERE smrl.revisionId = :revisionId",
SignedMarkRevocationList.class)
@@ -49,7 +46,7 @@ public class SignedMarkRevocationListDao {
/** Save the given {@link SignedMarkRevocationList} */
static void save(SignedMarkRevocationList signedMarkRevocationList) {
jpaTm().transact(() -> jpaTm().insert(signedMarkRevocationList));
tm().transact(() -> tm().insert(signedMarkRevocationList));
logger.atInfo().log(
"Inserted %,d signed mark revocations into Cloud SQL.",
signedMarkRevocationList.revokes.size());
@@ -22,7 +22,6 @@ import static com.google.common.base.Strings.emptyToNull;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static com.google.common.collect.Maps.filterValues;
import static google.registry.model.CacheUtils.memoizeWithShortExpiration;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.CollectionUtils.entriesToImmutableMap;
import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
@@ -59,7 +58,7 @@ public final class Registries {
() ->
tm().transact(
() -> {
EntityManager entityManager = jpaTm().getEntityManager();
EntityManager entityManager = tm().getEntityManager();
Stream<?> resultStream =
entityManager
.createQuery("SELECT tldStr, tldType FROM Tld")
@@ -15,7 +15,7 @@
package google.registry.model.tld;
import static com.google.common.base.Preconditions.checkNotNull;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableList;
import google.registry.model.domain.RegistryLock;
@@ -26,31 +26,29 @@ public final class RegistryLockDao {
/** Returns the {@link RegistryLock} referred to by this revision ID, or empty if none exists. */
public static Optional<RegistryLock> getByRevisionId(long revisionId) {
jpaTm().assertInTransaction();
return Optional.ofNullable(jpaTm().getEntityManager().find(RegistryLock.class, revisionId));
tm().assertInTransaction();
return Optional.ofNullable(tm().getEntityManager().find(RegistryLock.class, revisionId));
}
/** Returns the most recent version of the {@link RegistryLock} referred to by the code. */
public static Optional<RegistryLock> getByVerificationCode(String verificationCode) {
jpaTm().assertInTransaction();
tm().assertInTransaction();
Long revisionId =
jpaTm()
.query(
tm().query(
"SELECT MAX(revisionId) FROM RegistryLock WHERE verificationCode ="
+ " :verificationCode",
Long.class)
.setParameter("verificationCode", verificationCode)
.getSingleResult();
return Optional.ofNullable(revisionId)
.map(revision -> jpaTm().getEntityManager().find(RegistryLock.class, revision));
.map(revision -> tm().getEntityManager().find(RegistryLock.class, revision));
}
/** Returns all lock objects that this registrar has created, including pending locks. */
public static ImmutableList<RegistryLock> getLocksByRegistrarId(String registrarId) {
jpaTm().assertInTransaction();
tm().assertInTransaction();
return ImmutableList.copyOf(
jpaTm()
.query(
tm().query(
"SELECT lock FROM RegistryLock lock WHERE lock.registrarId = :registrarId"
+ " AND lock.unlockCompletionTime IS NULL ORDER BY lock.domainName ASC",
RegistryLock.class)
@@ -64,9 +62,8 @@ public final class RegistryLockDao {
* <p>Returns empty if this domain hasn't been locked before.
*/
public static Optional<RegistryLock> getMostRecentByRepoId(String repoId) {
jpaTm().assertInTransaction();
return jpaTm()
.query(
tm().assertInTransaction();
return tm().query(
"SELECT lock FROM RegistryLock lock WHERE lock.repoId = :repoId"
+ " ORDER BY lock.revisionId DESC",
RegistryLock.class)
@@ -83,9 +80,8 @@ public final class RegistryLockDao {
* {@link #getMostRecentByRepoId(String)} in that it only returns verified locks.
*/
public static Optional<RegistryLock> getMostRecentVerifiedLockByRepoId(String repoId) {
jpaTm().assertInTransaction();
return jpaTm()
.query(
tm().assertInTransaction();
return tm().query(
"SELECT lock FROM RegistryLock lock WHERE lock.repoId = :repoId AND"
+ " lock.lockCompletionTime IS NOT NULL AND lock.unlockCompletionTime IS NULL"
+ " ORDER BY lock.revisionId DESC",
@@ -103,9 +99,8 @@ public final class RegistryLockDao {
* {@link #getMostRecentByRepoId(String)} in that it only returns verified unlocks.
*/
public static Optional<RegistryLock> getMostRecentVerifiedUnlockByRepoId(String repoId) {
jpaTm().assertInTransaction();
return jpaTm()
.query(
tm().assertInTransaction();
return tm().query(
"SELECT lock FROM RegistryLock lock WHERE lock.repoId = :repoId AND"
+ " lock.unlockCompletionTime IS NOT NULL ORDER BY lock.revisionId DESC",
RegistryLock.class)
@@ -116,8 +111,8 @@ public final class RegistryLockDao {
}
public static RegistryLock save(RegistryLock registryLock) {
jpaTm().assertInTransaction();
tm().assertInTransaction();
checkNotNull(registryLock, "Null registry lock cannot be saved");
return jpaTm().getEntityManager().merge(registryLock);
return tm().getEntityManager().merge(registryLock);
}
}
@@ -19,7 +19,7 @@ import static com.google.common.collect.ImmutableList.toImmutableList;
import static google.registry.config.RegistryConfig.getDomainLabelListCacheDuration;
import static google.registry.config.RegistryConfig.getSingletonCachePersistDuration;
import static google.registry.config.RegistryConfig.getStaticPremiumListMaxCachedEntries;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.CollectionUtils.isNullOrEmpty;
import com.github.benmanes.caffeine.cache.LoadingCache;
@@ -135,11 +135,10 @@ public class PremiumListDao {
/** Saves the given premium list (and its premium list entries) to Cloud SQL. */
public static PremiumList save(PremiumList premiumList) {
jpaTm()
.transact(
tm().transact(
() -> {
jpaTm().insert(premiumList);
jpaTm().getEntityManager().flush(); // This populates the revisionId.
tm().insert(premiumList);
tm().getEntityManager().flush(); // This populates the revisionId.
long revisionId = premiumList.getRevisionId();
if (!isNullOrEmpty(premiumList.getLabelsToPrices())) {
@@ -148,7 +147,7 @@ public class PremiumListDao {
.getLabelsToPrices()
.forEach(
(key, value) -> entries.add(PremiumEntry.create(revisionId, value, key)));
jpaTm().insertAll(entries.build());
tm().insertAll(entries.build());
}
});
premiumListCache.invalidate(premiumList.getName());
@@ -156,27 +155,23 @@ public class PremiumListDao {
}
public static void delete(PremiumList premiumList) {
jpaTm()
.transact(
tm().transact(
() -> {
Optional<PremiumList> persistedList = getLatestRevision(premiumList.getName());
if (persistedList.isPresent()) {
jpaTm()
.query("DELETE FROM PremiumEntry WHERE revisionId = :revisionId")
tm().query("DELETE FROM PremiumEntry WHERE revisionId = :revisionId")
.setParameter("revisionId", persistedList.get().getRevisionId())
.executeUpdate();
jpaTm().delete(persistedList.get());
tm().delete(persistedList.get());
}
});
premiumListCache.invalidate(premiumList.getName());
}
private static Optional<PremiumList> getLatestRevisionUncached(String premiumListName) {
return jpaTm()
.transact(
return tm().transact(
() ->
jpaTm()
.query(
tm().query(
"FROM PremiumList WHERE name = :name ORDER BY revisionId DESC",
PremiumList.class)
.setParameter("name", premiumListName)
@@ -191,11 +186,9 @@ public class PremiumListDao {
* <p>This is an expensive operation and should only be used when the entire list is required.
*/
public static List<PremiumEntry> loadPremiumEntries(PremiumList premiumList) {
return jpaTm()
.transact(
return tm().transact(
() ->
jpaTm()
.query(
tm().query(
"FROM PremiumEntry pe WHERE pe.revisionId = :revisionId",
PremiumEntry.class)
.setParameter("revisionId", premiumList.getRevisionId())
@@ -207,11 +200,9 @@ public class PremiumListDao {
* retrieval so it should only be done in a cached context.
*/
static Optional<BigDecimal> getPriceForLabelUncached(RevisionIdAndLabel revisionIdAndLabel) {
return jpaTm()
.transact(
return tm().transact(
() ->
jpaTm()
.query(
tm().query(
"SELECT pe.price FROM PremiumEntry pe WHERE pe.revisionId = :revisionId"
+ " AND pe.domainLabel = :label",
BigDecimal.class)
@@ -22,7 +22,7 @@ import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.config.RegistryConfig.getDomainLabelListCacheDuration;
import static google.registry.model.tld.label.ReservationType.FULLY_BLOCKED;
import static google.registry.persistence.transaction.QueryComposer.Comparator.EQ;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.CollectionUtils.nullToEmpty;
import static org.joda.time.DateTimeZone.UTC;
@@ -78,8 +78,7 @@ public final class ReservedList
@PreRemove
void preRemove() {
jpaTm()
.query("DELETE FROM ReservedEntry WHERE revision_id = :revisionId")
tm().query("DELETE FROM ReservedEntry WHERE revision_id = :revisionId")
.setParameter("revisionId", revisionId)
.executeUpdate();
}
@@ -101,7 +100,7 @@ public final class ReservedList
entry -> {
// We can safely change the revision id since it's "Insignificant".
entry.revisionId = revisionId;
jpaTm().insert(entry);
tm().insert(entry);
});
}
}
@@ -200,10 +199,9 @@ public final class ReservedList
public synchronized ImmutableMap<String, ReservedListEntry> getReservedListEntries() {
if (reservedListMap == null) {
reservedListMap =
jpaTm()
.transact(
tm().transact(
() ->
jpaTm()
tm()
.createQueryComposer(ReservedListEntry.class)
.where("revisionId", EQ, revisionId)
.stream()
@@ -14,7 +14,7 @@
package google.registry.model.tld.label;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
import com.google.common.flogger.FluentLogger;
@@ -31,7 +31,7 @@ public class ReservedListDao {
public static void save(ReservedList reservedList) {
checkArgumentNotNull(reservedList, "Must specify reservedList");
logger.atInfo().log("Saving reserved list %s to Cloud SQL.", reservedList.getName());
jpaTm().transact(() -> jpaTm().insert(reservedList));
tm().transact(() -> tm().insert(reservedList));
logger.atInfo().log(
"Saved reserved list %s with %d entries to Cloud SQL.",
reservedList.getName(), reservedList.getReservedListEntries().size());
@@ -39,7 +39,7 @@ public class ReservedListDao {
/** Deletes a reserved list from Cloud SQL. */
public static void delete(ReservedList reservedList) {
jpaTm().transact(() -> jpaTm().delete(reservedList));
tm().transact(() -> tm().delete(reservedList));
}
/**
@@ -47,11 +47,9 @@ public class ReservedListDao {
* exists.
*/
public static Optional<ReservedList> getLatestRevision(String reservedListName) {
return jpaTm()
.transact(
return tm().transact(
() ->
jpaTm()
.query(
tm().query(
"FROM ReservedList WHERE revisionId IN "
+ "(SELECT MAX(revisionId) FROM ReservedList WHERE name = :name)",
ReservedList.class)
@@ -66,11 +64,9 @@ public class ReservedListDao {
* <p>This means that at least one reserved list revision must exist for the given name.
*/
public static boolean checkExists(String reservedListName) {
return jpaTm()
.transact(
return tm().transact(
() ->
jpaTm()
.query("SELECT 1 FROM ReservedList WHERE name = :name", Integer.class)
tm().query("SELECT 1 FROM ReservedList WHERE name = :name", Integer.class)
.setParameter("name", reservedListName)
.setMaxResults(1)
.getResultList()
@@ -18,7 +18,7 @@ import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.base.Preconditions.checkState;
import static com.google.common.collect.ImmutableMap.toImmutableMap;
import static google.registry.persistence.transaction.QueryComposer.Comparator.EQ;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.github.benmanes.caffeine.cache.LoadingCache;
import com.google.common.annotations.VisibleForTesting;
@@ -104,8 +104,7 @@ public class ClaimsList extends ImmutableObject {
@PreRemove
void preRemove() {
jpaTm()
.query("DELETE FROM ClaimsEntry WHERE revision_id = :revisionId")
tm().query("DELETE FROM ClaimsEntry WHERE revision_id = :revisionId")
.setParameter("revisionId", revisionId)
.executeUpdate();
}
@@ -123,7 +122,7 @@ public class ClaimsList extends ImmutableObject {
if (labelsToKeys != null) {
labelsToKeys.forEach(
(domainLabel, claimKey) ->
jpaTm().insert(new ClaimsEntry(revisionId, domainLabel, claimKey)));
tm().insert(new ClaimsEntry(revisionId, domainLabel, claimKey)));
}
}
@@ -169,10 +168,9 @@ public class ClaimsList extends ImmutableObject {
public ImmutableMap<String, String> getLabelsToKeys() {
if (labelsToKeys == null) {
labelsToKeys =
jpaTm()
.transact(
tm().transact(
() ->
jpaTm()
tm()
.createQueryComposer(ClaimsEntry.class)
.where("revisionId", EQ, revisionId)
.stream()
@@ -191,8 +189,7 @@ public class ClaimsList extends ImmutableObject {
*/
public long size() {
if (labelsToKeys == null) {
return jpaTm()
.createQueryComposer(ClaimsEntry.class)
return tm().createQueryComposer(ClaimsEntry.class)
.where("revisionId", EQ, revisionId)
.count();
}
@@ -209,11 +206,9 @@ public class ClaimsList extends ImmutableObject {
if (labelsToKeys != null) {
return Optional.ofNullable(labelsToKeys.get(label));
}
return jpaTm()
.transact(
return tm().transact(
() ->
jpaTm()
.createQueryComposer(ClaimsEntry.class)
tm().createQueryComposer(ClaimsEntry.class)
.where("revisionId", EQ, revisionId)
.where("domainLabel", EQ, label)
.first()
@@ -16,7 +16,7 @@ package google.registry.model.tmch;
import static google.registry.config.RegistryConfig.getClaimsListCacheDuration;
import static google.registry.persistence.transaction.QueryComposer.Comparator.EQ;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import com.github.benmanes.caffeine.cache.LoadingCache;
@@ -51,7 +51,7 @@ public class ClaimsListDao {
/** Saves the given {@link ClaimsList} to Cloud SQL. */
public static void save(ClaimsList claimsList) {
jpaTm().transact(() -> jpaTm().insert(claimsList));
tm().transact(() -> tm().insert(claimsList));
CACHE.put(ClaimsListDao.class, claimsList);
}
@@ -65,15 +65,12 @@ public class ClaimsListDao {
* doesn't exist.
*/
private static ClaimsList getUncached() {
return jpaTm()
.transact(
return tm().transact(
() -> {
Long revisionId =
jpaTm()
.query("SELECT MAX(revisionId) FROM ClaimsList", Long.class)
tm().query("SELECT MAX(revisionId) FROM ClaimsList", Long.class)
.getSingleResult();
return jpaTm()
.createQueryComposer(ClaimsList.class)
return tm().createQueryComposer(ClaimsList.class)
.where("revisionId", EQ, revisionId)
.first();
})
@@ -15,7 +15,7 @@
package google.registry.model.tmch;
import static com.google.common.base.Preconditions.checkNotNull;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import google.registry.model.common.CrossTldSingleton;
import java.util.Optional;
@@ -39,7 +39,7 @@ public final class TmchCrl extends CrossTldSingleton {
/** Returns the singleton instance of this entity, without memoization. */
public static Optional<TmchCrl> get() {
return jpaTm().transact(() -> jpaTm().loadSingleton(TmchCrl.class));
return tm().transact(() -> tm().loadSingleton(TmchCrl.class));
}
/**
@@ -49,14 +49,13 @@ public final class TmchCrl extends CrossTldSingleton {
* and actually newer than the one currently in the database.
*/
public static void set(final String crl, final String url) {
jpaTm()
.transact(
tm().transact(
() -> {
TmchCrl tmchCrl = new TmchCrl();
tmchCrl.updated = jpaTm().getTransactionTime();
tmchCrl.updated = tm().getTransactionTime();
tmchCrl.crl = checkNotNull(crl, "crl");
tmchCrl.url = checkNotNull(url, "url");
jpaTm().put(tmchCrl);
tm().put(tmchCrl);
});
}
@@ -14,7 +14,7 @@
package google.registry.persistence.transaction;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableList;
import java.util.Collection;
@@ -104,7 +104,7 @@ public class CriteriaQueryBuilder<T> {
/** Creates a query builder that will SELECT from the given class. */
public static <T> CriteriaQueryBuilder<T> create(Class<T> clazz) {
return create(jpaTm(), clazz);
return create(tm(), clazz);
}
/** Creates a query builder for the given entity manager. */
@@ -20,8 +20,8 @@ import java.lang.reflect.Proxy;
* A dummy implementation for {@link JpaTransactionManager} which throws exception when any of its
* method is invoked.
*
* <p>This is used to initialize the {@link TransactionManagerFactory#jpaTm()} when running unit
* tests, because obviously we cannot connect to the actual Cloud SQL backend in a unit test.
* <p>This is used to initialize the {@link TransactionManagerFactory#tm()} when running unit tests,
* because obviously we cannot connect to the actual Cloud SQL backend in a unit test.
*
* <p>If a unit test needs to access the Cloud SQL database, it must add {@code
* JpaTransactionManagerExtension} as a JUnit extension in the test class.
@@ -14,7 +14,7 @@
package google.registry.persistence.transaction;
import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.base.Function;
import com.google.common.collect.ImmutableList;
@@ -210,7 +210,7 @@ public abstract class QueryComposer<T> {
}
public void addToCriteriaQueryBuilder(CriteriaQueryBuilder queryBuilder) {
CriteriaBuilder criteriaBuilder = jpaTm().getEntityManager().getCriteriaBuilder();
CriteriaBuilder criteriaBuilder = tm().getEntityManager().getCriteriaBuilder();
queryBuilder.where(
fieldName, comparator.getComparisonFactory().apply(criteriaBuilder), value);
}
@@ -19,22 +19,16 @@ import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
import com.google.appengine.api.utils.SystemProperty;
import com.google.appengine.api.utils.SystemProperty.Environment.Value;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Suppliers;
import google.registry.config.RegistryEnvironment;
import google.registry.persistence.DaggerPersistenceComponent;
import google.registry.tools.RegistryToolEnvironment;
import google.registry.util.NonFinalForTesting;
import java.util.Optional;
import java.util.function.Supplier;
/** Factory class to create {@link TransactionManager} instance. */
// TODO: Rename this to PersistenceFactory and move to persistence package.
public final class TransactionManagerFactory {
/** Optional override to manually set the transaction manager per-test. */
private static Optional<TransactionManager> tmForTest = Optional.empty();
/** Supplier for jpaTm so that it is initialized only once, upon first usage. */
@NonFinalForTesting
private static Supplier<JpaTransactionManager> jpaTm =
@@ -77,42 +71,22 @@ public final class TransactionManagerFactory {
return SystemProperty.environment.value() == Value.Production;
}
/**
* Returns the {@link TransactionManager} instance.
*
* <p>Returns the {@link JpaTransactionManager} or replica based on the possible manually
* specified per-test transaction manager.
*/
public static TransactionManager tm() {
return tmForTest.orElseGet(TransactionManagerFactory::jpaTm);
}
/**
* Returns {@link JpaTransactionManager} instance.
*
* <p>Between invocations of {@link TransactionManagerFactory#setJpaTm} every call to this method
* returns the same instance.
*/
public static JpaTransactionManager jpaTm() {
public static JpaTransactionManager tm() {
return jpaTm.get();
}
/** Returns a read-only {@link JpaTransactionManager} instance if configured. */
public static JpaTransactionManager replicaJpaTm() {
public static JpaTransactionManager replicaTm() {
return replicaJpaTm.get();
}
/**
* Returns a {@link TransactionManager} that uses a replica database if one exists.
*
* <p>In Datastore mode, this is unchanged from the regular transaction manager. In SQL mode,
* however, this will be a reference to the read-only replica database if one is configured.
*/
public static TransactionManager replicaTm() {
return replicaJpaTm();
}
/** Sets the return of {@link #jpaTm()} to the given instance of {@link JpaTransactionManager}. */
/** Sets the return of {@link #tm()} to the given instance of {@link JpaTransactionManager}. */
public static void setJpaTm(Supplier<JpaTransactionManager> jpaTmSupplier) {
checkArgumentNotNull(jpaTmSupplier, "jpaTmSupplier");
checkState(
@@ -122,7 +96,7 @@ public final class TransactionManagerFactory {
jpaTm = Suppliers.memoize(jpaTmSupplier::get);
}
/** Sets the value of {@link #replicaJpaTm()} to the given {@link JpaTransactionManager}. */
/** Sets the value of {@link #replicaTm()} to the given {@link JpaTransactionManager}. */
public static void setReplicaJpaTm(Supplier<JpaTransactionManager> replicaJpaTmSupplier) {
checkArgumentNotNull(replicaJpaTmSupplier, "replicaJpaTmSupplier");
checkState(
@@ -133,7 +107,7 @@ public final class TransactionManagerFactory {
}
/**
* Makes {@link #jpaTm()} return the {@link JpaTransactionManager} instance provided by {@code
* Makes {@link #tm()} return the {@link JpaTransactionManager} instance provided by {@code
* jpaTmSupplier} from now on. This method should only be called by an implementor of {@link
* org.apache.beam.sdk.harness.JvmInitializer}.
*/
@@ -141,24 +115,4 @@ public final class TransactionManagerFactory {
checkArgumentNotNull(jpaTmSupplier, "jpaTmSupplier");
jpaTm = Suppliers.memoize(jpaTmSupplier::get);
}
/**
* Sets the return of {@link #tm()} to the given instance of {@link TransactionManager}.
*
* <p>DO NOT CALL THIS DIRECTLY IF POSSIBLE. Strongly prefer the use of <code>TmOverrideExtension
* </code> in test code instead.
*
* <p>Used when overriding the per-test transaction manager for dual-database tests. Should be
* matched with a corresponding invocation of {@link #removeTmOverrideForTest()} either at the end
* of the test or in an <code>@AfterEach</code> handler.
*/
@VisibleForTesting
public static void setTmOverrideForTest(TransactionManager newTmOverride) {
tmForTest = Optional.of(newTmOverride);
}
/** Resets the overridden transaction manager post-test. */
public static void removeTmOverrideForTest() {
tmForTest = Optional.empty();
}
}
@@ -16,7 +16,7 @@ package google.registry.rdap;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.model.EppResourceUtils.loadByForeignKeyCached;
import static google.registry.persistence.transaction.TransactionManagerFactory.replicaJpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.replicaTm;
import static google.registry.request.Action.Method.GET;
import static google.registry.request.Action.Method.HEAD;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
@@ -203,13 +203,13 @@ public class RdapDomainSearchAction extends RdapSearchActionBase {
int querySizeLimit = RESULT_SET_SIZE_SCALING_FACTOR * rdapResultSetMaxSize;
RdapResultSet<Domain> resultSet;
resultSet =
replicaJpaTm()
replicaTm()
.transact(
() -> {
CriteriaBuilder criteriaBuilder =
replicaJpaTm().getEntityManager().getCriteriaBuilder();
replicaTm().getEntityManager().getCriteriaBuilder();
CriteriaQueryBuilder<Domain> queryBuilder =
CriteriaQueryBuilder.create(replicaJpaTm(), Domain.class)
CriteriaQueryBuilder.create(replicaTm(), Domain.class)
.where(
"domainName",
criteriaBuilder::like,
@@ -239,7 +239,7 @@ public class RdapDomainSearchAction extends RdapSearchActionBase {
int querySizeLimit = RESULT_SET_SIZE_SCALING_FACTOR * rdapResultSetMaxSize;
RdapResultSet<Domain> resultSet;
resultSet =
replicaJpaTm()
replicaTm()
.transact(
() -> {
CriteriaQueryBuilder<Domain> builder =
@@ -305,7 +305,7 @@ public class RdapDomainSearchAction extends RdapSearchActionBase {
// incomplete result set if a search asks for something like "ns*", but we need to enforce a
// limit in order to avoid arbitrarily long-running queries.
Optional<String> desiredRegistrar = getDesiredRegistrar();
return replicaJpaTm()
return replicaTm()
.transact(
() -> {
CriteriaQueryBuilder<Host> builder =
@@ -319,7 +319,7 @@ public class RdapDomainSearchAction extends RdapSearchActionBase {
builder =
builder.where(
"currentSponsorRegistrarId",
replicaJpaTm().getEntityManager().getCriteriaBuilder()::equal,
replicaTm().getEntityManager().getCriteriaBuilder()::equal,
desiredRegistrar.get());
}
return getMatchingResources(builder, true, maxNameserversInFirstStage)
@@ -438,11 +438,11 @@ public class RdapDomainSearchAction extends RdapSearchActionBase {
parameters.put("desiredRegistrar", desiredRegistrar.get());
}
hostKeys =
replicaJpaTm()
replicaTm()
.transact(
() -> {
javax.persistence.Query query =
replicaJpaTm()
replicaTm()
.getEntityManager()
.createNativeQuery(queryBuilder.toString())
.setMaxResults(maxNameserversInFirstStage);
@@ -475,37 +475,37 @@ public class RdapDomainSearchAction extends RdapSearchActionBase {
int numHostKeysSearched = 0;
for (List<VKey<Host>> chunk : Iterables.partition(hostKeys, 30)) {
numHostKeysSearched += chunk.size();
replicaJpaTm()
.transact(
() -> {
for (VKey<Host> hostKey : hostKeys) {
CriteriaQueryBuilder<Domain> queryBuilder =
CriteriaQueryBuilder.create(replicaJpaTm(), Domain.class)
.whereFieldContains("nsHosts", hostKey)
.orderByAsc("domainName");
CriteriaBuilder criteriaBuilder =
replicaJpaTm().getEntityManager().getCriteriaBuilder();
if (!shouldIncludeDeleted()) {
queryBuilder =
queryBuilder.where(
"deletionTime", criteriaBuilder::greaterThan, getRequestTime());
}
if (cursorString.isPresent()) {
queryBuilder =
queryBuilder.where(
"domainName", criteriaBuilder::greaterThan, cursorString.get());
}
replicaJpaTm()
.criteriaQuery(queryBuilder.build())
.getResultStream()
.filter(this::isAuthorized)
.forEach(
(domain) -> {
Hibernate.initialize(domain.getDsData());
domainSetBuilder.add(domain);
});
replicaTm()
.transact(
() -> {
for (VKey<Host> hostKey : hostKeys) {
CriteriaQueryBuilder<Domain> queryBuilder =
CriteriaQueryBuilder.create(replicaTm(), Domain.class)
.whereFieldContains("nsHosts", hostKey)
.orderByAsc("domainName");
CriteriaBuilder criteriaBuilder =
replicaTm().getEntityManager().getCriteriaBuilder();
if (!shouldIncludeDeleted()) {
queryBuilder =
queryBuilder.where(
"deletionTime", criteriaBuilder::greaterThan, getRequestTime());
}
});
if (cursorString.isPresent()) {
queryBuilder =
queryBuilder.where(
"domainName", criteriaBuilder::greaterThan, cursorString.get());
}
replicaTm()
.criteriaQuery(queryBuilder.build())
.getResultStream()
.filter(this::isAuthorized)
.forEach(
(domain) -> {
Hibernate.initialize(domain.getDsData());
domainSetBuilder.add(domain);
});
}
});
}
List<Domain> domains = domainSetBuilder.build().asList();
metricInformationBuilder.setNumHostsRetrieved(numHostKeysSearched);
@@ -14,7 +14,7 @@
package google.registry.rdap;
import static google.registry.persistence.transaction.TransactionManagerFactory.replicaJpaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.replicaTm;
import static google.registry.rdap.RdapUtils.getRegistrarByIanaIdentifier;
import static google.registry.rdap.RdapUtils.getRegistrarByName;
import static google.registry.request.Action.Method.GET;
@@ -71,7 +71,7 @@ public class RdapEntityAction extends RdapActionBase {
if (ROID_PATTERN.matcher(pathSearchString).matches()) {
VKey<Contact> contactVKey = VKey.create(Contact.class, pathSearchString);
Optional<Contact> contact =
replicaJpaTm().transact(() -> replicaJpaTm().loadByKeyIfPresent(contactVKey));
replicaTm().transact(() -> replicaTm().loadByKeyIfPresent(contactVKey));
// As per Andy Newton on the regext mailing list, contacts by themselves have no role, since
// they are global, and might have different roles for different domains.
if (contact.isPresent() && isAuthorized(contact.get())) {

Some files were not shown because too many files have changed in this diff Show More