1
0
mirror of https://github.com/google/nomulus synced 2026-06-09 16:33:02 +00:00

Compare commits

...

31 Commits

Author SHA1 Message Date
Pavlo Tkach 19e03dbd2e Update proxy and nomulus cluster resources (#2858) 2025-10-24 20:19:41 +00:00
Pavlo Tkach fc1eb162f2 Remove Primary Contact from users editing screen (#2856) 2025-10-24 20:12:18 +00:00
gbrodman ed25854fbc Add unique index for not-deleted domain names (#2853)
This is a backstop against multiple domain creations for the same domain
name getting through
2025-10-24 15:38:23 +00:00
Juan Celhay 0aa6bc6aaa Change regex format in release cb file (#2854) 2025-10-23 19:49:29 +00:00
Juan Celhay ff4c326ebe Delete step to push to release repo, trigger next release steps based on tag format (#2833)
* Change release cb file

* Add brackets around tag variable

* Redo tag matching

* Have tag matcher like the one in cb dev
2025-10-21 18:52:42 +00:00
Pavlo Tkach 51b579871a Anonymize support users in console history, add minor UI updates (#2851) 2025-10-17 18:57:40 +00:00
gbrodman b144aafb22 Use transaction time for deletion time cache ticker (#2848)
Basically, what happened is that the cache's expireAfterWrite was being
called some number of milliseconds (say, 50-100) after the transaction
was started. That method used the transaction time instead of the
current time, so as a result the entries were sticking around 50-100ms
longer in the cache than they should have been.

This fix contains two parts, each of which I believe would be sufficient
on their own to fix the issue:
1. Use the currentTime passed in in Expiry::expireAfterCreate
2. Use the transaction time in the cache's Ticker. This keeps everything
   on the same schedule.
2025-10-16 20:01:17 +00:00
Weimin Yu ddd955e156 Fix dependency of Gradle task for schema test (#2849)
Problem not showing up because all use cases run this test after
`build`.
2025-10-16 15:33:28 +00:00
gbrodman 6863f678f1 Allow Gradle to use more heap space (#2847)
During the release process, we are seeing the message "Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)" which seemingly can be caused by OOMs
2025-10-13 18:25:08 +00:00
gbrodman 6bd90e967b Add more hash indexes used during common flows (#2845)
I analyzed SQL statements run during the following flows and EXPLAIN
ANALYZEd each of them to figure out if there are any additional hash
indexes we could add that could be particularly helpful. Note: it's not
worth adding a hash index on the host_repo_id field in DomainHost
because so many rows (domains) use the same host.

- domain create
- domain delete
- domain info
- domain renew
- domain update
- host create
- host delete
- host update

I skipped the ones that use the read-only replica, as well as contact
flows (we're getting rid of them), and domain transfer/restore-related
flows as those are extremely infrequent.
2025-10-13 18:07:47 +00:00
gbrodman 5faf3d283c Differentiate between inserts and updates in flows (#2846)
Updates (AKA merges) run an extra SELECT statement to figure out if the
resource exists so that it can merge the entity into the existing object
in Hibernate's schema. When we're inserting new rows (such as new poll
messages or resource creates), we know that we don't need to do that
merge. Doing this should save us some SELECT statements (this has borne
out to be the truth in alpha)
2025-10-13 15:43:18 +00:00
gbrodman 149fb66ac5 Add cache for deletion times of existing domains (#2840)
This should help in instances of popular domains dropping, since we
won't need to do an additional two database loads every time (assuming
the deletion time is in the future).
2025-10-09 17:22:24 +00:00
gbrodman 8c96940a27 Only load from ClaimsList once when filling the cache (#2843) 2025-10-09 16:57:21 +00:00
Ben McIlwain 9c5510f05d Add a rate limiter to remove all domain contacts action (#2838)
The maximum QPS defaults to 10, but can also be specified at runtime through
use of a query-string parameter.

BUG = http://b/439636188
2025-10-02 22:15:19 +00:00
gbrodman 84884de77b Verify existence of TLDs and registrars for tokens (#2837)
Just in case someone makes a typo when running the commands
2025-10-02 20:10:58 +00:00
Ben McIlwain d6c35df9bc Ignore single domain failures in remove contacts from all domains action (#2836)
When running the action in sandbox on 1.5M domains, it failed a few times
updating individual domains (requiring a manual restart of the entire action).
It's better to just log the individual failures for manual inspection and then
otherwise continue running the action to process the vast majority of other
updates that won't fail.

BUG = http://b/439636188
2025-10-02 18:58:23 +00:00
Juan Celhay 7caa0ec9d6 Add environment configuration files to .gitignore (#2830)
* Add environment configuration files to .gitignore

* Delete config files from repo

* Refactor release cb file to delete config file lines from gitignore

* Reorder env files

* Add README for config files
2025-10-02 18:36:43 +00:00
Weimin Yu ee3866ec4a Allow top level tld creation in Sandbox (#2835)
Add a flag to the CreateCdnsTld command to bypass the dns name format
check in Sandbox (limiting names to `*.test.`). With this flag, we
can create TLDs for RST testing in Sandbox.

Note that if the new flag is wrongly set for a disallowed name, the
request to the Cloud DNS API will fail. The format check in the command
just provides a user-friendly error message.
2025-10-01 14:20:33 +00:00
gbrodman 97d0b7680f Add hash indexes for common use cases (#2834)
I went through all the SQL statements generated by some sample
DomainCreateFlow and DomainDeleteFlow cases to find situations where we
were either SELECTing from, or UPDATEing, tables with a direct "field =
value" format. These are the situations that I found where we can add
hash indexes. This does two things:

1. Makes these queries slight faster, since these are usually queries on
   columns that are either unique or very close to unique, and O(1) is
   faster than O(log(n))
2. Spreads around the optimistic predicate locks on the previously-used
   btree indexes. Many of our serialization errors came from the fact
   that we were autogenerating incrementing ID values for various
   tables, meaning that SELECTs, INSERTs, and UPDATEs would all try to
   take predicate locks out on the same page of the btree index. Using a
   hash index means that the page locks will be spread out to various
   index pages, rather than conflicting with each other.

Running load tests on alpha I see significant improvements in speed and
error rates. Speed is hard to quantify due to the nature of the way the
load tests distribute tasks among the queues but it could be more than
50% improvement, and serialization errors in the logs drop by more than
90%.
2025-09-29 22:16:24 +00:00
Pavlo Tkach 5700a008d6 Add console history frontend (#2832) 2025-09-26 21:25:03 +00:00
Ben McIlwain dc9f5b99bc Add a batch action to remove all contacts from domains (#2827)
This implements the first part of Minimum Data Set phase 3, wherein we delete
all contact data. This action is necessary to leave a permanent record on the
domain (in the form of a domain history entry) documenting when the contacts
were removed by the administrative user.

Then, after this has finished removing all contact assocations, we can simply
empty out or drop the Contact/ContactHistory tables and associated join tables.
2025-09-25 20:47:17 +00:00
Ben McIlwain d3c6de7a38 Modify the base Latin LGR with our intended changes to improve security (#2829) 2025-09-24 21:04:37 +00:00
Ben McIlwain 3c3303c16a Add ICANN's reference Latin LGR in RFC 7940 XML format (#2828)
In the next commit I will make changes to this file so it supports just the
basic Latin characters that we want, but it's good to check the base version in
so that we can see diffs.

This was downloaded from https://www.icann.org/sites/default/files/packages/lgr/lgr-second-level-latin-script-25oct24-en.xml
2025-09-19 16:42:46 +00:00
Nilay Shah 2a86a1bbe9 Skip user loading for proxy service account (#2825)
* Skip user loading for proxy service account

Reduces database load by skipping the User entity lookup for the proxy
service account during OIDC authentication.

The high volume of EPP "hello" and "login" commands from the proxy
service account results in a constant database load. These lookups
are unnecessary as the proxy service account is not expected to have a
corresponding User object.

This change optimizes the authentication flow by checking for the proxy
service account email *before* attempting to load a User from the
database. This bypasses the database transaction entirely for these
high-volume requests.

This approach is more efficient than caching, as it eliminates the
database lookup for the proxy service account altogether, rather than
just caching the result.

* comment added and service account llokup time improved

* comment updated for more clarity
2025-09-16 18:48:39 +00:00
gbrodman ea148ac13e Show success message on password reset (#2826) 2025-09-16 18:39:19 +00:00
Nilay Shah 06299ccb86 Add cache for User entities in OIDC auth flow (#2822)
* Add cache for User entities in OIDC auth flow

* refactor: Address review feedback

- Refactor database call into a single, reusable method
- Increase the default cache size to 200
- Remove .recordStats() and using spy for testing
- Split unit tests into separate implementation test that use Mockito spies instead of checking internal cache stats
2025-09-12 07:43:32 +00:00
gbrodman 732c30b359 Remove registry-lock-related fields from RegistrarPoc (#2818)
We've moved these over to the User class, so we should remove these for
clarity. In addition, we should make it clear (in Java at least) that
the field in the RegistryLock object refers to the email address used
for the lock in question.
2025-09-11 15:29:06 +00:00
gbrodman ee5a2d3916 Include internal registrars in the console (#2821)
This allows us to also check / modify the CharlestonRoad registrar in
the console, and also allows us to test actions (like password reset)
using that registrar in the prod environment.
2025-09-05 20:37:23 +00:00
gbrodman 2b5643df4c Sort registrars list in console (#2820)
This was bugging me slightly
2025-09-05 18:44:17 +00:00
Pavlo Tkach 6bbd7a2290 Update proxy resources, increase ssl handshake timeout (#2819) 2025-09-05 18:09:55 +00:00
Weimin Yu 77ab80f3dc Fix OOM in UploadBsaUnavailableDomains action (#2817)
* Fix OOM in UploadBsaUnavailableDomains action

The action was using string concatenation to generate the upload content.
This causes an OOM when string length exceeds 25MB on our current VM.

This PR witches to streaming upload.

Also added an HTTP upload test.

* Fix OOM in UploadBsaUnavailableDomains action

The action was using string concatenation to generate the upload content.
This causes an OOM when string length exceeds 25MB on our current VM.

This PR witches to streaming upload.

Also added an HTTP upload test.
2025-09-03 18:25:56 +00:00
139 changed files with 4148 additions and 6153 deletions
+7
View File
@@ -18,6 +18,13 @@ gjf.out
# virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml
hs_err_pid*
# Environment-specific configuration files
core/src/main/java/google/registry/config/files/nomulus-config-alpha.yaml
core/src/main/java/google/registry/config/files/nomulus-config-crash.yaml
core/src/main/java/google/registry/config/files/nomulus-config-production.yaml
core/src/main/java/google/registry/config/files/nomulus-config-qa.yaml
core/src/main/java/google/registry/config/files/nomulus-config-sandbox.yaml
######################################################################
# Eclipse Ignores
+1 -1
View File
@@ -56,7 +56,7 @@ PROPERTIES_HEADER = """\
# nom_build), run ./nom_build --help.
#
# DO NOT EDIT THIS FILE BY HAND
org.gradle.jvmargs=-Xmx1024m
org.gradle.jvmargs=-Xmx2048m
org.gradle.caching=true
org.gradle.parallel=true
"""
+1 -1
View File
@@ -1,7 +1,7 @@
{
"/console-api":
{
"target": "http://localhost:8080",
"target": "http://[::1]:8080",
"secure": false,
"logLevel": "debug",
"changeOrigin": true
+7 -1
View File
@@ -26,6 +26,7 @@ import SecurityComponent from './settings/security/security.component';
import { SettingsComponent } from './settings/settings.component';
import { SupportComponent } from './support/support.component';
import RdapComponent from './settings/rdap/rdap.component';
import { HistoryComponent } from './history/history.component';
import { PasswordResetVerifyComponent } from './shared/components/passwordReset/passwordResetVerify.component';
export interface RouteWithIcon extends Route {
@@ -64,13 +65,18 @@ export const routes: RouteWithIcon[] = [
title: 'Dashboard',
iconName: 'view_comfy_alt',
},
// { path: 'tlds', component: TldsComponent, title: "TLDs", iconName: "event_list" },
{
path: DomainListComponent.PATH,
component: DomainListComponent,
title: 'Domains',
iconName: 'view_list',
},
{
path: HistoryComponent.PATH,
component: HistoryComponent,
// title: 'History',
// iconName: 'history',
},
{
path: SettingsComponent.PATH,
component: SettingsComponent,
+4 -2
View File
@@ -56,13 +56,14 @@ import { GlobalLoaderService } from './shared/services/globalLoader.service';
import { UserDataService } from './shared/services/userData.service';
import { SnackBarModule } from './snackbar.module';
import { SupportComponent } from './support/support.component';
import { TldsComponent } from './tlds/tlds.component';
import { ForceFocusDirective } from './shared/directives/forceFocus.directive';
import RdapComponent from './settings/rdap/rdap.component';
import RdapEditComponent from './settings/rdap/rdapEdit.component';
import { PocReminderComponent } from './shared/components/pocReminder/pocReminder.component';
import { PasswordResetVerifyComponent } from './shared/components/passwordReset/passwordResetVerify.component';
import { PasswordInputForm } from './shared/components/passwordReset/passwordInputForm.component';
import { HistoryComponent } from './history/history.component';
import { HistoryListComponent } from './history/historyList.component';
@NgModule({
declarations: [SelectedRegistrarWrapper],
@@ -81,6 +82,8 @@ export class SelectedRegistrarModule {}
EppPasswordEditComponent,
ForceFocusDirective,
HeaderComponent,
HistoryComponent,
HistoryListComponent,
HomeComponent,
LocationBackDirective,
NavigationComponent,
@@ -104,7 +107,6 @@ export class SelectedRegistrarModule {}
SettingsComponent,
SettingsContactComponent,
SupportComponent,
TldsComponent,
UserLevelVisibility,
],
bootstrap: [AppComponent],
@@ -0,0 +1,62 @@
<app-selected-registrar-wrapper>
<div class="history-log">
<h1 class="mat-headline-4" forceFocus>
Registrar Console Activity History
</h1>
<mat-tab-group
[elementId]="getElementIdForUserLog()"
class="history-log__tabs"
>
<mat-tab label="Registrar Activity">
<div class="spacer"></div>
<app-history-list
[historyRecords]="historyService.historyRecordsRegistrar()"
[isLoading]="isLoading"
/>
</mat-tab>
<mat-tab label="User Activity">
<div class="spacer"></div>
<form (ngSubmit)="loadHistory()" #form="ngForm">
<section>
<mat-form-field appearance="outline">
<mat-label>Console User Email: </mat-label>
<input
matInput
id="email"
type="email"
name="consoleUserEmail"
required
email
[(ngModel)]="consoleUserEmail"
#emailControl="ngModel"
/>
</mat-form-field>
</section>
<div class="spacer"></div>
<button
mat-flat-button
color="primary"
type="submit"
aria-label="Search user history"
[disabled]="!form.valid"
>
Search
</button>
</form>
<div class="spacer"></div>
<app-history-list
[historyRecords]="historyService.historyRecordsUser()"
[isLoading]="isLoading"
/>
</mat-tab>
</mat-tab-group>
</div>
<app-history-list
[elementId]="getElementIdForUserLog()"
[isReverse]="true"
[historyRecords]="historyService.historyRecordsUser()"
[isLoading]="isLoading"
/>
</app-selected-registrar-wrapper>
@@ -1,4 +1,4 @@
// Copyright 2024 The Nomulus Authors. All Rights Reserved.
// Copyright 2025 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
@@ -12,17 +12,11 @@
// See the License for the specific language governing permissions and
// limitations under the License.
.console-tlds {
&__cards {
display: flex;
border-top: 1px solid #ddd;
padding: 1rem;
}
&__card {
max-width: 300px;
}
&__card-links {
display: flex;
flex-direction: column;
.history-log {
font-family: "Roboto", sans-serif;
max-width: 760px;
.spacer {
margin: 20px 0;
}
}
@@ -0,0 +1,80 @@
// Copyright 2025 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { Component, effect } from '@angular/core';
import { UserDataService } from '../shared/services/userData.service';
import { BackendService } from '../shared/services/backend.service';
import { RegistrarService } from '../registrar/registrar.service';
import { HistoryService } from './history.service';
import { MatSnackBar } from '@angular/material/snack-bar';
import {
GlobalLoader,
GlobalLoaderService,
} from '../shared/services/globalLoader.service';
import { HttpErrorResponse } from '@angular/common/http';
import { RESTRICTED_ELEMENTS } from '../shared/directives/userLevelVisiblity.directive';
@Component({
selector: 'app-history',
templateUrl: './history.component.html',
styleUrls: ['./history.component.scss'],
providers: [HistoryService],
standalone: false,
})
export class HistoryComponent implements GlobalLoader {
public static PATH = 'history';
consoleUserEmail: string = '';
isLoading: boolean = false;
constructor(
private backendService: BackendService,
private registrarService: RegistrarService,
protected historyService: HistoryService,
protected globalLoader: GlobalLoaderService,
protected userDataService: UserDataService,
private _snackBar: MatSnackBar
) {
effect(() => {
if (registrarService.registrarId()) {
this.loadHistory();
}
});
}
getElementIdForUserLog() {
return RESTRICTED_ELEMENTS.ACTIVITY_PER_USER;
}
loadingTimeout() {
this._snackBar.open('Timeout loading records history');
}
loadHistory() {
this.globalLoader.startGlobalLoader(this);
this.isLoading = true;
this.historyService
.getHistoryLog(this.registrarService.registrarId(), this.consoleUserEmail)
.subscribe({
error: (err: HttpErrorResponse) => {
this._snackBar.open(err.error || err.message);
this.isLoading = false;
},
next: () => {
this.globalLoader.stopGlobalLoader(this);
this.isLoading = false;
},
});
}
}
@@ -0,0 +1,46 @@
// Copyright 2025 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { Injectable, signal } from '@angular/core';
import { BackendService } from '../shared/services/backend.service';
import { tap } from 'rxjs';
export interface HistoryRecord {
modificationTime: string;
type: string;
description: string;
actingUser: {
emailAddress: string;
};
}
@Injectable()
export class HistoryService {
historyRecordsRegistrar = signal<HistoryRecord[]>([]);
historyRecordsUser = signal<HistoryRecord[]>([]);
constructor(private backendService: BackendService) {}
getHistoryLog(registrarId: string, userEmail?: string) {
return this.backendService.getHistoryLog(registrarId, userEmail).pipe(
tap((historyRecords: HistoryRecord[]) => {
if (userEmail) {
this.historyRecordsUser.set(historyRecords);
} else {
this.historyRecordsRegistrar.set(historyRecords);
}
})
);
}
}
@@ -0,0 +1,50 @@
@if (!isLoading && historyRecords.length == 0) {
<div class="history-list__no-records">
<mat-icon class="history-list__no-records-icon secondary-text"
>apps_outage</mat-icon
>
<h1>No records found</h1>
</div>
} @else {
<mat-card>
<mat-card-content>
<mat-list role="list">
<ng-container *ngFor="let item of historyRecords; let last = last">
<mat-list-item class="history-list__item">
<mat-icon
[ngClass]="getIconClass(item.type)"
class="history-list__icon"
>
{{ getIconForType(item.type) }}
</mat-icon>
<div class="history-list__content">
<div class="history-list__description">
<span class="history-list__description--main">{{
item.type
}}</span>
<div>
<mat-chip
*ngIf="parseDescription(item.description).detail"
class="history-list__chip"
>
{{ parseDescription(item.description).detail }}
</mat-chip>
</div>
</div>
<div class="history-list__user">
<b>User - {{ item.actingUser.emailAddress }}</b>
</div>
</div>
<span class="history-list__timestamp">
{{ item.modificationTime | date : "MMM d, y, h:mm a" }}
</span>
</mat-list-item>
<mat-divider *ngIf="!last"></mat-divider>
</ng-container>
</mat-list>
</mat-card-content>
</mat-card>
}
@@ -0,0 +1,81 @@
// Copyright 2025 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
.history-list {
font-family: "Roboto", sans-serif;
&__item {
display: flex;
align-items: center;
// Override default mat-list-item height to fit content
height: auto !important;
padding: 16px 0;
}
&__no-records {
display: flex;
justify-content: center;
align-items: center;
flex-direction: column;
}
&__no-records-icon {
width: 4rem;
height: 4rem;
font-size: 4rem;
margin-top: 1.5rem;
}
&__icon {
margin-right: 16px;
&--update {
color: #1976d2;
}
&--security {
color: #d32f2f;
}
}
&__description {
&--main {
font-size: 1rem;
font-weight: 500;
color: rgba(0, 0, 0, 0.87);
margin-bottom: 1em;
}
}
&__content {
flex-grow: 1;
display: flex;
flex-direction: column;
gap: 4px;
margin-right: 16px;
}
&__chip {
margin: 0.5rem 0;
}
&__user {
font-size: 0.9rem;
color: rgba(0, 0, 0, 0.6);
}
&__timestamp {
color: rgba(0, 0, 0, 0.6);
white-space: nowrap;
text-align: right;
}
}
@@ -0,0 +1,66 @@
// Copyright 2025 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { ChangeDetectionStrategy, Component, Input } from '@angular/core';
import { HistoryRecord } from './history.service';
@Component({
selector: 'app-history-list',
templateUrl: './historyList.component.html',
styleUrls: ['./historyList.component.scss'],
changeDetection: ChangeDetectionStrategy.OnPush,
standalone: false,
})
export class HistoryListComponent {
@Input() historyRecords: HistoryRecord[] = [];
@Input() isLoading: boolean = false;
getIconForType(type: string): string {
switch (type) {
case 'REGISTRAR_UPDATE':
return 'edit';
case 'REGISTRAR_SECURITY_UPDATE':
return 'security';
default:
return 'history'; // A fallback icon
}
}
getIconClass(type: string): string {
switch (type) {
case 'REGISTRAR_UPDATE':
return 'history-log__icon--update';
case 'REGISTRAR_SECURITY_UPDATE':
return 'history-log__icon--security';
default:
return '';
}
}
parseDescription(description: string): {
main: string;
detail: string | null;
} {
if (!description) {
return { main: 'N/A', detail: null };
}
const parts = description.split('|');
const detail = parts.length > 1 ? parts[1].replace(/_/g, ' ') : parts[0];
return {
main: parts[0],
detail: detail,
};
}
}
@@ -25,7 +25,10 @@ export class RegistrarSelectorComponent {
registrarInput = signal<string>(this.registrarService.registrarId());
filteredOptions?: string[];
allRegistrarIds = computed(() =>
this.registrarService.registrars().map((r) => r.registrarId)
this.registrarService
.registrars()
.map((r) => r.registrarId)
.sort()
);
constructor(protected registrarService: RegistrarService) {
@@ -57,6 +57,11 @@
[(ngModel)]="contactService.contactInEdit.emailAddress"
[ngModelOptions]="{ standalone: true }"
[disabled]="emailAddressIsDisabled()"
[matTooltip]="
emailAddressIsDisabled()
? 'Reach out to registry customer support to update email address'
: ''
"
/>
</mat-form-field>
@@ -84,6 +89,7 @@
<h1>Contact Type</h1>
<p class="console-app__contact-required">
<mat-icon color="accent">error</mat-icon>Required to select at least one
(primary contact can't be updated)
</p>
<div class="">
<ng-container
@@ -24,6 +24,7 @@ import {
PasswordResults,
} from './passwordInputForm.component';
import EppPasswordEditComponent from 'src/app/settings/security/eppPasswordEdit.component';
import { MatSnackBar } from '@angular/material/snack-bar';
export interface PasswordResetVerifyResponse {
registrarId: string;
@@ -54,7 +55,8 @@ export class PasswordResetVerifyComponent {
protected backendService: BackendService,
protected registrarService: RegistrarService,
private route: ActivatedRoute,
private router: Router
private router: Router,
private _snackBar: MatSnackBar
) {}
ngOnInit() {
@@ -99,7 +101,10 @@ export class PasswordResetVerifyComponent {
this.isLoading = false;
this.errorMessage = err.error;
},
next: (_) => this.router.navigate(['']),
next: (_) => {
this.router.navigate(['']);
this._snackBar.open('Password reset completed successfully');
},
});
}
}
@@ -16,6 +16,7 @@ import { Directive, ElementRef, Input, effect } from '@angular/core';
import { UserDataService } from '../services/userData.service';
export enum RESTRICTED_ELEMENTS {
ACTIVITY_PER_USER,
REGISTRAR_ELEMENT,
OTE,
USERS,
@@ -28,9 +29,10 @@ export const DISABLED_ELEMENTS_PER_ROLE = {
RESTRICTED_ELEMENTS.REGISTRAR_ELEMENT,
RESTRICTED_ELEMENTS.OTE,
RESTRICTED_ELEMENTS.SUSPEND,
RESTRICTED_ELEMENTS.ACTIVITY_PER_USER,
],
SUPPORT_LEAD: [],
SUPPORT_AGENT: [],
SUPPORT_AGENT: [RESTRICTED_ELEMENTS.ACTIVITY_PER_USER],
};
@Directive({
@@ -40,6 +42,8 @@ export const DISABLED_ELEMENTS_PER_ROLE = {
export class UserLevelVisibility {
@Input() elementId!: RESTRICTED_ELEMENTS | null;
@Input() isReverse: boolean = false;
constructor(
private userDataService: UserDataService,
private el: ElementRef
@@ -56,9 +60,9 @@ export class UserLevelVisibility {
// @ts-ignore
(DISABLED_ELEMENTS_PER_ROLE[globalRole] || []).includes(this.elementId)
) {
this.el.nativeElement.style.display = 'none';
this.el.nativeElement.style.display = this.isReverse ? '' : 'none';
} else {
this.el.nativeElement.style.display = '';
this.el.nativeElement.style.display = this.isReverse ? 'none' : '';
}
}
}
@@ -31,6 +31,7 @@ import { Contact } from '../../settings/contact/contact.service';
import { EppPasswordBackendModel } from '../../settings/security/security.service';
import { UserData } from './userData.service';
import { PasswordResetVerifyResponse } from '../components/passwordReset/passwordResetVerify.component';
import { HistoryRecord } from '../../history/history.service';
@Injectable()
export class BackendService {
@@ -123,6 +124,16 @@ export class BackendService {
.pipe(catchError((err) => this.errorCatcher<DomainListResult>(err)));
}
getHistoryLog(registrarId: string, userEmail?: string) {
return this.http
.get<HistoryRecord[]>(
userEmail
? `/console-api/history?registrarId=${registrarId}&consoleUserEmail=${userEmail}`
: `/console-api/history?registrarId=${registrarId}`
)
.pipe(catchError((err) => this.errorCatcher<HistoryRecord[]>(err)));
}
getRegistrars(): Observable<Registrar[]> {
return this.http
.get<Registrar[]>('/console-api/registrars')
@@ -1 +0,0 @@
<div class="console-tlds__cards"></div>
@@ -1,38 +0,0 @@
// Copyright 2024 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { ComponentFixture, TestBed } from '@angular/core/testing';
import { TldsComponent } from './tlds.component';
import { MaterialModule } from '../material.module';
describe('TldsComponent', () => {
let component: TldsComponent;
let fixture: ComponentFixture<TldsComponent>;
beforeEach(async () => {
await TestBed.configureTestingModule({
imports: [MaterialModule],
declarations: [TldsComponent],
}).compileComponents();
fixture = TestBed.createComponent(TldsComponent);
component = fixture.componentInstance;
fixture.detectChanges();
});
it('should create', () => {
expect(component).toBeTruthy();
});
});
@@ -29,7 +29,7 @@
></mat-label
>
<mat-select [(ngModel)]="user().role" name="userRole">
<mat-option value="PRIMARY_CONTACT">Editor</mat-option>
<mat-option value="TECH_CONTACT">Editor</mat-option>
<mat-option value="ACCOUNT_MANAGER">Viewer</mat-option>
</mat-select>
</mat-form-field>
+2
View File
@@ -58,6 +58,8 @@ def fragileTestPatterns = [
// Changes cache timeouts and for some reason appears to have contention
// with other tests.
"google/registry/whois/WhoisCommandFactoryTest.*",
// Breaks random other tests when running with standardTests.
"google/registry/bsa/UploadBsaUnavailableDomainsActionTest.*",
// Currently changes a global configuration parameter that for some reason
// results in timestamp inversions for other tests. TODO(mmuller): fix.
"google/registry/flows/host/HostInfoFlowTest.*",
@@ -29,9 +29,11 @@ import static google.registry.request.RequestParameters.extractRequiredParameter
import static google.registry.request.RequestParameters.extractSetOfDatetimeParameters;
import com.google.common.collect.ImmutableSet;
import com.google.common.util.concurrent.RateLimiter;
import dagger.Module;
import dagger.Provides;
import google.registry.request.Parameter;
import jakarta.inject.Named;
import jakarta.servlet.http.HttpServletRequest;
import java.util.Optional;
import org.joda.time.DateTime;
@@ -137,4 +139,18 @@ public class BatchModule {
static boolean provideIsFast(HttpServletRequest req) {
return extractBooleanParameter(req, PARAM_FAST);
}
private static final int DEFAULT_MAX_QPS = 10;
@Provides
@Parameter("maxQps")
static int provideMaxQps(HttpServletRequest req) {
return extractOptionalIntParameter(req, "maxQps").orElse(DEFAULT_MAX_QPS);
}
@Provides
@Named("removeAllDomainContacts")
static RateLimiter provideRemoveAllDomainContactsRateLimiter(@Parameter("maxQps") int maxQps) {
return RateLimiter.create(maxQps);
}
}
@@ -15,7 +15,6 @@
package google.registry.batch;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.request.Action.Method.POST;
import static google.registry.tools.LockOrUnlockDomainCommand.REGISTRY_LOCK_STATUSES;
@@ -30,8 +29,6 @@ import google.registry.groups.GmailClient;
import google.registry.model.domain.Domain;
import google.registry.model.domain.RegistryLock;
import google.registry.model.eppcommon.StatusValue;
import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.RegistrarPoc;
import google.registry.model.tld.RegistryLockDao;
import google.registry.persistence.VKey;
import google.registry.request.Action;
@@ -70,12 +67,14 @@ public class RelockDomainAction implements Runnable {
"""
The domain %s was successfully re-locked.
Please contact support at %s if you have any questions.""";
Please contact support at %s if you have any questions.\
""";
private static final String RELOCK_NON_RETRYABLE_FAILURE_EMAIL_TEMPLATE =
"""
There was an error when automatically re-locking %s. Error message: %s
Please contact support at %s if you have any questions.""";
Please contact support at %s if you have any questions.\
""";
private static final String RELOCK_TRANSIENT_FAILURE_EMAIL_TEMPLATE =
"There was an unexpected error when automatically re-locking %s. We will continue retrying "
+ "the lock for five hours. Please contact support at %s if you have any questions";
@@ -171,7 +170,7 @@ public class RelockDomainAction implements Runnable {
domainLockUtils.administrativelyApplyLock(
oldLock.getDomainName(),
oldLock.getRegistrarId(),
oldLock.getRegistrarPocId(),
oldLock.getRegistryLockEmail(),
oldLock.isSuperuser());
logger.atInfo().log("Re-locked domain %s.", oldLock.getDomainName());
response.setStatus(SC_OK);
@@ -221,7 +220,7 @@ public class RelockDomainAction implements Runnable {
EmailMessage.newBuilder()
.setBody(body)
.setSubject(String.format("Error re-locking domain %s", oldLock.getDomainName()))
.setRecipients(getEmailRecipients(oldLock.getRegistrarId()))
.setRecipients(ImmutableSet.of(getEmailRecipient(oldLock)))
.build());
}
@@ -250,7 +249,7 @@ public class RelockDomainAction implements Runnable {
EmailMessage.newBuilder()
.setBody(body)
.setSubject(String.format("Successful re-lock of domain %s", oldLock.getDomainName()))
.setRecipients(getEmailRecipients(oldLock.getRegistrarId()))
.setRecipients(ImmutableSet.of(getEmailRecipient(oldLock)))
.build());
}
@@ -261,7 +260,7 @@ public class RelockDomainAction implements Runnable {
// For an unexpected failure, notify both the lock-enabled contacts and our alerting email
ImmutableSet<InternetAddress> allRecipients =
new ImmutableSet.Builder<InternetAddress>()
.addAll(getEmailRecipients(oldLock.getRegistrarId()))
.add(getEmailRecipient(oldLock))
.add(alertRecipientAddress)
.build();
gmailClient.sendEmail(
@@ -281,31 +280,12 @@ public class RelockDomainAction implements Runnable {
.build());
}
private ImmutableSet<InternetAddress> getEmailRecipients(String registrarId) {
Registrar registrar =
Registrar.loadByRegistrarIdCached(registrarId)
.orElseThrow(
() ->
new IllegalStateException(String.format("Unknown registrar %s", registrarId)));
ImmutableSet<String> registryLockEmailAddresses =
registrar.getContacts().stream()
.filter(RegistrarPoc::isRegistryLockAllowed)
.map(RegistrarPoc::getRegistryLockEmailAddress)
.filter(Optional::isPresent)
.map(Optional::get)
.collect(toImmutableSet());
ImmutableSet.Builder<InternetAddress> builder = new ImmutableSet.Builder<>();
// can't use streams due to the 'throws' in the InternetAddress constructor
for (String registryLockEmailAddress : registryLockEmailAddresses) {
try {
builder.add(new InternetAddress(registryLockEmailAddress));
} catch (AddressException e) {
// This shouldn't stop any other emails going out, so swallow it
logger.atWarning().log("Invalid email address '%s'.", registryLockEmailAddress);
}
private InternetAddress getEmailRecipient(RegistryLock lock) {
try {
return new InternetAddress(lock.getRegistryLockEmail());
} catch (AddressException e) {
// this really shouldn't happen
throw new RuntimeException(e);
}
return builder.build();
}
}
@@ -0,0 +1,246 @@
// Copyright 2025 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.batch;
import static com.google.common.base.Preconditions.checkState;
import static com.google.common.net.MediaType.PLAIN_TEXT_UTF_8;
import static google.registry.flows.FlowUtils.marshalWithLenientRetry;
import static google.registry.model.common.FeatureFlag.FeatureName.MINIMUM_DATASET_CONTACTS_PROHIBITED;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import static google.registry.util.ResourceUtils.readResourceUtf8;
import static jakarta.servlet.http.HttpServletResponse.SC_INTERNAL_SERVER_ERROR;
import static jakarta.servlet.http.HttpServletResponse.SC_NO_CONTENT;
import static jakarta.servlet.http.HttpServletResponse.SC_OK;
import static java.nio.charset.StandardCharsets.US_ASCII;
import com.google.common.base.Ascii;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.google.common.flogger.FluentLogger;
import com.google.common.util.concurrent.RateLimiter;
import google.registry.config.RegistryConfig.Config;
import google.registry.flows.EppController;
import google.registry.flows.EppRequestSource;
import google.registry.flows.PasswordOnlyTransportCredentials;
import google.registry.flows.StatelessRequestSessionMetadata;
import google.registry.model.common.FeatureFlag;
import google.registry.model.contact.Contact;
import google.registry.model.domain.DesignatedContact;
import google.registry.model.domain.Domain;
import google.registry.model.eppcommon.ProtocolDefinition;
import google.registry.model.eppoutput.EppOutput;
import google.registry.persistence.VKey;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.request.lock.LockHandler;
import jakarta.inject.Inject;
import jakarta.inject.Named;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.logging.Level;
import javax.annotation.Nullable;
import org.joda.time.Duration;
/**
* An action that removes all contacts from all active (non-deleted) domains.
*
* <p>This implements part 1 of phase 3 of the Minimum Dataset migration, wherein we remove all uses
* of contact objects in preparation for later removing all contact data from the system.
*
* <p>This runs as a singly threaded, resumable action that loads batches of domains still
* containing contacts, and runs a superuser domain update on each one to remove the contacts,
* leaving behind a record recording that update.
*/
@Action(
service = GaeService.BACKEND,
path = RemoveAllDomainContactsAction.PATH,
method = Action.Method.POST,
auth = Auth.AUTH_ADMIN)
public class RemoveAllDomainContactsAction implements Runnable {
public static final String PATH = "/_dr/task/removeAllDomainContacts";
private static final String LOCK_NAME = "Remove all domain contacts";
private static final String CONTACT_FMT = "<domain:contact type=\"%s\">%s</domain:contact>";
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private final EppController eppController;
private final String registryAdminClientId;
private final LockHandler lockHandler;
private final RateLimiter rateLimiter;
private final Response response;
private final String updateDomainXml;
private int successes = 0;
private int failures = 0;
private static final int BATCH_SIZE = 10000;
@Inject
RemoveAllDomainContactsAction(
EppController eppController,
@Config("registryAdminClientId") String registryAdminClientId,
LockHandler lockHandler,
@Named("removeAllDomainContacts") RateLimiter rateLimiter,
Response response) {
this.eppController = eppController;
this.registryAdminClientId = registryAdminClientId;
this.lockHandler = lockHandler;
this.rateLimiter = rateLimiter;
this.response = response;
this.updateDomainXml =
readResourceUtf8(RemoveAllDomainContactsAction.class, "domain_remove_contacts.xml");
}
@Override
public void run() {
checkState(
tm().transact(() -> FeatureFlag.isActiveNow(MINIMUM_DATASET_CONTACTS_PROHIBITED)),
"Minimum dataset migration must be completed prior to running this action");
response.setContentType(PLAIN_TEXT_UTF_8);
Callable<Void> runner =
() -> {
try {
runLocked();
response.setStatus(SC_OK);
} catch (Exception e) {
logger.atSevere().withCause(e).log("Errored out during execution.");
response.setStatus(SC_INTERNAL_SERVER_ERROR);
response.setPayload(String.format("Errored out with cause: %s", e));
}
return null;
};
if (!lockHandler.executeWithLocks(runner, null, Duration.standardHours(1), LOCK_NAME)) {
// Send a 200-series status code to prevent this conflicting action from retrying.
response.setStatus(SC_NO_CONTENT);
response.setPayload("Could not acquire lock; already running?");
}
}
private void runLocked() {
logger.atInfo().log("Removing contacts on all active domains.");
List<String> domainRepoIdsBatch;
do {
domainRepoIdsBatch =
tm().<List<String>>transact(
() ->
tm().getEntityManager()
.createQuery(
"""
SELECT repoId FROM Domain WHERE deletionTime = :end_of_time AND NOT (
adminContact IS NULL AND billingContact IS NULL
AND registrantContact IS NULL AND techContact IS NULL)
""")
.setParameter("end_of_time", END_OF_TIME)
.setMaxResults(BATCH_SIZE)
.getResultList());
for (String domainRepoId : domainRepoIdsBatch) {
rateLimiter.acquire();
runDomainUpdateFlow(domainRepoId);
}
} while (!domainRepoIdsBatch.isEmpty());
String msg =
String.format(
"Finished; %d domains were successfully updated and %d errored out.",
successes, failures);
logger.at(failures == 0 ? Level.INFO : Level.WARNING).log(msg);
response.setPayload(msg);
}
private void runDomainUpdateFlow(String repoId) {
// Create a new transaction that the flow's execution will be enlisted in that loads the domain
// transactionally. This way we can ensure that nothing else has modified the domain in question
// in the intervening period since the query above found it. If a single domain update fails
// permanently, log it and move on to not block processing all the other domains.
try {
boolean success = tm().transact(() -> runDomainUpdateFlowInner(repoId));
if (success) {
successes++;
} else {
failures++;
}
} catch (Throwable t) {
logger.atWarning().withCause(t).log(
"Failed updating domain with repoId %s; skipping.", repoId);
}
}
/**
* Runs the actual domain update flow and returns whether the contact removals were successful.
*/
private boolean runDomainUpdateFlowInner(String repoId) {
Domain domain = tm().loadByKey(VKey.create(Domain.class, repoId));
if (!domain.getDeletionTime().equals(END_OF_TIME)) {
// Domain has been deleted since the action began running; nothing further to be
// done here.
logger.atInfo().log("Nothing to process for deleted domain '%s'.", domain.getDomainName());
return false;
}
logger.atInfo().log("Attempting to remove contacts on domain '%s'.", domain.getDomainName());
StringBuilder sb = new StringBuilder();
ImmutableMap<VKey<? extends Contact>, Contact> contacts =
tm().loadByKeys(
domain.getContacts().stream()
.map(DesignatedContact::getContactKey)
.collect(ImmutableSet.toImmutableSet()));
// Collect all the (non-registrant) contacts referenced by the domain and compile an EPP XML
// string that removes each one.
for (DesignatedContact designatedContact : domain.getContacts()) {
@Nullable Contact contact = contacts.get(designatedContact.getContactKey());
if (contact == null) {
logger.atWarning().log(
"Domain '%s' referenced contact with repo ID '%s' that couldn't be" + " loaded.",
domain.getDomainName(), designatedContact.getContactKey().getKey());
continue;
}
sb.append(
String.format(
CONTACT_FMT,
Ascii.toLowerCase(designatedContact.getType().name()),
contact.getContactId()))
.append("\n");
}
String compiledXml =
updateDomainXml
.replace("%DOMAIN%", domain.getDomainName())
.replace("%CONTACTS%", sb.toString());
EppOutput output =
eppController.handleEppCommand(
new StatelessRequestSessionMetadata(
registryAdminClientId, ProtocolDefinition.getVisibleServiceExtensionUris()),
new PasswordOnlyTransportCredentials(),
EppRequestSource.BACKEND,
false,
true,
compiledXml.getBytes(US_ASCII));
if (output.isSuccess()) {
logger.atInfo().log(
"Successfully removed contacts from domain '%s'.", domain.getDomainName());
} else {
logger.atWarning().log(
"Failed removing contacts from domain '%s' with error %s.",
domain.getDomainName(), new String(marshalWithLenientRetry(output), US_ASCII));
}
return output.isSuccess();
}
}
@@ -25,16 +25,16 @@ import static google.registry.request.Action.Method.GET;
import static google.registry.request.Action.Method.POST;
import static jakarta.servlet.http.HttpServletResponse.SC_INTERNAL_SERVER_ERROR;
import static java.nio.charset.StandardCharsets.US_ASCII;
import static java.nio.charset.StandardCharsets.UTF_8;
import com.google.cloud.storage.BlobId;
import com.google.common.base.Joiner;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedSet;
import com.google.common.collect.Ordering;
import com.google.common.flogger.FluentLogger;
import com.google.common.hash.Hasher;
import com.google.common.hash.Hashing;
import com.google.common.io.ByteSource;
import google.registry.bsa.api.BsaCredential;
import google.registry.config.RegistryConfig.Config;
import google.registry.gcs.GcsUtils;
@@ -47,10 +47,13 @@ import google.registry.request.auth.Auth;
import google.registry.util.Clock;
import jakarta.inject.Inject;
import jakarta.persistence.TypedQuery;
import java.io.ByteArrayOutputStream;
import java.io.BufferedInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.OutputStreamWriter;
import java.io.PipedInputStream;
import java.io.PipedOutputStream;
import java.io.Writer;
import java.util.Optional;
import java.util.zip.GZIPOutputStream;
@@ -60,14 +63,17 @@ import okhttp3.OkHttpClient;
import okhttp3.Request;
import okhttp3.RequestBody;
import okhttp3.Response;
import okio.BufferedSink;
import org.jetbrains.annotations.NotNull;
import org.jetbrains.annotations.Nullable;
import org.joda.time.DateTime;
/**
* Daily action that uploads unavailable domain names on applicable TLDs to BSA.
*
* <p>The upload is a single zipped text file containing combined details for all BSA-enrolled TLDs.
* The text is a newline-delimited list of punycoded fully qualified domain names, and contains all
* domains on each TLD that are registered and/or reserved.
* The text is a newline-delimited list of punycoded fully qualified domain names with a trailing
* newline at the end, and contains all domains on each TLD that are registered and/or reserved.
*
* <p>The file is also uploaded to GCS to preserve it as a record for ourselves.
*/
@@ -118,7 +124,7 @@ public class UploadBsaUnavailableDomainsAction implements Runnable {
// TODO(mcilwain): Implement a date Cursor, have the cronjob run frequently, and short-circuit
// the run if the daily upload is already completed.
DateTime runTime = clock.nowUtc();
String unavailableDomains = Joiner.on("\n").join(getUnavailableDomains(runTime));
ImmutableSortedSet<String> unavailableDomains = getUnavailableDomains(runTime);
if (unavailableDomains.isEmpty()) {
logger.atWarning().log("No unavailable domains found; terminating.");
emailSender.sendNotification(
@@ -136,12 +142,16 @@ public class UploadBsaUnavailableDomainsAction implements Runnable {
}
/** Uploads the unavailable domains list to GCS in the unavailable domains bucket. */
boolean uploadToGcs(String unavailableDomains, DateTime runTime) {
boolean uploadToGcs(ImmutableSortedSet<String> unavailableDomains, DateTime runTime) {
logger.atInfo().log("Uploading unavailable names file to GCS in bucket %s", gcsBucket);
BlobId blobId = BlobId.of(gcsBucket, createFilename(runTime));
// `gcsUtils.openOutputStream` returns a buffered stream
try (OutputStream gcsOutput = gcsUtils.openOutputStream(blobId);
Writer osWriter = new OutputStreamWriter(gcsOutput, US_ASCII)) {
osWriter.write(unavailableDomains);
for (var domainName : unavailableDomains) {
osWriter.write(domainName);
osWriter.write("\n");
}
return true;
} catch (Exception e) {
logger.atSevere().withCause(e).log(
@@ -150,10 +160,14 @@ public class UploadBsaUnavailableDomainsAction implements Runnable {
}
}
boolean uploadToBsa(String unavailableDomains, DateTime runTime) {
boolean uploadToBsa(ImmutableSortedSet<String> unavailableDomains, DateTime runTime) {
try {
byte[] gzippedContents = gzipUnavailableDomains(unavailableDomains);
String sha512Hash = ByteSource.wrap(gzippedContents).hash(Hashing.sha512()).toString();
Hasher sha512Hasher = Hashing.sha512().newHasher();
unavailableDomains.stream()
.map(name -> name + "\n")
.forEachOrdered(line -> sha512Hasher.putString(line, UTF_8));
String sha512Hash = sha512Hasher.hash().toString();
String filename = createFilename(runTime);
OkHttpClient client = new OkHttpClient().newBuilder().build();
@@ -169,7 +183,9 @@ public class UploadBsaUnavailableDomainsAction implements Runnable {
.addFormDataPart(
"file",
String.format("%s.gz", filename),
RequestBody.create(gzippedContents, MediaType.parse("application/octet-stream")))
new StreamingRequestBody(
gzippedStream(unavailableDomains),
MediaType.parse("application/octet-stream")))
.build();
Request request =
@@ -196,15 +212,6 @@ public class UploadBsaUnavailableDomainsAction implements Runnable {
}
}
private byte[] gzipUnavailableDomains(String unavailableDomains) throws IOException {
try (ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream()) {
try (GZIPOutputStream gzipOutputStream = new GZIPOutputStream(byteArrayOutputStream)) {
gzipOutputStream.write(unavailableDomains.getBytes(US_ASCII));
}
return byteArrayOutputStream.toByteArray();
}
}
private static String createFilename(DateTime runTime) {
return String.format("unavailable_domains_%s.txt", runTime.toString());
}
@@ -280,4 +287,65 @@ public class UploadBsaUnavailableDomainsAction implements Runnable {
private static String toDomain(String domainLabel, Tld tld) {
return String.format("%s.%s", domainLabel, tld.getTldStr());
}
private InputStream gzippedStream(ImmutableSortedSet<String> unavailableDomains)
throws IOException {
PipedInputStream inputStream = new PipedInputStream();
PipedOutputStream outputStream = new PipedOutputStream(inputStream);
new Thread(
() -> {
try {
gzipUnavailableDomains(outputStream, unavailableDomains);
} catch (Throwable e) {
logger.atSevere().withCause(e).log("Failed to gzip unavailable domains.");
try {
// This will cause the next read to throw an IOException.
inputStream.close();
} catch (IOException ignore) {
// Won't happen for `PipedInputStream.close()`
}
}
})
.start();
return inputStream;
}
private void gzipUnavailableDomains(
PipedOutputStream outputStream, ImmutableSortedSet<String> unavailableDomains)
throws IOException {
// `GZIPOutputStream` is buffered.
try (GZIPOutputStream gzipOutputStream = new GZIPOutputStream(outputStream)) {
for (String name : unavailableDomains) {
var line = name + "\n";
gzipOutputStream.write(line.getBytes(US_ASCII));
}
}
}
private static class StreamingRequestBody extends RequestBody {
private final BufferedInputStream inputStream;
private final MediaType mediaType;
StreamingRequestBody(InputStream inputStream, MediaType mediaType) {
this.inputStream = new BufferedInputStream(inputStream);
this.mediaType = mediaType;
}
@Nullable
@Override
public MediaType contentType() {
return mediaType;
}
@Override
public void writeTo(@NotNull BufferedSink bufferedSink) throws IOException {
byte[] buffer = new byte[2048];
int bytesRead;
while ((bytesRead = inputStream.read(buffer)) != -1) {
bufferedSink.write(buffer, 0, bytesRead);
}
}
}
}
@@ -0,0 +1,13 @@
# Nomulus Environment Configuration
The configuration files for the different Nomulus environments are not included in this repository. To configure and run a specific environment, you will need to create the corresponding YAML configuration file in this directory.
The following is a list of the environment configuration files that you may need to create:
* `nomulus-config-alpha.yaml`
* `nomulus-config-crash.yaml`
* `nomulus-config-qa.yaml`
* `nomulus-config-sandbox.yaml`
* `nomulus-config-production.yaml`
Please create the relevant file for the environment you intend to use and populate it with the necessary configuration details.
@@ -1 +0,0 @@
# Add environment-specific configuration here.
@@ -1 +0,0 @@
# Add environment-specific configuration here.
@@ -1 +0,0 @@
# Add environment-specific configuration here.
@@ -1 +0,0 @@
# Add environment-specific configuration here.
@@ -1 +0,0 @@
# Add environment-specific configuration here.
@@ -56,9 +56,10 @@ public final class FlowUtils {
}
}
/** Persists the saves and deletes in an {@link EntityChanges} to the DB. */
/** Persists the inserts, updates, and deletes in an {@link EntityChanges} to the DB. */
public static void persistEntityChanges(EntityChanges entityChanges) {
tm().putAll(entityChanges.getSaves());
tm().insertAll(entityChanges.getInserts());
tm().updateAll(entityChanges.getUpdates());
tm().delete(entityChanges.getDeletes());
}
@@ -19,25 +19,30 @@ import com.google.common.collect.ImmutableSet;
import google.registry.model.ImmutableObject;
import google.registry.persistence.VKey;
/** A record that encapsulates database entities to both save and delete. */
/** A record that encapsulates database entities to insert, update, and delete. */
public record EntityChanges(
ImmutableSet<ImmutableObject> saves, ImmutableSet<VKey<ImmutableObject>> deletes) {
ImmutableSet<ImmutableObject> inserts,
ImmutableSet<ImmutableObject> updates,
ImmutableSet<VKey<ImmutableObject>> deletes) {
public ImmutableSet<ImmutableObject> getSaves() {
return saves;
public ImmutableSet<ImmutableObject> getInserts() {
return inserts;
}
public ImmutableSet<ImmutableObject> getUpdates() {
return updates;
}
;
public ImmutableSet<VKey<ImmutableObject>> getDeletes() {
return deletes;
}
;
public static Builder newBuilder() {
// Default both entities to save and entities to delete to empty sets, so that the build()
// method won't subsequently throw an exception if one doesn't end up being applicable.
// Default inserts, updates, and deletes to empty sets, so that the build() method won't
// subsequently throw an exception if one doesn't end up being applicable.
return new AutoBuilder_EntityChanges_Builder()
.setSaves(ImmutableSet.of())
.setInserts(ImmutableSet.of())
.setUpdates(ImmutableSet.of())
.setDeletes(ImmutableSet.of());
}
@@ -45,12 +50,21 @@ public record EntityChanges(
@AutoBuilder
public interface Builder {
Builder setSaves(ImmutableSet<ImmutableObject> entitiesToSave);
Builder setInserts(ImmutableSet<ImmutableObject> entitiesToInsert);
ImmutableSet.Builder<ImmutableObject> savesBuilder();
ImmutableSet.Builder<ImmutableObject> insertsBuilder();
default Builder addSave(ImmutableObject entityToSave) {
savesBuilder().add(entityToSave);
default Builder addInsert(ImmutableObject entityToInsert) {
insertsBuilder().add(entityToInsert);
return this;
}
Builder setUpdates(ImmutableSet<ImmutableObject> entitiesToUpdate);
ImmutableSet.Builder<ImmutableObject> updatesBuilder();
default Builder addUpdate(ImmutableObject entityToUpdate) {
updatesBuilder().add(entityToUpdate);
return this;
}
@@ -18,7 +18,6 @@ import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.dns.DnsUtils.requestDomainDnsRefresh;
import static google.registry.flows.FlowUtils.persistEntityChanges;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.verifyResourceDoesNotExist;
import static google.registry.flows.domain.DomainFlowUtils.COLLISION_MESSAGE;
import static google.registry.flows.domain.DomainFlowUtils.checkAllowedAccessToTld;
import static google.registry.flows.domain.DomainFlowUtils.checkHasBillingAccount;
@@ -224,6 +223,7 @@ public final class DomainCreateFlow implements MutatingFlow {
@Inject DomainCreateFlowCustomLogic flowCustomLogic;
@Inject DomainFlowTmchUtils tmchUtils;
@Inject DomainPricingLogic pricingLogic;
@Inject DomainDeletionTimeCache domainDeletionTimeCache;
@Inject DomainCreateFlow() {}
@@ -239,13 +239,13 @@ public final class DomainCreateFlow implements MutatingFlow {
validateRegistrarIsLoggedIn(registrarId);
verifyRegistrarIsActive(registrarId);
extensionManager.validate();
verifyDomainDoesNotExist();
DateTime now = tm().getTransactionTime();
DomainCommand.Create command = cloneAndLinkReferences((Create) resourceCommand, now);
Period period = command.getPeriod();
verifyUnitIsYears(period);
int years = period.getValue();
validateRegistrationPeriod(years);
verifyResourceDoesNotExist(Domain.class, targetId, now, registrarId);
// Validate that this is actually a legal domain name on a TLD that the registrar has access to.
InternetDomainName domainName = validateDomainName(command.getDomainName());
String domainLabel = domainName.parts().getFirst();
@@ -357,11 +357,11 @@ public final class DomainCreateFlow implements MutatingFlow {
domainHistoryId, registrationExpirationTime, isAnchorTenant, allocationToken);
PollMessage.Autorenew autorenewPollMessage =
createAutorenewPollMessage(domainHistoryId, registrationExpirationTime);
ImmutableSet.Builder<ImmutableObject> entitiesToSave = new ImmutableSet.Builder<>();
entitiesToSave.add(createBillingEvent, autorenewBillingEvent, autorenewPollMessage);
ImmutableSet.Builder<ImmutableObject> entitiesToInsert = new ImmutableSet.Builder<>();
entitiesToInsert.add(createBillingEvent, autorenewBillingEvent, autorenewPollMessage);
// Bill for EAP cost, if any.
if (!feesAndCredits.getEapCost().isZero()) {
entitiesToSave.add(createEapBillingEvent(feesAndCredits, createBillingEvent));
entitiesToInsert.add(createEapBillingEvent(feesAndCredits, createBillingEvent));
}
ImmutableSet<ReservationType> reservationTypes = getReservationTypes(domainName);
@@ -404,12 +404,13 @@ public final class DomainCreateFlow implements MutatingFlow {
DomainHistory domainHistory =
buildDomainHistory(domain, tld, now, period, tld.getAddGracePeriodLength());
if (reservationTypes.contains(NAME_COLLISION)) {
entitiesToSave.add(
entitiesToInsert.add(
createNameCollisionOneTimePollMessage(targetId, domainHistory, registrarId, now));
}
entitiesToSave.add(domain, domainHistory);
entitiesToInsert.add(domain, domainHistory);
ImmutableSet.Builder<ImmutableObject> entitiesToUpdate = new ImmutableSet.Builder<>();
if (allocationToken.isPresent() && allocationToken.get().getTokenType().isOneTimeUse()) {
entitiesToSave.add(
entitiesToUpdate.add(
AllocationTokenFlowUtils.redeemToken(
allocationToken.get(), domainHistory.getHistoryEntryId()));
}
@@ -422,7 +423,10 @@ public final class DomainCreateFlow implements MutatingFlow {
.setNewDomain(domain)
.setHistoryEntry(domainHistory)
.setEntityChanges(
EntityChanges.newBuilder().setSaves(entitiesToSave.build()).build())
EntityChanges.newBuilder()
.setInserts(entitiesToInsert.build())
.setUpdates(entitiesToUpdate.build())
.build())
.setYears(years)
.build());
persistEntityChanges(entityChanges);
@@ -649,6 +653,15 @@ public final class DomainCreateFlow implements MutatingFlow {
.build();
}
private void verifyDomainDoesNotExist() throws ResourceCreateContentionException {
Optional<DateTime> previousDeletionTime =
domainDeletionTimeCache.getDeletionTimeForDomain(targetId);
if (previousDeletionTime.isPresent()
&& !tm().getTransactionTime().isAfter(previousDeletionTime.get())) {
throw new ResourceCreateContentionException(targetId);
}
}
private static BillingEvent createEapBillingEvent(
FeesAndCredits feesAndCredits, BillingEvent createBillingEvent) {
return new BillingEvent.Builder()
@@ -151,7 +151,7 @@ public final class DomainDeleteFlow implements MutatingFlow, SqlStatementLogging
verifyDeleteAllowed(existingDomain, tld, now);
flowCustomLogic.afterValidation(
AfterValidationParameters.newBuilder().setExistingDomain(existingDomain).build());
ImmutableSet.Builder<ImmutableObject> entitiesToSave = new ImmutableSet.Builder<>();
ImmutableSet.Builder<ImmutableObject> entitiesToInsert = new ImmutableSet.Builder<>();
Domain.Builder builder;
if (existingDomain.getStatusValues().contains(StatusValue.PENDING_TRANSFER)) {
builder =
@@ -221,7 +221,7 @@ public final class DomainDeleteFlow implements MutatingFlow, SqlStatementLogging
} else {
PollMessage.OneTime deletePollMessage =
createDeletePollMessage(existingDomain, domainHistoryId, deletionTime);
entitiesToSave.add(deletePollMessage);
entitiesToInsert.add(deletePollMessage);
builder.setDeletePollMessage(deletePollMessage.createVKey());
}
}
@@ -230,7 +230,7 @@ public final class DomainDeleteFlow implements MutatingFlow, SqlStatementLogging
// registrar other than the sponsoring registrar (which will necessarily be a superuser).
if (durationUntilDelete.isLongerThan(Duration.ZERO)
&& !registrarId.equals(existingDomain.getPersistedCurrentSponsorRegistrarId())) {
entitiesToSave.add(
entitiesToInsert.add(
createImmediateDeletePollMessage(existingDomain, domainHistoryId, now, deletionTime));
}
@@ -239,7 +239,7 @@ public final class DomainDeleteFlow implements MutatingFlow, SqlStatementLogging
for (GracePeriod gracePeriod : existingDomain.getGracePeriods()) {
// No cancellation is written if the grace period was not for a billable event.
if (gracePeriod.hasBillingEvent()) {
entitiesToSave.add(
entitiesToInsert.add(
BillingCancellation.forGracePeriod(gracePeriod, now, domainHistoryId, targetId));
if (gracePeriod.getBillingEvent() != null) {
// Take the amount of registration time being refunded off the expiration time.
@@ -271,7 +271,7 @@ public final class DomainDeleteFlow implements MutatingFlow, SqlStatementLogging
// ResourceDeleteFlow since it's listed in serverApproveEntities.
requestDomainDnsRefresh(existingDomain.getDomainName());
entitiesToSave.add(newDomain, domainHistory);
entitiesToInsert.add(domainHistory);
EntityChanges entityChanges =
flowCustomLogic.beforeSave(
BeforeSaveParameters.newBuilder()
@@ -279,7 +279,10 @@ public final class DomainDeleteFlow implements MutatingFlow, SqlStatementLogging
.setNewDomain(newDomain)
.setHistoryEntry(domainHistory)
.setEntityChanges(
EntityChanges.newBuilder().setSaves(entitiesToSave.build()).build())
EntityChanges.newBuilder()
.setInserts(entitiesToInsert.build())
.addUpdate(newDomain)
.build())
.build());
BeforeResponseReturnData responseData =
flowCustomLogic.beforeResponse(
@@ -0,0 +1,133 @@
// Copyright 2025 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.flows.domain;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.github.benmanes.caffeine.cache.CacheLoader;
import com.github.benmanes.caffeine.cache.Caffeine;
import com.github.benmanes.caffeine.cache.Expiry;
import com.github.benmanes.caffeine.cache.LoadingCache;
import com.github.benmanes.caffeine.cache.Ticker;
import com.google.common.collect.ImmutableSet;
import google.registry.model.ForeignKeyUtils;
import google.registry.model.domain.Domain;
import google.registry.util.DateTimeUtils;
import java.util.Optional;
import org.joda.time.DateTime;
/**
* Functionally-static loading cache that keeps track of deletion (AKA drop) times for domains.
*
* <p>Some domain names may have many create requests issued shortly before (and directly after) the
* name is released due to a previous registrant deleting it. In those cases, caching the deletion
* time of the existing domain allows us to short-circuit the request and avoid any load on the
* database checking the existing domain (at least, in cases where the request hits a particular
* node more than once).
*
* <p>The cache is fairly short-lived (as we're concerned about many requests at basically the same
* time), and entries also expire when the drop actually happens. If the domain is re-created after
* a drop, the next load attempt will populate the cache with a deletion time of END_OF_TIME, which
* will be read from the cache by subsequent attempts.
*
* <p>We take advantage of the fact that Caffeine caches don't store nulls returned from the
* CacheLoader, so a null result (meaning the domain doesn't exist) won't affect future calls (this
* avoids a stale-cache situation where the cache "thinks" the domain doesn't exist, but it does).
* Put another way, if a domain really doesn't exist, we'll re-attempt the database load every time.
*
* <p>We don't explicitly set the cache inside domain create/delete flows, in case the transaction
* fails at commit time. It's better to have stale data, or to require an additional database load,
* than to have incorrect data.
*
* <p>Note: this should be injected as a singleton -- it's essentially static, but we have it as a
* non-static object for concurrent testing purposes.
*/
public class DomainDeletionTimeCache {
// Max expiry time is ten minutes
private static final long NANOS_IN_ONE_MILLISECOND = 100000L;
private static final long MAX_EXPIRY_NANOS = 10L * 60L * 1000L * NANOS_IN_ONE_MILLISECOND;
private static final int MAX_ENTRIES = 500;
/**
* Expire after the max duration, or after the domain is set to drop (whichever comes first).
*
* <p>If the domain has already been deleted (the deletion time is <= now), the entry will
* immediately be expired/removed.
*
* <p>NB: the Expiry class requires the return value in <b>nanoseconds</b>, not milliseconds
*/
private static final Expiry<String, DateTime> EXPIRY_POLICY =
new Expiry<>() {
@Override
public long expireAfterCreate(String key, DateTime value, long currentTime) {
// Watch out for Long overflow
long deletionTimeNanos =
value.equals(DateTimeUtils.END_OF_TIME)
? Long.MAX_VALUE
: value.getMillis() * NANOS_IN_ONE_MILLISECOND;
long nanosUntilDeletion = deletionTimeNanos - currentTime;
return Math.max(0L, Math.min(MAX_EXPIRY_NANOS, nanosUntilDeletion));
}
/** Reset the time entirely on update, as if we were creating the entry anew. */
@Override
public long expireAfterUpdate(
String key, DateTime value, long currentTime, long currentDuration) {
return expireAfterCreate(key, value, currentTime);
}
/** Reads do not change the expiry duration. */
@Override
public long expireAfterRead(
String key, DateTime value, long currentTime, long currentDuration) {
return currentDuration;
}
};
/** Attempt to load the domain's deletion time if the domain exists. */
private static final CacheLoader<String, DateTime> CACHE_LOADER =
(domainName) -> {
ForeignKeyUtils.MostRecentResource mostRecentResource =
ForeignKeyUtils.loadMostRecentResources(
Domain.class, ImmutableSet.of(domainName), false)
.get(domainName);
return mostRecentResource == null ? null : mostRecentResource.deletionTime();
};
// Unfortunately, maintenance tasks aren't necessarily already in a transaction
private static final Ticker TRANSACTION_TIME_TICKER =
() -> tm().reTransact(() -> tm().getTransactionTime().getMillis() * NANOS_IN_ONE_MILLISECOND);
public static DomainDeletionTimeCache create() {
return new DomainDeletionTimeCache(
Caffeine.newBuilder()
.ticker(TRANSACTION_TIME_TICKER)
.expireAfter(EXPIRY_POLICY)
.maximumSize(MAX_ENTRIES)
.build(CACHE_LOADER));
}
private final LoadingCache<String, DateTime> cache;
private DomainDeletionTimeCache(LoadingCache<String, DateTime> cache) {
this.cache = cache;
}
/** Returns the domain's deletion time, or null if it doesn't currently exist. */
public Optional<DateTime> getDeletionTimeForDomain(String domainName) {
return Optional.ofNullable(cache.get(domainName));
}
}
@@ -1,4 +1,4 @@
// Copyright 2024 The Nomulus Authors. All Rights Reserved.
// Copyright 2025 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
@@ -12,12 +12,19 @@
// See the License for the specific language governing permissions and
// limitations under the License.
import { Component } from '@angular/core';
package google.registry.flows.domain;
@Component({
selector: 'app-tlds',
templateUrl: './tlds.component.html',
styleUrls: ['./tlds.component.scss'],
standalone: false,
})
export class TldsComponent {}
import dagger.Module;
import dagger.Provides;
import jakarta.inject.Singleton;
/** Dagger module to provide the {@link DomainDeletionTimeCache}. */
@Module
public class DomainDeletionTimeCacheModule {
@Provides
@Singleton
public static DomainDeletionTimeCache provideDomainDeletionTimeCache() {
return DomainDeletionTimeCache.create();
}
}
@@ -711,7 +711,7 @@ public class DomainFlowUtils {
BillingRecurrence newBillingRecurrence =
existingBillingRecurrence.asBuilder().setRecurrenceEndTime(newEndTime).build();
tm().put(newBillingRecurrence);
tm().update(newBillingRecurrence);
return newBillingRecurrence;
}
@@ -245,11 +245,13 @@ public final class DomainRenewFlow implements MutatingFlow {
.build();
DomainHistory domainHistory =
buildDomainHistory(newDomain, now, command.getPeriod(), tld.getRenewGracePeriodLength());
ImmutableSet.Builder<ImmutableObject> entitiesToSave = new ImmutableSet.Builder<>();
entitiesToSave.add(
newDomain, domainHistory, explicitRenewEvent, newAutorenewEvent, newAutorenewPollMessage);
ImmutableSet.Builder<ImmutableObject> entitiesToInsert = new ImmutableSet.Builder<>();
ImmutableSet.Builder<ImmutableObject> entitiesToUpdate = new ImmutableSet.Builder<>();
entitiesToInsert.add(
domainHistory, explicitRenewEvent, newAutorenewEvent, newAutorenewPollMessage);
entitiesToUpdate.add(newDomain);
if (allocationToken.isPresent() && allocationToken.get().getTokenType().isOneTimeUse()) {
entitiesToSave.add(
entitiesToUpdate.add(
AllocationTokenFlowUtils.redeemToken(
allocationToken.get(), domainHistory.getHistoryEntryId()));
}
@@ -262,7 +264,10 @@ public final class DomainRenewFlow implements MutatingFlow {
.setYears(years)
.setHistoryEntry(domainHistory)
.setEntityChanges(
EntityChanges.newBuilder().setSaves(entitiesToSave.build()).build())
EntityChanges.newBuilder()
.setInserts(entitiesToInsert.build())
.setUpdates(entitiesToUpdate.build())
.build())
.build());
BeforeResponseReturnData responseData =
flowCustomLogic.beforeResponse(
@@ -146,18 +146,18 @@ public final class DomainRestoreRequestFlow implements MutatingFlow {
verifyRestoreAllowed(command, existingDomain, feeUpdate, feesAndCredits, now);
HistoryEntryId domainHistoryId = createHistoryEntryId(existingDomain);
historyBuilder.setRevisionId(domainHistoryId.getRevisionId());
ImmutableSet.Builder<ImmutableObject> entitiesToSave = new ImmutableSet.Builder<>();
ImmutableSet.Builder<ImmutableObject> entitiesToInsert = new ImmutableSet.Builder<>();
DateTime newExpirationTime =
existingDomain.getRegistrationExpirationTime().plusYears(isExpired ? 1 : 0);
// Restore the expiration time on the deleted domain, except if that's already passed, then add
// a year and bill for it immediately, with no grace period.
if (isExpired) {
entitiesToSave.add(
entitiesToInsert.add(
createRenewBillingEvent(domainHistoryId, feesAndCredits.getRenewCost(), now));
}
// Always bill for the restore itself.
entitiesToSave.add(
entitiesToInsert.add(
createRestoreBillingEvent(domainHistoryId, feesAndCredits.getRestoreCost(), now));
BillingRecurrence autorenewEvent =
@@ -166,12 +166,14 @@ public final class DomainRestoreRequestFlow implements MutatingFlow {
.setRecurrenceEndTime(END_OF_TIME)
.setDomainHistoryId(domainHistoryId)
.build();
entitiesToInsert.add(autorenewEvent);
PollMessage.Autorenew autorenewPollMessage =
newAutorenewPollMessage(existingDomain)
.setEventTime(newExpirationTime)
.setAutorenewEndTime(END_OF_TIME)
.setDomainHistoryId(domainHistoryId)
.build();
entitiesToInsert.add(autorenewPollMessage);
Domain newDomain =
performRestore(
existingDomain,
@@ -181,8 +183,9 @@ public final class DomainRestoreRequestFlow implements MutatingFlow {
now,
registrarId);
DomainHistory domainHistory = buildDomainHistory(newDomain, now);
entitiesToSave.add(newDomain, domainHistory, autorenewEvent, autorenewPollMessage);
tm().putAll(entitiesToSave.build());
entitiesToInsert.add(domainHistory);
tm().update(newDomain);
tm().insertAll(entitiesToInsert.build());
if (existingDomain.getDeletePollMessage() != null) {
tm().delete(existingDomain.getDeletePollMessage());
}
@@ -172,7 +172,7 @@ public final class DomainTransferApproveFlow implements MutatingFlow {
.setDomainHistoryId(domainHistoryId)
.build());
ImmutableList.Builder<ImmutableObject> entitiesToSave = new ImmutableList.Builder<>();
ImmutableList.Builder<ImmutableObject> entitiesToInsert = new ImmutableList.Builder<>();
// If we are within an autorenew grace period, cancel the autorenew billing event and don't
// increase the registration time, since the transfer subsumes the autorenew's extra year.
GracePeriod autorenewGrace =
@@ -184,7 +184,7 @@ public final class DomainTransferApproveFlow implements MutatingFlow {
// then the gaining registrar is not charged for the one-year renewal and the losing registrar
// still needs to be charged for the auto-renew.
if (billingEvent.isPresent()) {
entitiesToSave.add(
entitiesToInsert.add(
BillingCancellation.forGracePeriod(autorenewGrace, now, domainHistoryId, targetId));
}
}
@@ -259,14 +259,11 @@ public final class DomainTransferApproveFlow implements MutatingFlow {
PollMessage gainingClientPollMessage =
createGainingTransferPollMessage(
targetId, newDomain.getTransferData(), newExpirationTime, now, domainHistoryId);
billingEvent.ifPresent(entitiesToSave::add);
entitiesToSave.add(
autorenewEvent,
gainingClientPollMessage,
gainingClientAutorenewPollMessage,
newDomain,
domainHistory);
tm().putAll(entitiesToSave.build());
billingEvent.ifPresent(entitiesToInsert::add);
entitiesToInsert.add(
autorenewEvent, gainingClientPollMessage, gainingClientAutorenewPollMessage, domainHistory);
tm().update(newDomain);
tm().insertAll(entitiesToInsert.build());
// Delete the billing event and poll messages that were written in case the transfer would have
// been implicitly server approved.
tm().delete(existingDomain.getTransferData().getServerApproveEntities());
@@ -110,8 +110,8 @@ public final class DomainTransferCancelFlow implements MutatingFlow {
Domain newDomain =
denyPendingTransfer(existingDomain, TransferStatus.CLIENT_CANCELLED, now, registrarId);
DomainHistory domainHistory = buildDomainHistory(newDomain, tld, now);
tm().putAll(
newDomain,
tm().update(newDomain);
tm().insertAll(
domainHistory,
createLosingTransferPollMessage(
targetId, newDomain.getTransferData(), null, domainHistoryId));
@@ -109,8 +109,8 @@ public final class DomainTransferRejectFlow implements MutatingFlow {
Domain newDomain =
denyPendingTransfer(existingDomain, TransferStatus.CLIENT_REJECTED, now, registrarId);
DomainHistory domainHistory = buildDomainHistory(newDomain, tld, now);
tm().putAll(
newDomain,
tm().update(newDomain);
tm().insertAll(
domainHistory,
createGainingTransferPollMessage(
targetId, newDomain.getTransferData(), null, now, domainHistoryId));
@@ -283,11 +283,9 @@ public final class DomainTransferRequestFlow implements MutatingFlow {
asyncTaskEnqueuer.enqueueAsyncResave(
newDomain.createVKey(), now, ImmutableSortedSet.of(automaticTransferTime));
tm().putAll(
new ImmutableSet.Builder<>()
.add(newDomain, domainHistory, requestPollMessage)
.addAll(serverApproveEntities)
.build());
tm().put(newDomain);
tm().putAll(serverApproveEntities);
tm().insertAll(domainHistory, requestPollMessage);
return responseBuilder
.setResultFromCode(SUCCESS_WITH_ACTION_PENDING)
.setResData(createResponse(period, existingDomain, newDomain, now))
@@ -190,14 +190,16 @@ public final class DomainUpdateFlow implements MutatingFlow {
if (requiresDnsUpdate(existingDomain, newDomain)) {
requestDomainDnsRefresh(targetId);
}
ImmutableSet.Builder<ImmutableObject> entitiesToSave = new ImmutableSet.Builder<>();
entitiesToSave.add(newDomain, domainHistory);
ImmutableSet.Builder<ImmutableObject> entitiesToInsert = new ImmutableSet.Builder<>();
ImmutableSet.Builder<ImmutableObject> entitiesToUpdate = new ImmutableSet.Builder<>();
entitiesToUpdate.add(newDomain);
entitiesToInsert.add(domainHistory);
Optional<BillingEvent> statusUpdateBillingEvent =
createBillingEventForStatusUpdates(existingDomain, newDomain, domainHistory, now);
statusUpdateBillingEvent.ifPresent(entitiesToSave::add);
statusUpdateBillingEvent.ifPresent(entitiesToInsert::add);
Optional<PollMessage.OneTime> serverStatusUpdatePollMessage =
createPollMessageForServerStatusUpdates(existingDomain, newDomain, domainHistory, now);
serverStatusUpdatePollMessage.ifPresent(entitiesToSave::add);
serverStatusUpdatePollMessage.ifPresent(entitiesToInsert::add);
EntityChanges entityChanges =
flowCustomLogic.beforeSave(
BeforeSaveParameters.newBuilder()
@@ -205,7 +207,10 @@ public final class DomainUpdateFlow implements MutatingFlow {
.setNewDomain(newDomain)
.setExistingDomain(existingDomain)
.setEntityChanges(
EntityChanges.newBuilder().setSaves(entitiesToSave.build()).build())
EntityChanges.newBuilder()
.setInserts(entitiesToInsert.build())
.setUpdates(entitiesToUpdate.build())
.build())
.build());
persistEntityChanges(entityChanges);
return responseBuilder.build();
@@ -126,7 +126,8 @@ public final class HostCreateFlow implements MutatingFlow {
.setSuperordinateDomain(superordinateDomain.map(Domain::createVKey).orElse(null))
.build();
historyBuilder.setType(HOST_CREATE).setHost(newHost);
ImmutableSet<ImmutableObject> entitiesToSave = ImmutableSet.of(newHost, historyBuilder.build());
ImmutableSet<ImmutableObject> entitiesToInsert =
ImmutableSet.of(newHost, historyBuilder.build());
if (superordinateDomain.isPresent()) {
tm().update(
superordinateDomain
@@ -138,7 +139,7 @@ public final class HostCreateFlow implements MutatingFlow {
// they are only written as NS records from the referencing domain.
requestHostDnsRefresh(targetId);
}
tm().insertAll(entitiesToSave);
tm().insertAll(entitiesToInsert);
return responseBuilder.setResData(HostCreateData.create(targetId, now)).build();
}
@@ -52,7 +52,6 @@ import google.registry.flows.annotations.ReportingSpec;
import google.registry.flows.exceptions.ResourceHasClientUpdateProhibitedException;
import google.registry.model.EppResource;
import google.registry.model.ForeignKeyUtils;
import google.registry.model.ImmutableObject;
import google.registry.model.domain.Domain;
import google.registry.model.domain.metadata.MetadataExtension;
import google.registry.model.eppcommon.StatusValue;
@@ -198,16 +197,12 @@ public final class HostUpdateFlow implements MutatingFlow {
.setPersistedCurrentSponsorRegistrarId(newPersistedRegistrarId)
.build();
verifyHasIpsIffIsExternal(command, existingHost, newHost);
ImmutableSet.Builder<ImmutableObject> entitiesToInsert = new ImmutableSet.Builder<>();
ImmutableSet.Builder<ImmutableObject> entitiesToUpdate = new ImmutableSet.Builder<>();
entitiesToUpdate.add(newHost);
if (isHostRename) {
updateSuperordinateDomains(existingHost, newHost);
}
enqueueTasks(existingHost, newHost);
entitiesToInsert.add(historyBuilder.setType(HOST_UPDATE).setHost(newHost).build());
tm().updateAll(entitiesToUpdate.build());
tm().insertAll(entitiesToInsert.build());
tm().update(newHost);
tm().insert(historyBuilder.setType(HOST_UPDATE).setHost(newHost).build());
return responseBuilder.build();
}
@@ -290,7 +285,7 @@ public final class HostUpdateFlow implements MutatingFlow {
&& newHost.isSubordinate()
&& Objects.equals(
existingHost.getSuperordinateDomain(), newHost.getSuperordinateDomain())) {
tm().put(
tm().update(
tm().loadByKey(existingHost.getSuperordinateDomain())
.asBuilder()
.removeSubordinateHost(existingHost.getHostName())
@@ -299,14 +294,14 @@ public final class HostUpdateFlow implements MutatingFlow {
return;
}
if (existingHost.isSubordinate()) {
tm().put(
tm().update(
tm().loadByKey(existingHost.getSuperordinateDomain())
.asBuilder()
.removeSubordinateHost(existingHost.getHostName())
.build());
}
if (newHost.isSubordinate()) {
tm().put(
tm().update(
tm().loadByKey(newHost.getSuperordinateDomain())
.asBuilder()
.addSubordinateHost(newHost.getHostName())
@@ -0,0 +1,661 @@
<?xml version="1.0" encoding="utf-8"?>
<lgr xmlns="urn:ietf:params:xml:ns:lgr-1.0">
<meta>
<version comment="Latin LGR">1</version>
<date>2025-10-01</date>
<language>und-Latn</language>
<unicode-version>2</unicode-version>
<description type="text/html"><![CDATA[
<div class="instructions">
<h2>INSTRUCTIONS</h2>
<ul>
<li>These instructions cover how to adopt an LGR based on this reference LGR for a given
zone and how to prepare the file for deposit in the IANA Repository of IDN Practices.</li>
<li>As described the IANA procedure (https://www.iana.org/help/idn-repository-procedure) an
LGR MUST contain the following elements in its header:
<ul style="list-style-type:square;">
<li>Script or Language Designator (see below for guidance) </li>
<li>Version Number (this must increase with each amendment to the LGR, even if the updates
are limited to the header itself) </li>
<li>Effective Date (the date at which the policy becomes applicable in operational use) </li>
<li>Registry Contact Details (contact name, email address, and/or phone number)</li>
</ul>
</li>
<li>The following information is optional:
<ul style="list-style-type:square;">
<li>Document creation date</li>
<li>Applicable Domain(s)</li>
<li>Changes made to the Reference LGR before adopting</li>
</ul>
</li>
</ul>
<p>Please add or modify the following items in the <b>XML source code for this file</b> before
depositing the document in the IANA Repository. (https://www.iana.org/domains/idn-tables)</p>
<h3>Meta Data</h3>
<p>Note: version numbers start at 1. RFC 7940 recommends using simple integers. The version comment is optional,
please replace or delete the default comment. Version comments may be used by some tools as part of the page header.</p>
<p><code>&lt;version comment=&quot;</code>[Please replace (or delete) the optional comment]<code>&quot;&gt;</code>[Please fill in version number, starting at 1]<code>&lt;/version&gt;</code></p>
<p><code>&lt;date&gt;</code>2025-10-01<code>&lt;/date&gt;</code></p>
<p><code>&lt;validity-start&gt;</code>2025-10-01<code>&lt;/validity-start&gt;</code></p>
<p>Note: the scope element may be repeated, so that the same document can serve for multiple domains.</p>
<p><code>&lt;scope type=&quot;domain&quot;&gt;</code>[Please provide, in &quot;.domain&quot; format]<code>&lt;/scope&gt;</code></p>
<p><strong>Registry Contact Information:</strong></p>
<p>Please fill in the <a href="#registry_contact_details">Registry Contact Details</a>.</p>
<p><strong>Change History</strong></p>
<p>If you made technical modifications to the LGR, please summarize them in the <a href="#change_history">Change History</a> (and also note the details in the appropriate section of the description).</p>
<section id="registry_contact_details">
<h2>Registry Contact Details</h2>
<ul style="list-style:none;">
<li><b>Contact Name:</b> Ben McIlwain</li>
<li><b>Email address:</b> nomulus-discuss@google.com</li>
</ul>
</section>
<h1>Label Generation Rules for the Latin Script</h1>
<h2>Overview</h2>
<p>This document specifies a set of Label Generation Rules (LGR) for the Latin script for the second level domain or domains identified above.
The starting point for the development of this LGR can be found in the related Root Zone LGR [RZ-LGR-Latn].
The format of this file follows [RFC 7940].
This LGR is adapted from the “Reference LGR for the Second Level for the Latin Script” [Ref-LGR-und-Latn], for details, see <a href="#change_history">Change History</a> below.</p>
<p>For details and additional background on the Latin script, see “Proposal for a Latin Script Root Zone Label Generation Rule-Set (LGR)” [Proposal-Latin].</p>
<h2>Repertoire</h2>
<p>The repertoire contains the 164 letters needed to write hundreds of languages in the Latin script.
The repertoire is a subset of [Unicode 11.0.0]. For details, see Section 5, “Repertoire” in [Proposal-Latin].
(The proposal cited has been adopted for the Latin script portion of the Root Zone LGR.)</p>
<p>For the second level, the repertoire has been augmented with the ASCII digits, U+0030 to U+0039, plus U+002D HYPHEN-MINUS, for a total of 175 repertoire elements.</p>
<p>Any code points outside the Latin Script repertoire that are targets for
out-of-repertoire variants would be included here only if the variant is listed
in this file. In this case they are identified as a reflexive (identity) variant
of type “out-of-repertoire-var”. Whether or not they are listed, they do not
form part of the repertoire.</p>
<p><b>Repertoire Listing:</b> Each code point or range is tagged with the script or scripts with which the code point is used and one or more other character categories. For each repertoire element,
one or more references document sufficient justification for inclusion in the repertoire; see the <a href="#ref_desc_sec_References">“References”</a> below.
For code points that are part of the repertoire, comments identify the languages using the code point along with their [EGIDS] level.</p>
<h3>Background on Script and Principal Languages Using It</h3>
<p>The Latin script is a major writing system of the world, and the most widely used in terms of
the number of languages and speakers, with circa 70% of the worlds readers and writers making use of
this script. From a list of 1,189 languages using the Latin script [Omniglot]
the 212 languages that were taken into consideration contain all 182 languages with [EGIDS]
level 1&ndash;4 together with many languages with EGIDS level 5, each spoken by more than
1 million estimated speakers. Altogether over 100 languages are cited here to justify
specific additions to the repertoire, but many other languages may also be written using
some subset of the repertoire of this LGR. In a few cases, code points were excluded in [MSR-5] due to
security concerns; for the affected languages, only a subrepertoire could be supported.
More details in Section 3, “Background on Script and Principal Languages Using It”
of [Proposal-Latin].</p>
<h2>Variants</h2>
<p>The variants defined in this LGR are limited to those required for use in zones not shared with any other script.
As such, this LGR does not define cross-script variants. However, using this LGR concurrently with any LGR for Armenian, Cyrillic, and Greek in the same zone will introduce some in-script variants due to cross-script variant transitivity. This will also create potential cross-script issues when used with the same LGRs.
For details, see Section 6, “Variants” in [Proposal-Latin].
Mitigation of these in-script and cross-script variants can be addressed by using the Common LGR.
For details, see Section 3, “Use of Multiple Reference LGRs in the Same Zone” in [Level-2-Overview].
In addition to variants defined by this LGR, the full variant information related to this script and added by concurrent use with the Armenian, Cyrillic, and Greek LGR(s) can be found
in the following LGR: [Ref-LGR-Latin-Full-Variant-Script]
</p>
<p>In particular, the Latin LGR contains a number of in-script variants resulting from transitivity with
variants needed if the LGR is used concurrently with other LGRs from the related scripts, primarily
due to variants with the Greek script.
These variant definitions are required when using this LGR together with the Common LGR in label processing,
but they also reflect a certain consistency in the approach to variants across typographically related scripts.
They can be removed if the LGR is used strictly as standalone.<p>
<p>All other in-script variants defined in this LGR largely follow the methodology defined in
Section 6, “Variants”, in [Proposal-Latin]. In a separate appendix that proposal identifies additional
candidates for visually confusable code points (see [Proposal-Latin-Appendices]). They provide data
on the basis of which additional variants, or fallback variants (see following) might be defined. However,
doing so would require additional review and analysis that has not been carried out for this version of the
LGR.</p>
<p>The LGR defines certain allocatable fallback variants as
described in Section 4.5.5 “Allocatable Fallback Variants” in [Level-2-Overview]. A fallback variant is a variant label that uses
substitute code points for code points or sequences not available (or not allowed) in some contexts, that would
otherwise be required for a linguistically accurate rendering of some label.</p>
<p>When “fallback” variants are defined, two labels may be allocated: a single label with the spelling preferred by the
applicant, plus a single fallback variant for that label.
The fallback exclusively uses the fallback characters for any characters for which fallbacks are defined, while the
“preferred” label may use any otherwise valid mix of code points.
If the fallback variant is the one applied for, no other variant label is allocatable.</p>
<p>An allocatable fallback variant exists for the following pairs where the second element of each pair is the fallback:</p>
<ul>
<h3>In-script Variant Mapping Types</h3>
<p>In each of the fallback variant pairs defined above, the mapping type from the first element to the second is of type
“fallback”, while the variant type for the other direction is “blocked”. In addition, the first element of each pair uses the
reflexive mapping “r-original”.
(By convention, the prefix “r-” marks a type used in a reflexive variant mapping, that is, it represents an instance
of the original code point at that location in a variant label, see Section 5.3.4 in [RFC 7940].)</p>
<p><b>Variant Disposition:</b> Except for limited exceptions for the fallback variants defined above, variants defined here result
in a variant label disposition of “blocked”.</p>
<p>The specification of variants in this LGR follows the guidelines in [RFC 8228].</p>
<h2>Character Classes</h2>
<p>This LGR does not define named character classes.</p>
<h2>Whole Label Evaluation (WLE) and Context rules</h2>
<h3>Default Whole Label Evaluation Rules and Actions</h3>
<p>By default, the LGR includes the rules and actions to implement the following restrictions mandated by the IDNA protocol. They are marked with &#x235F;.</p>
<ul>
<li><b>Hyphen Restrictions</b> &mdash; restrictions on the allowable placement of hyphens (no leading/ending hyphen
and no hyphen in positions 3 and 4). These restrictions are described in Section 4.2.3.1 of RFC 5891 [320].
They are implemented here as context rule on U+002D (-) HYPHEN-MINUS.</li>
<li><b>Leading Combining Marks</b> &mdash; restrictions on the allowable placement of combining marks
(no leading combining mark). This rule is described in Section 4.2.3.2 of RFC 5891 [320].</li>
</ul>
<h3>Latin-specific Rules</h3>
<h2>Actions</h2>
<h3>Default Actions</h3>
<p>This LGR includes the default actions for LGRs as well as the action needed to
invalidate labels with misplaced combining marks. They are marked with &#x235F;.
For a description see [RFC 7940].</p>
<p>Default actions that are
triggered by the LGR-specific variant types described above limit the “allocatable” variant
labels to those containing only “ss”, dotted “i” or hyphen variants, while
disallowing mixed use of “ss” and “ß”, Dotless i and “i”, or middle-dot and hyphen respectively, except
as in the original applied-for label.</p>
<p>Note that the mapping types for variants are not symmetric: they depend on which code point is considered
the source or the target in a given mapping. As specified in [RFC 7940], when mapping types are evaluated
code points in a label that are unchanged use the type of their “reflexive” mapping.
Per [RFC 7940] the actions are always applied one after the other, and the evaluation stops at the first
action that assigns a disposition to a given label.</p>
<h3>Script-specific Actions</h3>
<p>An action has been defined to invalidate labels containing the sequence U+00B7 U+006C U+00B7 which could otherwise result in two Ela Geminada sequences overlapping.</p>
<h2>Methodology and Contributors</h2>
<p>The LGR in this document has been adapted from the corresponding Reference LGR for the Second Level. The Second Level Reference LGR for the Latin Script was developed by Michel Suignard and Asmus Freytag, based on the Root Zone LGR for the Latin
script and information contained or referenced therein; see [RZ-LGR-Latn]. Suitable extensions for the second level have been applied according to the [Guidelines] and with community input.
The original proposal for a Root Zone LGR for the Latin script, that this LGR is based on, was developed by the Latin Generation Panel.
For more information on methodology and contributors to the underlying Root Zone LGR, see Sections 4 and 8 in [Proposal-Latin], as well as [RZ-LGR-Overview].</p>
<section id="change_history">
<h3>Changes from Version Dated 25 October 2024</h3>
<p>Adopted from the Second Level Reference LGR for the Latin Script [Ref-LGR-und-Latn] with security improvements implemented by removing confusable variants.</p>
</section>
<h2>References</h2>
<p>The following general references are cited in this document:</p>
<dl class="references">
<dt>[EGIDS]</dt>
<dd>Lewis and Simons, “EGIDS: Expanded Graded Intergenerational Disruption Scale,”
documented in [SIL-Ethnologue] and summarized here:
https://en.wikipedia.org/wiki/Expanded_Graded_Intergenerational_Disruption_Scale_(EGIDS)</dd>
<dt>[Guidelines]</dt>
<dd>ICANN, “Guidelines for Developing Reference LGRs for the Second Level”, (Los Angeles, California: ICANN, 27 May 2020), https://www.icann.org/en/system/files/files/lgr-guidelines-second-level-27may20-en.pdf</dd>
<dt>[Level-2-Overview]</dt>
<dd>Internet Corporation for Assigned Names and Numbers, (ICANN),“Reference Label Generation Rules (LGR) for the Second Level: Overview and Summary” (PDF),
(Los Angeles, California: ICANN, 25 October 2024), https://www.icann.org/en/system/files/files/level2-lgr-overview-summary-25oct24-en.pdf
</dd>
<dt>[MSR-5]</dt>
<dd>Integration Panel, “Maximal Starting Repertoire — MSR-5 Overview and Rationale”, 24 June 2021,
https://www.icann.org/en/system/files/files/msr-5-overview-24jun21-en.pdf</dd>
<dt>[Omniglot]</dt>
<dd>Omniglot, “Writing Systems by Language”, https://www.omniglot.com/writing/langalph.htm (accessed on 13 January 2022)</dd>
<dt>[Proposal-Latin]</dt>
<dd>Latin Generation Panel, “Proposal for a Latin Script Root Zone Label Generation Rule-Set (LGR)”, 27 January 2022 (PDF), https://www.icann.org/en/system/files/files/proposal-latin-lgr-27jan22-en.pdf</dd>
<dt>[Proposal-Latin-Appendices]</dt>
<dd>Appendices to “Proposal for a Latin Script Root Zone Label Generation Rule-Set (LGR)”,
27 January 2022 (ZIP), https://www.icann.org/en/system/files/files/proposal-latin-lgr-appendices-27jan22-en.zip</dd>
<dt>[Ref-LGR-und-Latn]</dt>
<dd>ICANN, Second Level Reference Label Generation Rules for the Latin Script (und-Latn), 25 October 2024 (XML)
https://www.icann.org/sites/default/files/packages/lgr/lgr-second-level-latin-script-25oct24-en.xml
non-normative HTML presentation: https://www.icann.org/sites/default/files/packages/lgr/lgr-second-level-latin-script-25oct24-en.html</dd>
<dt>[Ref-LGR-Latin-Full-Variant-Script]</dt>
<dd>ICANN, Second Level Reference Label Generation Rules for the Latin Script (und-Latn), 25 October 2024 (XML)
https://www.icann.org/sites/default/files/packages/lgr/lgr-second-level-latin-full-variant-script-25oct24-en.xml
non-normative HTML presentation: https://www.icann.org/sites/default/files/packages/lgr/lgr-second-level-latin-full-variant-script-25oct24-en.html</dd>
<dt>[RFC 7940]</dt>
<dd>Davies, K. and A. Freytag, “Representing Label Generation Rulesets Using XML”,
RFC 7940, August 2016, https://www.rfc-editor.org/info/rfc7940</dd>
<dt>[RFC 8228]</dt>
<dd>A. Freytag, “Guidance on Designing Label Generation Rulesets (LGRs) Supporting Variant Labels”, RFC 8228, August 2017,
https://www.rfc-editor.org/info/rfc8228</dd>
<dt>[RZ-LGR-Overview]</dt>
<dd>Integration Panel, “Root Zone Label Generation Rules (RZ LGR-5): Overview and Summary”, 26 May 2022 (PDF), https://www.icann.org/sites/default/files/lgr/rz-lgr-5-overview-26may22-en.pdf</dd>
<dt>[RZ-LGR-5]</dt>
<dd>Integration Panel, “Root Zone Label Generation Rules (RZ-LGR-5)”, 26 May 2022 (XML), https://www.icann.org/sites/default/files/lgr/rz-lgr-5-common-26may22-en.xml <br/>
<i>non-normative HTML presentation: https://www.icann.org/sites/default/files/lgr/rz-lgr-5-common-26may22-en.html</i></dd>
<dt>[RZ-LGR-Latn]</dt>
<dd>ICANN, Root Zone Label Generation Rules for the Latin Script (und-Latn), 26 May 2022 (XML)
https://www.icann.org/sites/default/files/lgr/rz-lgr-5-latin-script-26may22-en.xml</dd>
<dt>[SIL-Ethnologue]</dt>
<dd>David M. Eberhard, Gary F. Simons &amp; Charles D. Fennig (eds.). 2021.
Ethnologue: Languages of the World, Twenty fourth edition. Dallas, Texas: SIL
International. Online version available as https://www.ethnologue.com</dd>
<dt>[Unicode 11.0.0]</dt>
<dd>The Unicode Consortium. The Unicode Standard, Version 11.0.0, (Mountain View, CA: The Unicode Consortium, 2018. ISBN 978-1-936213-19-1)
https://www.unicode.org/versions/Unicode11.0.0/</dd>
</dl>
<p>For references consulted particularly in designing the repertoire for the Latin Script for the second level
please see details in the <a href="#table_of_references">Table of References</a> below.</p>
<p>References [0] and up refer to the Unicode Standard versions in which corresponding code points
were initially encoded. References [101] and up correspond to a source given in [Proposal-Latin] for justifying
the inclusion of the corresponding code points. In the listing of <a href="#whole_label_evaluation_and_context_rules">whole label evaluation and context rules</a>,
reference [320] indicates the source for common rules.</p>
]]></description>
<references>
<reference id="0" comment="Any code point originally encoded in Unicode Version 1.1">The Unicode Standard, Version 1.1</reference>
<reference id="3" comment="Any code point originally encoded in Unicode Version 3.0">The Unicode Standard, Version 3.0</reference>
<reference id="8" comment="Any code point originally encoded in Unicode Version 5.0">The Unicode Standard, Version 5.0</reference>
<reference id="99" comment="Any code point cited is part of the Basic Latin (ASCII) set">C0 Controls and Basic Latin, The Unicode Standard https://unicode.org/charts/PDF/U0000.pdf</reference>
<reference id="100">ICANN, Second Level Reference Label Generation Rules for Spanish https://www.icann.org/sites/default/files/packages/lgr/lgr-second-level-spanish-30aug16-en.html (Accessed on 31 August 2018)</reference>
<reference id="101">Omniglot, Czech (čeština) https://www.omniglot.com/writing/czech.htm (Accessed on 31 August 2018)</reference>
<reference id="102">Omniglot, Icelandic (Íslenska) https://www.omniglot.com/writing/icelandic.htm (Accessed on 31 August 2018)</reference>
<reference id="103">Omniglot, Faroese (føroyskt mál) https://www.omniglot.com/writing/faroese.htm (Accessed on 31 August 2018)</reference>
<reference id="105">Omniglot, Chuukese (Chuuk) https://www.omniglot.com/writing/chuukese.htm (Accessed on 31 August 2018)</reference>
<reference id="106">ScriptSource, Galician written with Latin script https://scriptsource.org/cms/scripts/page.php?item_id=wrSys_detail&amp;key=gl-Latn (Accessed on 1 May 2023)</reference>
<reference id="107">Omniglot, Lule Sámi (julevsámegiella) https://www.omniglot.com/writing/lulesami.htm (Accessed on 31 August 2018)</reference>
<reference id="108">Wikipedia, Northern Sami https://en.wikipedia.org/wiki/Northern_Sami (Accessed on 4 September 2018)</reference>
<reference id="109">Omniglot, Vietnamese (tiếng việt / 㗂越) https://www.omniglot.com/writing/vietnamese.htm (Accessed on 4 September 2018)</reference>
<reference id="110">Omniglot, Romanian (limba română) https://www.omniglot.com/writing/romanian.htm (Accessed on 4 September 2018)</reference>
<reference id="113">Omniglot, Skolt Sámi (Sääˊmǩiõll / Nuõrttsää’m) https://www.omniglot.com/writing/skoltsami.htm (Accessed on 4 September 2018)</reference>
<reference id="114">Omniglot, French (français) https://www.omniglot.com/writing/french.htm (Accessed on 4 September 2018)</reference>
<reference id="115">Omniglot, West Frisian (Frysk) https://www.omniglot.com/writing/westfrisian.htm (Accessed on 4 September 2018)</reference>
<reference id="116">Omniglot, Friulian (furlan/marilenghe) https://www.omniglot.com/writing/friulian.htm (Accessed on 4 September 2018)</reference>
<reference id="117">Summer Institute of Linguistics, Pequeno dicionário: Xavante-Português, Português-Xavante https://www.sil.org/resources/archives/17019 (Accessed on 1 October 2020)</reference>
<reference id="119">Omniglot, German (Deutsch) https://www.omniglot.com/writing/german.htm (Accessed on 4 September 2018)</reference>
<reference id="120">Omniglot, Finnish (suomi) https://www.omniglot.com/writing/finnish.htm (Accessed on 4 September 2018)</reference>
<reference id="121">Omniglot, Turkmen (Türkmen dili / Түркмен дили) https://www.omniglot.com/writing/turkmen.htm (Accessed on 4 September 2018)</reference>
<reference id="122">Omniglot, Estonian (eesti keel) https://www.omniglot.com/writing/estonian.htm (Accessed on 4 September 2018)</reference>
<reference id="123">Omniglot, Swedish (svenska) https://www.omniglot.com/writing/swedish.htm (Accessed on 4 September 2018)</reference>
<reference id="124">Omniglot, Yapese (Waab) https://www.omniglot.com/writing/yapese.htm (Accessed on 4 September 2018)</reference>
<reference id="125">Omniglot, Dinka (Thuɔŋjäŋ) https://www.omniglot.com/writing/dinka.php (Accessed on 4 September 2018)</reference>
<reference id="126">Omniglot, Kaqchikel (Kaqchikel Chab’äl) https://www.omniglot.com/writing/kaqchikel.htm (Accessed on 4 September 2018)</reference>
<reference id="127">Omniglot, Bashkir/Bashkort (Башҡорт теле / Başqort tele) https://www.omniglot.com/writing/bashkir.htm (Accessed on 4 September 2018)</reference>
<reference id="128">Omniglot, Alsatian (Ëlsässisch) https://www.omniglot.com/writing/alsatian.htm (Accessed on 4 September 2018)</reference>
<reference id="129">Wikipedia, Nuer language https://en.wikipedia.org/wiki/Nuer_language (Accessed on 4 September 2018)</reference>
<reference id="130">Omniglot, Italian (italiano) https://www.omniglot.com/writing/italian.htm (Accessed on 4 September 2018)</reference>
<reference id="131">Wikipedia, Italian orthography https://en.wikipedia.org/wiki/Italian_orthography (Accessed on 4 September 2018)</reference>
<reference id="132">Omniglot, Wolof (Wollof) https://www.omniglot.com/writing/wolof.htm (Accessed on 4 September 2018)</reference>
<reference id="133">Omniglot, Latvian (latviešu valoda) https://www.omniglot.com/writing/latvian.htm (Accessed on 4 September 2018)</reference>
<reference id="134">Omniglot, Tongan (Faka-Tonga) https://www.omniglot.com/writing/tongan.htm (Accessed on 4 September 2018)</reference>
<reference id="135">Omniglot, Hawaiian (ʻŌlelo Hawaiʻi) https://www.omniglot.com/writing/hawaiian.htm (Accessed on 4 September 2018)</reference>
<reference id="136">Omniglot, Marshallese (kajin m̧ajeļ) https://www.omniglot.com/writing/marshallese.php (Accessed on 4 September 2018)</reference>
<reference id="137">Omniglot, Polish (polski) https://www.omniglot.com/writing/polish.htm (Accessed on 4 September 2018)</reference>
<reference id="138">Omniglot, Lithuanian (lietuvių kalba) https://www.omniglot.com/writing/lithuanian.htm (Accessed on 4 September 2018)</reference>
<reference id="139">Omniglot, Danish (dansk) https://www.omniglot.com/writing/danish.htm (Accessed on 4 September 2018)</reference>
<reference id="140">Omniglot, Chamorro (chamoru) https://www.omniglot.com/writing/chamorro.htm (Accessed on 4 September 2018)</reference>
<reference id="141">Omniglot, Umbundu (Úmbúndú) https://www.omniglot.com/writing/umbundu.htm (Accessed on 4 September 2018)</reference>
<reference id="142">Omniglot, Guaraní (Avañe’ẽ) https://www.omniglot.com/writing/guarani.htm (Accessed on 4 September 2018)</reference>
<reference id="143">Wikipedia, Guarani alphabet https://en.wikipedia.org/wiki/Guarani_alphabet (Accessed on 4 September 2018)</reference>
<reference id="144">Omniglot, Nauruan (Ekaiairũ Naoero) https://www.omniglot.com/writing/nauruan.htm (Accessed on 4 September 2018)</reference>
<reference id="145">Omniglot, Khoekhoe (Khoekhoegowab) https://www.omniglot.com/writing/khoekhoe.htm (Accessed on 4 September 2018)</reference>
<reference id="146">Omniglot, Nuer (Naath) https://www.omniglot.com/writing/nuer.htm (Accessed on 4 September 2018)</reference>
<reference id="147">Omniglot, Hausa (Harshen Hausa / هَرْشَن هَوْسَ) https://www.omniglot.com/writing/hausa.htm (Accessed on 4 September 2018)</reference>
<reference id="148">Omniglot, Dagaare https://www.omniglot.com/writing/dagaare.htm (Accessed on 4 September 2018)</reference>
<reference id="149">Omniglot, Fula (Fulfulde, Pulaar, PularFulaare) https://www.omniglot.com/writing/fula.htm (Accessed on 4 September 2018)</reference>
<reference id="150">Omniglot, Croatian (Hrvatski) https://www.omniglot.com/writing/croatian.htm (Accessed on 4 September 2018)</reference>
<reference id="151">Omniglot, Serbian (српски / srpski) https://www.omniglot.com/writing/serbian.htm (Accessed on 4 September 2018)</reference>
<reference id="152">Wikipedia, Polish language https://en.wikipedia.org/wiki/Polish_language (Accessed on 4 September 2018)</reference>
<reference id="153">Omniglot, Slovak (slovenčina) https://www.omniglot.com/writing/slovak.htm (Accessed on 4 September 2018)</reference>
<reference id="154">Evertype Publishing, Lithuanian lietuvių kalba Version 1.1 https://www.evertype.com/alphabets/lithuanian.pdf (Accessed on 4 September 2018)</reference>
<reference id="157">Omniglot, Turkish (Türkçe) https://www.omniglot.com/writing/turkish.htm (Accessed on 4 September 2018)</reference>
<reference id="158">Omniglot, Kurdish (Kurdî / کوردی) https://www.omniglot.com/writing/kurdish.htm (Accessed on 4 September 2018)</reference>
<reference id="159">Omniglot, Azerbaijani (آذربايجانجا ديلي / Азәрбајҹан дили / Azərbaycan dili) https://www.omniglot.com/writing/azeri.htm (Accessed on 4 September 2018)</reference>
<reference id="160">Omniglot, Basque (euskara) https://www.omniglot.com/writing/basque.htm (Accessed on 4 September 2018)</reference>
<reference id="161">Wikipedia, Basque language https://en.wikipedia.org/wiki/Basque_language#Writing_system (Accessed on 4 September 2018)</reference>
<reference id="163">Omniglot, Maltese (Malti) https://www.omniglot.com/writing/maltese.htm (Accessed on 4 September 2018)</reference>
<reference id="164">Omniglot, Venda (Tshivenḓa / Luvenḓa) https://www.omniglot.com/writing/venda.htm (Accessed on 4 September 2018)</reference>
<reference id="168">Omniglot, Brahui (Bráhuí / براوی) https://www.omniglot.com/writing/brahui.htm (Accessed on 4 September 2018)</reference>
<reference id="169">Wikipedia, Fon language https://en.wikipedia.org/wiki/Fon_language (Accessed on 4 September 2018)</reference>
<reference id="170">Omniglot, Ewe (Eʋegbe) https://www.omniglot.com/writing/ewe.htm (Accessed on 4 September 2018)</reference>
<reference id="172">Omniglot, Sorbian (hornjoserbsce/dolnoserbski) https://www.omniglot.com/writing/sorbian.htm (Accessed on 4 September 2018)</reference>
<reference id="173">Peace corps, Botswana, An Introduction to Setswana Language https://files.peacecorps.gov/multimedia/audio/languagelessons/botswana/Bw_Setswana_Language_Lessons.pdf (Accessed on 4 September 2018)</reference>
<reference id="174">Omniglot, Tswana (Setswana) https://www.omniglot.com/writing/tswana.php (Accessed on 4 September 2018)</reference>
<reference id="175">Wikipedia, Afrikaans https://en.wikipedia.org/wiki/Afrikaans (Accessed on 4 September 2018)</reference>
<reference id="176">Omniglot, Albanian (shqip / gjuha shqipe) https://www.omniglot.com/writing/albanian.htm (Accessed on 4 September 2018)</reference>
<reference id="177">Wikipedia, Albanian alphabet https://en.wikipedia.org/wiki/Albanian_alphabet (Accessed on 4 September 2018)</reference>
<reference id="179">Wikipedia, Uyghur Latin alphabet https://en.wikipedia.org/wiki/Uyghur_Latin_alphabet (Accessed on 4 September 2018)</reference>
<reference id="180">Omniglot, Drehu (Deʼu) https://www.omniglot.com/writing/drehu.php (Accessed on 4 September 2018)</reference>
<reference id="182">Omniglot, Haitian Creole (Kreyòl ayisyen) https://www.omniglot.com/writing/haitiancreole.htm (Accessed on 4 September 2018)</reference>
<reference id="183">Wikipedia, Haitian Creole https://en.wikipedia.org/wiki/Haitian_Creole#Orthography (Accessed on 4 September 2018)</reference>
<reference id="184">Omniglot, Minangkabau (Baso Minangkabau / باسو مينڠكاباو) https://www.omniglot.com/writing/minangkabau.htm (Accessed on 4 September 2018)</reference>
<reference id="185">Omniglot, Palauan (a tekoi er a Belau) https://www.omniglot.com/writing/palauan.htm (Accessed on 4 September 2018)</reference>
<reference id="186">Omniglot, Cubeo (pãmié) https://www.omniglot.com/writing/cubeo.htm (Accessed on 4 September 2018)</reference>
<reference id="187">Editorial Alberto Lleras Camargo, Diccionario Ilustrado Bilingüe cubeo-español español-cubeo https://www.sil.org/system/files/reapdata/10/58/27/10582785843693992331766506069073895620/40337_01.pdf (Accessed on 4 September 2018)</reference>
<reference id="188">Omniglot, Inari Saami (Anarâškielâ) https://www.omniglot.com/writing/inarisami.htm (Accessed on 4 September 2018)</reference>
<reference id="189">Omniglot, Compiled by Wolfram Siegel, DAGBANI https://www.omniglot.com/charts/dagbani.pdf (Accessed on 4 September 2018)</reference>
<reference id="190">Omniglot, Ewondo https://www.omniglot.com/writing/ewondo.php (Accessed on 4 September 2018)</reference>
<reference id="191">Omniglot, Luganda (Oluganda) https://www.omniglot.com/writing/ganda.php (Accessed on 4 September 2018)</reference>
<reference id="192">Omniglot, Adzera https://www.omniglot.com/writing/adzera.htm (Accessed on 4 September 2018)</reference>
<reference id="193">Omniglot, Ga (Gã) https://www.omniglot.com/writing/ga.htm (Accessed on 4 September 2018)</reference>
<reference id="194">Omniglot, Duala (Duálá) https://www.omniglot.com/writing/duala.php (Accessed on 4 September 2018)</reference>
<reference id="195">Omniglot, Soga (Lusoga) https://www.omniglot.com/writing/soga.htm (Accessed on 4 September 2018)</reference>
<reference id="196">Omniglot, Alur (Lur) https://www.omniglot.com/writing/alur.htm (Accessed on 4 September 2018)</reference>
<reference id="197">Omniglot, Mandinka (Mandinka kango / لغة مندنكا) https://www.omniglot.com/writing/mandinka.htm (Accessed on 4 September 2018)</reference>
<reference id="198">Omniglot, Acholi (Lwo) https://www.omniglot.com/writing/acholi.htm (Accessed on 4 September 2018)</reference>
<reference id="199">Omniglot, Bambara (Bamanankan) https://www.omniglot.com/writing/bambara.htm (Accessed on 4 September 2018)</reference>
<reference id="200">Omniglot, Raga (Hano) https://www.omniglot.com/writing/raga.htm (Accessed on 4 September 2018)</reference>
<reference id="201">Omniglot, Tatar (tatarça / татарча / تاتارچا) https://www.omniglot.com/writing/tatar.htm (Accessed on 4 September 2018)</reference>
<reference id="202">Omniglot, Zaza (Zazaki / زازاکی) https://www.omniglot.com/writing/zazaki.htm (Accessed on 4 September 2018)</reference>
<reference id="203">Wikipedia, Turkish alphabet https://en.wikipedia.org/wiki/Turkish_alphabet (Accessed on 4 September 2018)</reference>
<reference id="204">School of English, Adam Michiewicz University, Poznań, Poland, Poznań Studies in Contemporary Linguistics 43(1),2007, pp. 169-180, A Demographic Igbo Orthography https://www.degruyter.com/downloadpdf/j/psicl.2007.43.issue-1/v10010-007-0009-0/v10010-007-0009-0.pdf (Accessed on 4 September 2018)</reference>
<reference id="205">Omniglot, Igbo (Asụsụ Igbo) https://www.omniglot.com/writing/igbo.htm (Accessed on 4 September 2018)</reference>
<reference id="206">ItalianPod101, Italian Accents and Proper Italian Pronunciation https://www.italianpod101.com/italian-accents (Accessed on 4 September 2018)</reference>
<reference id="208">Reverso Dictionary, venerdì translation | Italian-English dictionary https://dictionary.reverso.net/italian-english/venerd%C3%AC (Accessed on 4 September 2018)</reference>
<reference id="209">Omniglot, Kikuyu (Gĩkũyũ) https://www.omniglot.com/writing/kikuyu.htm (Accessed on 4 September 2018)</reference>
<reference id="210">Omniglot, Hixkaryána https://www.omniglot.com/writing/hixkaryana.htm (Accessed on 4 September 2018)</reference>
<reference id="211">Omniglot, Maasai (ɔl Maa) https://www.omniglot.com/writing/maasai.htm (Accessed on 4 September 2018)</reference>
<reference id="212">Omniglot, Mossi (Mòoré) https://www.omniglot.com/writing/mossi.htm (Accessed on 4 September 2018)</reference>
<reference id="213">Omniglot, Jenesis. The Bible in Marshallese, 2009., Contributed by Wolfgang Kuhl https://www.omniglot.com/babel/marshallese.htm (Accessed on 4 September 2018)</reference>
<reference id="214">Wikipedia, Cedilla https://en.wikipedia.org/wiki/Cedilla#Marshallese (Accessed on 4 September 2018)</reference>
<reference id="215">Wikipedia, Marshallese language https://en.wikipedia.org/wiki/Marshallese_language#Display_issues (Accessed on 4 September 2018)</reference>
<reference id="216">Trussel, Marshallese-English Online Dictionary https://www.trussel2.com/MOD/ (Accessed on 4 September 2018)</reference>
<reference id="218">Omniglot, Susu (Sosoxi) https://www.omniglot.com/writing/susu.htm (Accessed on 4 September 2018)</reference>
<reference id="219">Omniglot, Zarma (Zarmaciine) https://www.omniglot.com/writing/zarma.htm (Accessed on 4 September 2018)</reference>
<reference id="220">Omniglot, Pitjantjatjara https://www.omniglot.com/writing/pitjantjatjara.htm (Accessed on 4 September 2018)</reference>
<reference id="221">Omniglot, Spanish (español/castellano) https://www.omniglot.com/writing/spanish.htm (Accessed on 4 September 2018)</reference>
<reference id="222">Omniglot, Filipino (wikang Filipino) https://www.omniglot.com/writing/filipino.htm (Accessed on 4 September 2018)</reference>
<reference id="223">Omniglot, Chavacano https://www.omniglot.com/writing/chavacano.php (Accessed on 4 September 2018)</reference>
<reference id="224">Wikipedia, Ilocano language https://en.wikipedia.org/wiki/Ilocano_language#Modern_alphabet (Accessed on 4 September 2018)</reference>
<reference id="225">Omniglot, Quechua (Runasimi) https://www.omniglot.com/writing/quechua.htm (Accessed on 4 September 2018)</reference>
<reference id="226">Wikipedia, Quechua alphabet https://en.wikipedia.org/wiki/Quechua_alphabet (Accessed on 4 September 2018)</reference>
<reference id="227">Omniglot, Cape Verdean Creole (Kriolu) https://www.omniglot.com/writing/kriol.php (Accessed on 4 September 2018)</reference>
<reference id="228">Omniglot, Waray-Waray https://www.omniglot.com/writing/waray.php (Accessed on 4 September 2018)</reference>
<reference id="229">Omniglot, Lozi (siLozi) https://www.omniglot.com/writing/lozi.htm (Accessed on 4 September 2018)</reference>
<reference id="230">africanlanguages.com, Sesotho sa Leboa (Northern Sotho) https://africanlanguages.com/northern_sotho/ (Accessed on 4 September 2018)</reference>
<reference id="232">Wikipedia, Chechen language https://en.wikipedia.org/wiki/Chechen_language (Accessed on 4 September 2018)</reference>
<reference id="233">Omniglot, Hungarian (magyar) https://www.omniglot.com/writing/hungarian.htm (Accessed on 4 September 2018)</reference>
<reference id="234">Wikipedia, Hungarian alphabet https://en.wikipedia.org/wiki/Hungarian_alphabet (Accessed on 4 September 2018)</reference>
<reference id="236">Omniglot, Lingala https://www.omniglot.com/writing/lingala.htm (Accessed on 4 September 2018)</reference>
<reference id="237">Omniglot, Akan https://www.omniglot.com/writing/akan.htm (Accessed on 4 September 2018)</reference>
<reference id="238">Wikipedia, Mossi language https://en.wikipedia.org/wiki/Mossi_language (Accessed on 4 September 2018)</reference>
<reference id="239">SIL-Sudan, OCCASIONAL PAPERS in the study of SUDANESE LANGUAGES No. 9 (p. 75) https://www.sil.org/system/files/reapdata/10/06/46/100646256099282892829790816212446104791/OPSL_9.pdf (Accessed on 4 September 2018)</reference>
<reference id="240">Omniglot, Kanuri https://www.omniglot.com/writing/kanuri.htm (Accessed on 4 September 2018)</reference>
<reference id="241">Omniglot, Bugis (Basa Ugi ) https://www.omniglot.com/writing/bugis.htm (Accessed on 4 September 2018)</reference>
<reference id="242">Omniglot, Mizo (Mizo ṭawng) https://www.omniglot.com/writing/mizo.htm (Accessed on 4 September 2018)</reference>
<reference id="243">Omniglot, Miskito (Mískitu) https://www.omniglot.com/writing/miskito.htm (Accessed on 4 September 2018)</reference>
<reference id="245">Wikipedia, Papiamento https://en.wikipedia.org/wiki/Papiamento (Accessed on 4 September 2018)</reference>
<reference id="246">Omniglot, Papiamento (Papiamentu) https://www.omniglot.com/writing/papiamento.php (Accessed on 4 September 2018)</reference>
<reference id="247">Omniglot, Chichewa (Chicheŵa) https://www.omniglot.com/writing/chichewa.php (Accessed on 4 September 2018)</reference>
<reference id="248">Native Languages of the Americas website, Vocabulary in Native American Languages: Mam Words https://www.native-languages.org/mam_words.htm (Accessed on 4 September 2018)</reference>
<reference id="249">Omniglot, Mam (Qyol Mam) https://www.omniglot.com/writing/mam.htm (Accessed on 4 September 2018)</reference>
<reference id="250">Wikipedia, Pulaar language https://en.wikipedia.org/wiki/Pulaar_language (Accessed on 4 September 2018)</reference>
<reference id="251">Wikipedia, Fula language https://en.wikipedia.org/wiki/Fula_language#Writing_systems (Accessed on 4 September 2018)</reference>
<reference id="252">Wikipedia, Polish alphabet https://en.wikipedia.org/wiki/Polish_alphabet (Accessed on 4 September 2018)</reference>
<reference id="253">Wikipedia, French orthography https://en.wikipedia.org/wiki/French_orthography (Accessed on 4 September 2018)</reference>
<reference id="254">Omniglot, Yoruba (Èdè Yorùbá) https://www.omniglot.com/writing/yoruba.htm (Accessed on 4 September 2018)</reference>
<reference id="255">Omniglot, Esperanto https://www.omniglot.com/writing/esperanto.htm (Accessed on 4 September 2018)</reference>
<reference id="256">Omniglot, Welsh (Cymraeg) https://www.omniglot.com/writing/welsh.htm (Accessed on 4 September 2018)</reference>
<reference id="257">Wikipedia, List of Latin-script letters https://en.wikipedia.org/wiki/List_of_Latin-script_letters (Accessed on 4 September 2018)</reference>
<reference id="258">Omniglot, Montenegrin https://www.omniglot.com/writing/montenegrin.htm (Accessed on 20 March 2019)</reference>
<reference id="275">Omniglot, Shavante https://www.omniglot.com/writing/shavante.php (Accessed on 24 September 2020)</reference>
<reference id="276">Wikipedia, Malagasy Language https://en.wikipedia.org/wiki/Malagasy_language (Accessed on 24 September 2020)</reference>
<reference id="277">Wikipedia, Serer language, https://en.wikipedia.org/wiki/Serer_language, accessed on 6 April 2021</reference>
<reference id="278">Wikipedia, Kpelle language, https://en.wikipedia.org/wiki/Kpelle_language, accessed on 6 April 2021</reference>
<reference id="279">Wikipedia, Catalan Orthography, https://en.wikipedia.org/wiki/Catalan_orthography, accessed on 28 November 2022</reference>
<reference id="280">Fundacio PuntCAT registry (.CAT), IDN table for CAT_ca Version 1.0, 12 February 2006, https://www.iana.org/domains/idn-tables/tables/cat_ca_1.0.html</reference>
<reference id="281">Fundacio PuntCAT registry (.CAT), Rules of the .cat domain, https://domini.cat/en/rules-of-the-cat-domain/</reference>
<reference id="320">RFC 5891, Internationalized Domain Names in Applications (IDNA): Protocol https://tools.ietf.org/html/rfc5891</reference>
</references>
</meta>
<data>
<char cp="002D" not-when="hyphen-minus-disallowed" tag="sc:Zyyy" ref="0" comment="HYPHEN-MINUS; &#x235F;" />
<char cp="0030" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT ZERO; &#x235F;" />
<char cp="0031" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT ONE; &#x235F;" />
<char cp="0032" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT TWO; &#x235F;" />
<char cp="0033" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT THREE; &#x235F;" />
<char cp="0034" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT FOUR; &#x235F;" />
<char cp="0035" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT FIVE; &#x235F;" />
<char cp="0036" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT SIX; &#x235F;" />
<char cp="0037" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT SEVEN; &#x235F;" />
<char cp="0038" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT EIGHT; &#x235F;" />
<char cp="0039" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT NINE; &#x235F;" />
<char cp="0061" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0062" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0063" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0064" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0065" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0066" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0067" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0068" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0069" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="006A" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="006B" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="006C" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="006D" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="006E" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="006F" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0070" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0071" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0072" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0073" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0074" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0075" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0076" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0077" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0078" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0079" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="007A" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="00E0" tag="sc:Latn" ref="0 106 114 130 131 132" comment="Italian (1), French (1), Galician (2), Wolof (4)" />
<char cp="00E2" tag="sc:Latn" ref="0 106 109 110 113 114 115 116 117 275" comment="Vietnamese (1), Romanian (1), Skolt Sami (2), French (1), Galician (2), West Frisian (1), Friulian (4), Xavante (4)" />
<char cp="00E3" tag="sc:Latn" ref="0 141 142 143 144 145" comment="Umbundu (3), Guarani (1), Nauruan (3), Khoekhoe (4)" />
<char cp="00E4" tag="sc:Latn" ref="0 107 119 120 121 122 123 124 125 126 127 128 129" comment="German (1), Finnish (1), Turkmen (1), Estonian (1), Swedish (1), Lule Sami (2), Yapese (2), Dinka (4), Kaqchikel (4), Bashkir (4), Alsatian (5), Nuer (4)" />
<char cp="00E5" tag="sc:Latn" ref="0 107 120 123 139 140" comment="Danish (1), Finnish (1), Chamorro (1), Swedish (1), Lule Sami (2)" />
<char cp="00E6" tag="sc:Latn" ref="0 102 103 139" comment="Danish (1), Icelandic (1), Faroese (2)" />
<char cp="00E7" tag="sc:Latn" ref="0 106 114 116 121 127 157 158 159 160 161" comment="Turkish (1), Turkmen (1), Kurdish (2), French (1), Azerbaijani (1), Basque (1), Galician (2), Friulian (4), Bashkir (4)" />
<char cp="00E8" tag="sc:Latn" ref="0 114 130 175 182 183" comment="French (1), Italian (1), Afrikaans (1), Haitian Creole (1), French (1)" />
<char cp="00E9" tag="sc:Latn" ref="0 100 101 102 105 106 114 115 117 130 132 275" comment="French (1), Italian (1), Spanish (1), Czech (1), Icelandic (1), Chuukese (2), Galician (2), Wolof (4), Xavante (4), West Frisian (2)" />
<char cp="00EA" tag="sc:Latn" ref="0 109 114 115 116 158 173 174 175" comment="French (1), Tswana (1), Afrikaans (1), Vietnamese (1), Kurdish (2), West Frisian (2), Friulian (4)" />
<char cp="00EB" tag="sc:Latn" ref="0 114 115 124 126 129 132 175 176 177 179 180" comment="Afrikaans (1), Albanian (1), French (1), Uyghur (2), Yapese (2), Wolof (4), Drehu (4), Kaqchikel (4), West Frisian (2), Nuer (4)" />
<char cp="00EC" tag="sc:Latn" ref="0 130 206 208" comment="Italian (1)" />
<char cp="00EE" tag="sc:Latn" ref="0 110 114 116 158 175" comment="Afrikaans (1), Romanian (1), Kurdish (2), French (1), Friulian (4)" />
<char cp="00F0" tag="sc:Latn" ref="0 102 103" comment="Faroese (2), Icelandic (1)" />
<char cp="00F2" tag="sc:Latn" ref="0 130 182 183" comment="Italian (1), Haitian Creole (1)" />
<char cp="00F4" tag="sc:Latn" ref="0 106 109 114 115 116 117 173 174 175 230 275" comment="Tswana (1), Afrikaans (1), Vietnamese (1), French (1), Northern Sotho (1), West Frisian (2), Galician (2), Friulian (4), Xavante (4)" />
<char cp="00F5" tag="sc:Latn" ref="0 113 117 122 141 142 143 144 145 275" comment="Estonian (1), Skolt Sami (2), Umbundu (3), Guarani (1), Nauruan (3), Xavante (4), Khoekhoe (4)" />
<char cp="00F6" tag="sc:Latn" ref="0 115 119 120 123 124 125 126 127 129 157 175 179 180 232" comment="German (1), Finnish (1), Afrikaans (1), Turkish (1), Swedish (1), Uygur (2), Yapese (2), Drehu (4), Kaqchikel (4), Dinka (4), Bashkir (4), Chechen (2), 1992 Version, West Frisian (2), Nuer (4)" />
<char cp="00F8" tag="sc:Latn" ref="0 103 139" comment="Danish (1), Faroese (2)" />
<char cp="00F9" tag="sc:Latn" ref="0 114 130 206 245 246 253" comment="Italian (1), French (1), Papiamento (1)" />
<char cp="00FB" tag="sc:Latn" ref="0 114 115 116 158 175 202 243" comment="Afrikaans (1), Kurdish (2), French (1), Miskito (2), West Frisian (2), Friulian (4), Zazaki (4)" />
<char cp="00FD" tag="sc:Latn" ref="0 101 102 103 121 142 143" comment="Turkmen (1), Czech (1), Icelandic (1), Faroese (2), Guarani (1)" />
<char cp="00FE" tag="sc:Latn" ref="0 102" comment="Icelandic (1)" />
<char cp="00FF" tag="sc:Latn" ref="0 114 253 257" comment="French (1)" />
<char cp="0103" tag="sc:Latn" ref="0 109 110" comment="Vietnamese (1), Romanian (1)" />
<char cp="0105" tag="sc:Latn" ref="0 137 138" comment="Polish (1), Lithuanian (1)" />
<char cp="0107" tag="sc:Latn" ref="0 150 151 152" comment="Croatian (1), Serbian (1), Polish (1)" />
<char cp="0109" tag="sc:Latn" ref="0 255" comment="Esperanto (3)" />
<char cp="010D" tag="sc:Latn" ref="0 108 133 150 151 153 154" comment="Croatian (1), Serbian (1), Latvian (1), Slovak (1), Northern Sami (2), Lithuanian (1)" />
<char cp="010F" tag="sc:Latn" ref="0 101 153" comment="Czech (1), Slovak (1)" />
<char cp="0111" tag="sc:Latn" ref="0 108 109 150 151 168" comment="Croatian (1), Serbian (1), Vietnamese (1), Northern Sami (2), Brahui (5)" />
<char cp="0113" tag="sc:Latn" ref="0 133 134 135 184" comment="Latvian (1), Hawaiian (2), Tongan (1), Minangkabau (5)" />
<char cp="0117" tag="sc:Latn" ref="0 138 154" comment="Lithuanian (1)" />
<char cp="0119" tag="sc:Latn" ref="0 138 152 154 185" comment="Polish (1), Palauan (2), Lithuanian (1)" />
<char cp="011B" tag="sc:Latn" ref="0 101 172" comment="Czech (1), Sorbian (4)" />
<char cp="011D" tag="sc:Latn" ref="0 255" comment="Esperanto (3)" />
<char cp="011F" tag="sc:Latn" ref="0 127 157 159 201 202" comment="Turkish (1), Tatar (2), Azeri (1), Bashkir (4), Zaza (5)" />
<char cp="0123" tag="sc:Latn" ref="0 133 168" comment="Latvian (1), Brahui (5)" />
<char cp="0125" tag="sc:Latn" ref="0 255" comment="Esperanto (3)" />
<char cp="0127" tag="sc:Latn" ref="0 163" comment="Maltese (1)" />
<char cp="012B" tag="sc:Latn" ref="0 133 134 135 138" comment="Latvian (1), Lithuanian (1), Hawaiian (2), Tongan (1)" />
<char cp="012F" tag="sc:Latn" ref="0 154" comment="Lithuanian (1)" />
<char cp="0135" tag="sc:Latn" ref="0 255" comment="Esperanto (3)" />
<char cp="0137" tag="sc:Latn" ref="0 133" comment="Latvian (1)" />
<char cp="013A" tag="sc:Latn" ref="0 153" comment="Slovak (1)" />
<char cp="013C" tag="sc:Latn" ref="0 133 168 213 214" comment="Latvian (1), Marshallese (1), Brahui (5)" />
<char cp="013E" tag="sc:Latn" ref="0 153" comment="Slovak (1)" />
<char cp="0142" tag="sc:Latn" ref="0 152" comment="Polish (1)" />
<char cp="0146" tag="sc:Latn" ref="0 133 136" comment="Latvian (1), Marshallese (1)" />
<char cp="0148" tag="sc:Latn" ref="0 101 121 153" comment="Turkmen (1), Czech (1), Slovak (1)" />
<char cp="0151" tag="sc:Latn" ref="0 233 234" comment="Hungarian (1)" />
<char cp="0153" tag="sc:Latn" ref="0 114 253" comment="French (1)" />
<char cp="0155" tag="sc:Latn" ref="0 153 168" comment="Slovak (1), Brahui (5)" />
<char cp="0159" tag="sc:Latn" ref="0 101 172" comment="Czech (1), Sorbian (4)" />
<char cp="015B" tag="sc:Latn" ref="0 152 258" comment="Polish (1), Montenegrin (1)" />
<char cp="015D" tag="sc:Latn" ref="0 255" comment="Esperanto (3)" />
<char cp="015F" tag="sc:Latn" ref="0 121 127 157 158 159 168 201 202" comment="Turkish (1), Turkmen (1), Kurdish (2), Tatar (2), Azeri (1), Bashkir (4), Brahui (5), Zaza (5)" />
<char cp="0161" tag="sc:Latn" ref="0 108 133 150 151 154 174 230" comment="Tswana (1), Croatian (1), Serbian (1), Latvian (1), Northern Sotho (1), Northern Sami (2), Lithuanian (1)" />
<char cp="0165" tag="sc:Latn" ref="0 101 153" comment="Czech (1), Slovak (1)" />
<char cp="0167" tag="sc:Latn" ref="0 108 168" comment="Northern Sami (2), Brahui (5)" />
<char cp="016B" tag="sc:Latn" ref="0 133 134 135 136 138 154" comment="Latvian (1), Hawaiian (2), Lithuanian (1), Marshallese (1), Tongan (1)" />
<char cp="016D" tag="sc:Latn" ref="0 255" comment="Esperanto (3)" />
<char cp="016F" tag="sc:Latn" ref="0 101" comment="Czech (1)" />
<char cp="0171" tag="sc:Latn" ref="0 233 234" comment="Hungarian (1)" />
<char cp="0173" tag="sc:Latn" ref="0 138 154" comment="Lithuanian (1)" />
<char cp="0175" tag="sc:Latn" ref="0 247 256" comment="Chichewa (3), Welsh (2)" />
<char cp="0177" tag="sc:Latn" ref="0 256" comment="Welsh (2)" />
<char cp="017A" tag="sc:Latn" ref="0 152 168 172 252 258" comment="Polish (1), Brahui (5), Sorbian (4), Montenegrin (1)" />
<char cp="017E" tag="sc:Latn" ref="0 108 121 133 150 151 153 154 232" comment="Lithuanian (1), Croatian (1), Serbian (1), Turkmen (1), Latvian (1), Slovak (1), Northern Sami (2), Chechen (2) 1925 Version" />
<char cp="0188" tag="sc:Latn" ref="0 277" comment="Serer (5)" />
<char cp="0199" tag="sc:Latn" ref="0 147" comment="Hausa (2)" />
<char cp="01A1" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="01A5" tag="sc:Latn" ref="0 277" comment="Serer (5)" />
<char cp="01AD" tag="sc:Latn" ref="0 277" comment="Serer (5)" />
<char cp="01B0" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="01B4" tag="sc:Latn" ref="0 148 149 251" comment="Dagaare - Burkina Faso (4), Fula (3)" />
<char cp="01E9" tag="sc:Latn" ref="0 113" comment="Skolt Sami (2)" />
<char cp="01EF" tag="sc:Latn" ref="0 113" comment="Skolt Sami (2)" />
<char cp="0219" tag="sc:Latn" ref="3 110" comment="Romanian (1)" />
<char cp="021B" tag="sc:Latn" ref="3 110" comment="Romanian (1)" />
<char cp="024D" tag="sc:Latn" ref="8 240" comment="Kanuri (3)" />
<char cp="0253" tag="sc:Latn" ref="0 147 148 250" comment="Hausa (2), Dagaare - Burkina Faso (4), Pulaar (3)" />
<char cp="0254" tag="sc:Latn" ref="0 129 146 148 169 170 189 190 193 194 236 237" comment="Dagaare - Burkina Faso (4), Dagbani (Dagomba) (4), Lingala (2), Akan (3), Ewondo (3), Fon (3), Nuer (4), Ga (4), Duala (3), EWE (3), Nuer (4)" />
<char cp="0256" tag="sc:Latn" ref="0 169 170" comment="Fon (3), Ewe (3)" />
<char cp="0257" tag="sc:Latn" ref="0 147 149 250" comment="Hausa (2), Fula (3)" />
<char cp="0259" tag="sc:Latn" ref="0 159 170 190 241" comment="Azeri, Azerbaijani (1), Ewondo (3), Ewe (3), Bugis (3)" />
<char cp="025B" tag="sc:Latn" ref="0 129 148 169 170 189 190 193 194 199 212 236 237 238" comment="Dagaare - Burkina Faso (4), Lingala (2), Akan (3), Ewondo (3), Dagbani (Dagomba), (4), Fon (3), Mossi (3), Ga (4), Ewe (3), Duala (3), Bambara (4), Nuer (4)" />
<char cp="0260" tag="sc:Latn" ref="0 278" comment="Kpelle (5)" />
<char cp="0268" tag="sc:Latn" ref="0 186 189 210 211" comment="Cubeo (3), Dagbani (Dagomba) (4), HIxkaryána (4), Maasai (5)" />
<char cp="0272" tag="sc:Latn" ref="0 199 218 219" comment="Susu (4), Zarma (4), Bambara (4)" />
<char cp="0289" tag="sc:Latn" ref="0 186 187 211" comment="Cubeo (3), Maasai (5)" />
<char cp="0292" tag="sc:Latn" ref="0 113 189" comment="Skolt Sami (2), Dagbani (Dagomba) (4)" />
<char cp="1E13" tag="sc:Latn" ref="0 164 257" comment="Venda (1)" />
<char cp="1E3D" tag="sc:Latn" ref="0 164 257" comment="Venda (1)" />
<char cp="1E49" tag="sc:Latn" ref="0 220" comment="Pitjantjatjara (4)" />
<char cp="1E4B" tag="sc:Latn" ref="0 164 257" comment="Venda (1)" />
<char cp="1E63" tag="sc:Latn" ref="0 254" comment="Yoruba (2)" />
<char cp="1E6D" tag="sc:Latn" ref="0 242" comment="Mizo (4)" />
<char cp="1E71" tag="sc:Latn" ref="0 164 257" comment="Venda (1)" />
<char cp="1E8D" tag="sc:Latn" ref="0 248 249" comment="Mam (4)" />
<char cp="1EA1" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EA5" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EA7" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EA9" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EAB" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EAD" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EAF" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EB1" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EB3" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EB5" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EB7" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EB9" tag="sc:Latn" ref="0 254" comment="Yoruba (2)" />
<char cp="1EBB" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EBF" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EC1" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EC3" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EC5" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EC7" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1ECB" tag="sc:Latn" ref="0 205" comment="Igbo (2)" />
<char cp="1ECD" tag="sc:Latn" ref="0 136 204 205 215 216 254" comment="Igbo (2), Yoruba (2), Marshallese (1)" />
<char cp="1ED1" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1ED3" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1ED5" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1ED7" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1ED9" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EDB" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EDD" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EDF" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EE1" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EE3" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EE5" tag="sc:Latn" ref="0 109 204 205" comment="Vietnamese (1), Igbo (2)" />
<char cp="1EE9" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EEB" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EED" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EEF" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EF1" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EF5" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EF9" tag="sc:Latn" ref="0 109 142" comment="Vietnamese (1), Guarani (1)" />
</data>
<!--Rules section goes here-->
<rules>
<!--Character class definitions go here-->
<!--Whole label evaluation and context rules go here-->
<rule name="leading-combining-mark" ref="320" comment="RFC 5891 restrictions on placement of combining marks &#x235F;">
<start />
<union>
<class property="gc:Mn" />
<class property="gc:Mc" />
</union>
</rule>
<rule name="hyphen-minus-disallowed" ref="320" comment="RFC 5891 restrictions on placement of U+002D HYPHEN-MINUS &#x235F;">
<choice>
<rule comment="no leading hyphen">
<look-behind>
<start />
</look-behind>
<anchor />
</rule>
<rule comment="no trailing hyphen">
<anchor />
<look-ahead>
<end />
</look-ahead>
</rule>
<rule comment="no consecutive hyphens in third and fourth">
<look-behind>
<start />
<any />
<any />
<char cp="002D" comment="hyphen-minus" />
</look-behind>
<anchor />
</rule>
</choice>
</rule>
<!--Action elements go here - order defines precedence-->
<action disp="invalid" match="leading-combining-mark" comment="labels with leading combining marks are invalid &#x235F;" />
<action disp="invalid" any-variant="out-of-repertoire-var" comment="any variant label with a code point out of repertoire is invalid &#x235F;" />
<action disp="invalid" match="dot-L-dot" comment="labels with one L sharing two middle dots are invalid" />
<action disp="blocked" any-variant="blocked" comment="any variant label containing blocked variants is blocked &#x235F;" />
<action disp="allocatable" all-variants="allocatable" comment="variant labels with all variants allocatable are allocatable &#x235F;" />
<action disp="allocatable" all-variants="fallback" comment="any label with all variants of type fallback is allocatable &#x235F;" />
<action disp="blocked" any-variant="fallback" comment="any variant label with a mix of variant forms is blocked &#x235F;" />
<action disp="valid" all-variants="r-original" comment="any remaining label containing only original code points is valid &#x235F;" />
<action disp="valid" comment="catch all (default action) &#x235F;" />
</rules>
</lgr>
@@ -86,7 +86,7 @@ public final class ForeignKeyUtils {
*/
public static <E extends EppResource> ImmutableMap<String, VKey<E>> load(
Class<E> clazz, Collection<String> foreignKeys, final DateTime now) {
return load(clazz, foreignKeys, false).entrySet().stream()
return loadMostRecentResources(clazz, foreignKeys, false).entrySet().stream()
.filter(e -> now.isBefore(e.getValue().deletionTime()))
.collect(toImmutableMap(Entry::getKey, e -> VKey.create(clazz, e.getValue().repoId())));
}
@@ -104,8 +104,9 @@ public final class ForeignKeyUtils {
* same max {@code deleteTime}, usually {@code END_OF_TIME}, lest this method throws an error due
* to duplicate keys.
*/
private static <E extends EppResource> ImmutableMap<String, MostRecentResource> load(
Class<E> clazz, Collection<String> foreignKeys, boolean useReplicaTm) {
public static <E extends EppResource>
ImmutableMap<String, MostRecentResource> loadMostRecentResources(
Class<E> clazz, Collection<String> foreignKeys, boolean useReplicaTm) {
String fkProperty = RESOURCE_TYPE_TO_FK_PROPERTY.get(clazz);
JpaTransactionManager tmToUse = useReplicaTm ? replicaTm() : tm();
return tmToUse.reTransact(
@@ -148,7 +149,7 @@ public final class ForeignKeyUtils {
ImmutableList<String> foreignKeys =
keys.stream().map(key -> (String) key.getKey()).collect(toImmutableList());
ImmutableMap<String, MostRecentResource> existingKeys =
ForeignKeyUtils.load(clazz, foreignKeys, true);
ForeignKeyUtils.loadMostRecentResources(clazz, foreignKeys, true);
// The above map only contains keys that exist in the database, so we re-add the
// missing ones with Optional.empty() values for caching.
return Maps.asMap(
@@ -234,7 +235,7 @@ public final class ForeignKeyUtils {
e -> VKey.create(clazz, e.getValue().get().repoId())));
}
record MostRecentResource(String repoId, DateTime deletionTime) {
public record MostRecentResource(String repoId, DateTime deletionTime) {
static MostRecentResource create(String repoId, DateTime deletionTime) {
return new MostRecentResource(repoId, deletionTime);
@@ -101,8 +101,15 @@ public final class RegistryLock extends UpdateAutoTimestampEntity implements Bui
@Column(nullable = false)
private String registrarId;
/** The POC that performed the action, or null if it was a superuser. */
@Expose private String registrarPocId;
/**
* The email address of the user that performed the action, or null if it was a superuser.
*
* <p>Note: this is misnamed in the database due to historical reasons, where we used the
* registrar POC ID as the email address rather than a separate specialized field.
*/
@Column(name = "registrarPocId")
@Expose
private String registryLockEmail;
/** When the lock is first requested. */
@AttributeOverrides({
@@ -161,8 +168,8 @@ public final class RegistryLock extends UpdateAutoTimestampEntity implements Bui
return registrarId;
}
public String getRegistrarPocId() {
return registrarPocId;
public String getRegistryLockEmail() {
return registryLockEmail;
}
public DateTime getLockRequestTime() {
@@ -255,7 +262,7 @@ public final class RegistryLock extends UpdateAutoTimestampEntity implements Bui
checkArgumentNotNull(getInstance().registrarId, "Registrar ID cannot be null");
checkArgumentNotNull(getInstance().verificationCode, "Verification code cannot be null");
checkArgument(
getInstance().registrarPocId != null || getInstance().isSuperuser,
getInstance().registryLockEmail != null || getInstance().isSuperuser,
"Registrar POC ID must be provided if superuser is false");
return super.build();
}
@@ -275,8 +282,8 @@ public final class RegistryLock extends UpdateAutoTimestampEntity implements Bui
return this;
}
public Builder setRegistrarPocId(String registrarPocId) {
getInstance().registrarPocId = registrarPocId;
public Builder setRegistryLockEmail(String registryLockEmail) {
getInstance().registryLockEmail = registryLockEmail;
return this;
}
@@ -14,16 +14,11 @@
package google.registry.model.registrar;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.base.Strings.isNullOrEmpty;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static com.google.common.io.BaseEncoding.base64;
import static google.registry.model.registrar.Registrar.checkValidEmail;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.CollectionUtils.nullToEmptyImmutableSortedCopy;
import static google.registry.util.PasswordUtils.SALT_SUPPLIER;
import static google.registry.util.PasswordUtils.hashPassword;
import static java.util.stream.Collectors.joining;
import com.google.common.annotations.VisibleForTesting;
@@ -38,7 +33,6 @@ import google.registry.model.Jsonifiable;
import google.registry.model.UnsafeSerializable;
import google.registry.persistence.VKey;
import google.registry.persistence.transaction.QueryComposer;
import google.registry.util.PasswordUtils;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.EnumType;
@@ -47,9 +41,7 @@ import jakarta.persistence.Id;
import jakarta.persistence.IdClass;
import java.io.Serializable;
import java.util.Map;
import java.util.Optional;
import java.util.Set;
import javax.annotation.Nullable;
/**
* A contact for a Registrar. Note, equality, hashCode and comparable have been overridden to only
@@ -107,9 +99,6 @@ public class RegistrarPoc extends ImmutableObject implements Jsonifiable, Unsafe
@Id @Expose public String registrarId;
/** External email address of this contact used for registry lock confirmations. */
String registryLockEmailAddress;
/** The voice number of the contact. */
@Expose String phoneNumber;
@@ -147,22 +136,10 @@ public class RegistrarPoc extends ImmutableObject implements Jsonifiable, Unsafe
@Expose
boolean visibleInDomainWhoisAsAbuse = false;
/**
* Whether the contact is allowed to set their registry lock password through the registrar
* console. This will be set to false on contact creation and when the user sets a password.
*/
/** Legacy field, around until we can remove the non-null constraint and the column from SQL. */
@Column(nullable = false)
boolean allowedToSetRegistryLockPassword = false;
/**
* A hashed password that exists iff this contact is registry-lock-enabled. The hash is a base64
* encoded SHA256 string.
*/
String registryLockPasswordHash;
/** Randomly generated hash salt. */
String registryLockPasswordSalt;
/**
* Helper to update the contacts associated with a Registrar. This requires querying for the
* existing contacts, deleting existing contacts that are not part of the given {@code contacts}
@@ -197,10 +174,6 @@ public class RegistrarPoc extends ImmutableObject implements Jsonifiable, Unsafe
return emailAddress;
}
public Optional<String> getRegistryLockEmailAddress() {
return Optional.ofNullable(registryLockEmailAddress);
}
public String getPhoneNumber() {
return phoneNumber;
}
@@ -229,24 +202,6 @@ public class RegistrarPoc extends ImmutableObject implements Jsonifiable, Unsafe
return new Builder(clone(this));
}
public boolean isAllowedToSetRegistryLockPassword() {
return allowedToSetRegistryLockPassword;
}
public boolean isRegistryLockAllowed() {
return !isNullOrEmpty(registryLockPasswordHash) && !isNullOrEmpty(registryLockPasswordSalt);
}
public boolean verifyRegistryLockPassword(String registryLockPassword) {
if (isNullOrEmpty(registryLockPassword)
|| isNullOrEmpty(registryLockPasswordSalt)
|| isNullOrEmpty(registryLockPasswordHash)) {
return false;
}
return PasswordUtils.verifyPassword(
registryLockPassword, registryLockPasswordHash, registryLockPasswordSalt);
}
/**
* Returns a string representation that's human friendly.
*
@@ -296,15 +251,12 @@ public class RegistrarPoc extends ImmutableObject implements Jsonifiable, Unsafe
return new JsonMapBuilder()
.put("name", name)
.put("emailAddress", emailAddress)
.put("registryLockEmailAddress", registryLockEmailAddress)
.put("phoneNumber", phoneNumber)
.put("faxNumber", faxNumber)
.put("types", getTypes().stream().map(Object::toString).collect(joining(",")))
.put("visibleInWhoisAsAdmin", visibleInWhoisAsAdmin)
.put("visibleInWhoisAsTech", visibleInWhoisAsTech)
.put("visibleInDomainWhoisAsAbuse", visibleInDomainWhoisAsAbuse)
.put("allowedToSetRegistryLockPassword", allowedToSetRegistryLockPassword)
.put("registryLockAllowed", isRegistryLockAllowed())
.put("id", getId())
.build();
}
@@ -344,14 +296,6 @@ public class RegistrarPoc extends ImmutableObject implements Jsonifiable, Unsafe
public RegistrarPoc build() {
checkNotNull(getInstance().registrarId, "Registrar ID cannot be null");
checkValidEmail(getInstance().emailAddress);
// Check allowedToSetRegistryLockPassword here because if we want to allow the user to set
// a registry lock password, we must also set up the correct registry lock email concurrently
// or beforehand.
if (getInstance().allowedToSetRegistryLockPassword) {
checkArgument(
!isNullOrEmpty(getInstance().registryLockEmailAddress),
"Registry lock email must not be null if allowing registry lock access");
}
return cloneEmptyToNull(super.build());
}
@@ -365,11 +309,6 @@ public class RegistrarPoc extends ImmutableObject implements Jsonifiable, Unsafe
return this;
}
public Builder setRegistryLockEmailAddress(@Nullable String registryLockEmailAddress) {
getInstance().registryLockEmailAddress = registryLockEmailAddress;
return this;
}
public Builder setPhoneNumber(String phoneNumber) {
getInstance().phoneNumber = phoneNumber;
return this;
@@ -409,28 +348,6 @@ public class RegistrarPoc extends ImmutableObject implements Jsonifiable, Unsafe
getInstance().visibleInDomainWhoisAsAbuse = visible;
return this;
}
public Builder setAllowedToSetRegistryLockPassword(boolean allowedToSetRegistryLockPassword) {
if (allowedToSetRegistryLockPassword) {
getInstance().registryLockPasswordSalt = null;
getInstance().registryLockPasswordHash = null;
}
getInstance().allowedToSetRegistryLockPassword = allowedToSetRegistryLockPassword;
return this;
}
public Builder setRegistryLockPassword(String registryLockPassword) {
checkArgument(
getInstance().allowedToSetRegistryLockPassword,
"Not allowed to set registry lock password for this contact");
checkArgument(
!isNullOrEmpty(registryLockPassword), "Registry lock password was null or empty");
byte[] salt = SALT_SUPPLIER.get();
getInstance().registryLockPasswordSalt = base64().encode(salt);
getInstance().registryLockPasswordHash = hashPassword(registryLockPassword, salt);
getInstance().allowedToSetRegistryLockPassword = false;
return this;
}
}
public static ImmutableList<RegistrarPoc> loadForRegistrar(String registrarId) {
@@ -15,7 +15,6 @@
package google.registry.model.tmch;
import static google.registry.config.RegistryConfig.getClaimsListCacheDuration;
import static google.registry.persistence.transaction.QueryComposer.Comparator.EQ;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
@@ -79,14 +78,11 @@ public class ClaimsListDao {
*/
private static ClaimsList getUncached() {
return tm().reTransact(
() -> {
Long revisionId =
tm().query("SELECT MAX(revisionId) FROM ClaimsList", Long.class)
.getSingleResult();
return tm().createQueryComposer(ClaimsList.class)
.where("revisionId", EQ, revisionId)
.first();
})
() ->
tm().query("FROM ClaimsList ORDER BY revisionId DESC", ClaimsList.class)
.setMaxResults(1)
.getResultStream()
.findFirst())
.orElse(ClaimsList.create(START_OF_TIME, ImmutableMap.of()));
}
@@ -30,6 +30,7 @@ import google.registry.export.DriveModule;
import google.registry.export.sheet.SheetsServiceModule;
import google.registry.flows.ServerTridProviderModule;
import google.registry.flows.custom.CustomLogicFactoryModule;
import google.registry.flows.domain.DomainDeletionTimeCacheModule;
import google.registry.groups.DirectoryModule;
import google.registry.groups.GmailModule;
import google.registry.groups.GroupsModule;
@@ -66,6 +67,7 @@ import jakarta.inject.Singleton;
CredentialModule.class,
CustomLogicFactoryModule.class,
DirectoryModule.class,
DomainDeletionTimeCacheModule.class,
DriveModule.class,
GmailModule.class,
GroupsModule.class,
@@ -23,6 +23,7 @@ import google.registry.batch.DeleteLoadTestDataAction;
import google.registry.batch.DeleteProberDataAction;
import google.registry.batch.ExpandBillingRecurrencesAction;
import google.registry.batch.RelockDomainAction;
import google.registry.batch.RemoveAllDomainContactsAction;
import google.registry.batch.ResaveAllEppResourcesPipelineAction;
import google.registry.batch.ResaveEntityAction;
import google.registry.batch.SendExpiringCertificateNotificationEmailAction;
@@ -270,6 +271,8 @@ interface RequestComponent {
ReadinessProbeActionFrontend readinessProbeActionFrontend();
RemoveAllDomainContactsAction removeAllDomainContactsAction();
RdapAutnumAction rdapAutnumAction();
RdapDomainAction rdapDomainAction();
@@ -26,6 +26,7 @@ import google.registry.export.DriveModule;
import google.registry.export.sheet.SheetsServiceModule;
import google.registry.flows.ServerTridProviderModule;
import google.registry.flows.custom.CustomLogicFactoryModule;
import google.registry.flows.domain.DomainDeletionTimeCacheModule;
import google.registry.groups.DirectoryModule;
import google.registry.groups.GmailModule;
import google.registry.groups.GroupsModule;
@@ -56,6 +57,7 @@ import jakarta.inject.Singleton;
CloudTasksUtilsModule.class,
CredentialModule.class,
CustomLogicFactoryModule.class,
DomainDeletionTimeCacheModule.class,
DirectoryModule.class,
DriveModule.class,
GmailModule.class,
@@ -23,6 +23,7 @@ import google.registry.batch.DeleteLoadTestDataAction;
import google.registry.batch.DeleteProberDataAction;
import google.registry.batch.ExpandBillingRecurrencesAction;
import google.registry.batch.RelockDomainAction;
import google.registry.batch.RemoveAllDomainContactsAction;
import google.registry.batch.ResaveAllEppResourcesPipelineAction;
import google.registry.batch.ResaveEntityAction;
import google.registry.batch.SendExpiringCertificateNotificationEmailAction;
@@ -153,6 +154,8 @@ public interface BackendRequestComponent {
RelockDomainAction relockDomainAction();
RemoveAllDomainContactsAction removeAllDomainContactsAction();
ResaveAllEppResourcesPipelineAction resaveAllEppResourcesPipelineAction();
ResaveEntityAction resaveEntityAction();
@@ -22,6 +22,7 @@ import google.registry.config.CredentialModule;
import google.registry.config.RegistryConfig.ConfigModule;
import google.registry.flows.ServerTridProviderModule;
import google.registry.flows.custom.CustomLogicFactoryModule;
import google.registry.flows.domain.DomainDeletionTimeCacheModule;
import google.registry.groups.DirectoryModule;
import google.registry.groups.GmailModule;
import google.registry.groups.GroupsModule;
@@ -50,6 +51,7 @@ import jakarta.inject.Singleton;
CustomLogicFactoryModule.class,
CloudTasksUtilsModule.class,
DirectoryModule.class,
DomainDeletionTimeCacheModule.class,
FrontendRequestComponentModule.class,
GmailModule.class,
GroupsModule.class,
@@ -23,6 +23,7 @@ import google.registry.config.RegistryConfig.ConfigModule;
import google.registry.export.DriveModule;
import google.registry.flows.ServerTridProviderModule;
import google.registry.flows.custom.CustomLogicFactoryModule;
import google.registry.flows.domain.DomainDeletionTimeCacheModule;
import google.registry.groups.DirectoryModule;
import google.registry.groups.GroupsModule;
import google.registry.groups.GroupssettingsModule;
@@ -47,6 +48,7 @@ import jakarta.inject.Singleton;
CustomLogicFactoryModule.class,
CloudTasksUtilsModule.class,
DirectoryModule.class,
DomainDeletionTimeCacheModule.class,
DriveModule.class,
GroupsModule.class,
GroupssettingsModule.class,
@@ -112,6 +112,15 @@ public abstract class OidcTokenAuthenticationMechanism implements Authentication
logger.atInfo().log("No email address from the OIDC token:\n%s", token.getPayload());
return AuthResult.NOT_AUTHENTICATED;
}
// Short-circuit the user lookup for known service accounts.
// This check bypasses the database lookup for high-volume
// traffic from trusted system accounts to reduce database load.
if (serviceAccountEmails.contains(email)) {
return AuthResult.createApp(email);
}
logger.atInfo().log("No service account found for email address %s, loading the User", email);
Optional<User> maybeUser =
tm().transact(() -> tm().loadByKeyIfPresent(VKey.create(User.class, email)));
stopwatch.tick("OidcTokenAuthenticationMechanism maybeUser loaded");
@@ -119,10 +128,7 @@ public abstract class OidcTokenAuthenticationMechanism implements Authentication
return AuthResult.createUser(maybeUser.get());
}
logger.atInfo().log("No end user found for email address %s", email);
if (serviceAccountEmails.stream().anyMatch(e -> e.equals(email))) {
return AuthResult.createApp(email);
}
logger.atInfo().log("No service account found for email address %s", email);
logger.atWarning().log(
"The email address %s is not tied to a principal with access to Nomulus", email);
return AuthResult.NOT_AUTHENTICATED;
@@ -47,6 +47,11 @@ final class CreateCdnsTld extends ConfirmingCommand {
)
String name;
@Parameter(
names = "--skip_sandbox_tld_check",
description = "In Sandbox, skip the dns_name format check.")
boolean skipSandboxTldCheck;
@Inject
@Config("projectId")
String projectId;
@@ -61,10 +66,15 @@ final class CreateCdnsTld extends ConfirmingCommand {
protected void init() {
// Sandbox talks to production Cloud DNS. As a result, we can't configure any domains with a
// suffix that might be used by customers on the same nameserver set. Limit the user to setting
// up *.test TLDs.
if (RegistryToolEnvironment.get() == RegistryToolEnvironment.SANDBOX
&& !dnsName.endsWith(".test.")) {
throw new IllegalArgumentException("Sandbox TLDs must be of the form \"*.test.\"");
// up *.test TLDs unless the user declares that the name is approved.
//
// The name format check simply provides a user-friendly error message. If the user wrongly
// declares name approval, the request to the Cloud DNS API will still fail.
if (RegistryToolEnvironment.get() == RegistryToolEnvironment.SANDBOX) {
if (!skipSandboxTldCheck && !dnsName.endsWith(".test.")) {
throw new IllegalArgumentException(
"Sandbox TLDs must be approved or in the form \"*.test.\"");
}
}
managedZone =
@@ -74,11 +74,12 @@ public final class DomainLockUtils {
* <p>The lock will not be applied until {@link #verifyVerificationCode} is called.
*/
public RegistryLock saveNewRegistryLockRequest(
String domainName, String registrarId, @Nullable String registrarPocId, boolean isAdmin) {
String domainName, String registrarId, @Nullable String registryLockEmail, boolean isAdmin) {
return tm().transact(
() ->
RegistryLockDao.save(
createLockBuilder(domainName, registrarId, registrarPocId, isAdmin).build()));
createLockBuilder(domainName, registrarId, registryLockEmail, isAdmin)
.build()));
}
/**
@@ -129,13 +130,13 @@ public final class DomainLockUtils {
* the case of relocks, isAdmin is determined by the previous lock.
*/
public RegistryLock administrativelyApplyLock(
String domainName, String registrarId, @Nullable String registrarPocId, boolean isAdmin) {
String domainName, String registrarId, @Nullable String registryLockEmail, boolean isAdmin) {
return tm().transact(
() -> {
DateTime now = tm().getTransactionTime();
RegistryLock newLock =
RegistryLockDao.save(
createLockBuilder(domainName, registrarId, registrarPocId, isAdmin)
createLockBuilder(domainName, registrarId, registryLockEmail, isAdmin)
.setLockCompletionTime(now)
.build());
applyLockStatuses(newLock, now, isAdmin);
@@ -235,7 +236,7 @@ public final class DomainLockUtils {
}
private RegistryLock.Builder createLockBuilder(
String domainName, String registrarId, @Nullable String registrarPocId, boolean isAdmin) {
String domainName, String registrarId, @Nullable String registryLockEmail, boolean isAdmin) {
DateTime now = tm().getTransactionTime();
Domain domain = getDomain(domainName, registrarId, now);
verifyDomainNotLocked(domain, isAdmin);
@@ -255,7 +256,7 @@ public final class DomainLockUtils {
.setDomainName(domainName)
.setRepoId(domain.getRepoId())
.setRegistrarId(registrarId)
.setRegistrarPocId(registrarPocId)
.setRegistryLockEmail(registryLockEmail)
.isSuperuser(isAdmin);
}
@@ -15,6 +15,7 @@
package google.registry.tools;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static com.google.common.collect.Sets.difference;
import static google.registry.model.billing.BillingBase.RenewalPriceBehavior.DEFAULT;
@@ -46,6 +47,9 @@ import google.registry.model.domain.token.AllocationToken;
import google.registry.model.domain.token.AllocationToken.RegistrationBehavior;
import google.registry.model.domain.token.AllocationToken.TokenStatus;
import google.registry.model.domain.token.AllocationToken.TokenType;
import google.registry.model.registrar.Registrar;
import google.registry.model.tld.Tld;
import google.registry.model.tld.Tlds;
import google.registry.persistence.VKey;
import google.registry.tools.params.MoneyParameter;
import google.registry.tools.params.TransitionListParameter.TokenStatusTransitions;
@@ -291,10 +295,12 @@ class GenerateAllocationTokensCommand implements Command {
!ImmutableList.of("").equals(allowedClientIds),
"Either omit --allowed_client_ids if all registrars are allowed, or include a"
+ " comma-separated list");
verifyAllRegistrarIdsExist(allowedClientIds);
checkArgument(
!ImmutableList.of("").equals(allowedTlds),
"Either omit --allowed_tlds if all TLDs are allowed, or include a comma-separated list");
verifyAllTldsExist(allowedTlds);
if (ImmutableList.of("").equals(allowedEppActions)) {
allowedEppActions = ImmutableList.of();
@@ -326,6 +332,34 @@ class GenerateAllocationTokensCommand implements Command {
}
}
static void verifyAllRegistrarIdsExist(@Nullable List<String> allowedClientIds) {
// a null/empty list means that all registrars are allowed
if (isNullOrEmpty(allowedClientIds)) {
return;
}
ImmutableSet<String> allRegistrarIds =
Registrar.loadAllKeysCached().stream()
.map(VKey::getKey)
.map(Object::toString)
.collect(toImmutableSet());
ImmutableList<String> badRegistrarIds =
allowedClientIds.stream()
.filter(id -> !allRegistrarIds.contains(id))
.collect(toImmutableList());
checkArgument(badRegistrarIds.isEmpty(), "Unknown registrar ID(s) %s", badRegistrarIds);
}
static void verifyAllTldsExist(@Nullable List<String> allowedTlds) {
// a null/empty list means that all TLDs are allowed
if (isNullOrEmpty(allowedTlds)) {
return;
}
ImmutableSet<String> allTlds = Tlds.getTldsOfType(Tld.TldType.REAL);
ImmutableList<String> badTlds =
allowedTlds.stream().filter(tld -> !allTlds.contains(tld)).collect(toImmutableList());
checkArgument(badTlds.isEmpty(), "Unknown REAL TLD(s) %s", badTlds);
}
private void verifyTokenStringsDoNotExist() {
ImmutableSet<String> existingTokenStrings =
getExistingTokenStrings(ImmutableSet.copyOf(tokenStrings));
@@ -85,12 +85,6 @@ final class RegistrarPocCommand extends MutatingCommand {
+ " and will be used as the console login email, if --login_email is not specified.")
String email;
@Nullable
@Parameter(
names = "--registry_lock_email",
description = "Email address used for registry lock confirmation emails")
String registryLockEmail;
@Nullable
@Parameter(
names = "--phone",
@@ -132,15 +126,6 @@ final class RegistrarPocCommand extends MutatingCommand {
arity = 1)
private Boolean visibleInDomainWhoisAsAbuse;
@Nullable
@Parameter(
names = "--allowed_to_set_registry_lock_password",
description =
"Allow this contact to set their registry lock password in the console,"
+ " enabling registry lock",
arity = 1)
private Boolean allowedToSetRegistryLockPassword;
@Parameter(
names = {"-o", "--output"},
description = "Output file when --mode=LIST",
@@ -235,9 +220,6 @@ final class RegistrarPocCommand extends MutatingCommand {
builder.setRegistrar(registrar);
builder.setName(name);
builder.setEmailAddress(email);
if (!isNullOrEmpty(registryLockEmail)) {
builder.setRegistryLockEmailAddress(registryLockEmail);
}
if (phone != null) {
builder.setPhoneNumber(phone.orElse(null));
}
@@ -255,9 +237,6 @@ final class RegistrarPocCommand extends MutatingCommand {
if (visibleInDomainWhoisAsAbuse != null) {
builder.setVisibleInDomainWhoisAsAbuse(visibleInDomainWhoisAsAbuse);
}
if (allowedToSetRegistryLockPassword != null) {
builder.setAllowedToSetRegistryLockPassword(allowedToSetRegistryLockPassword);
}
return builder.build();
}
@@ -269,9 +248,6 @@ final class RegistrarPocCommand extends MutatingCommand {
if (!isNullOrEmpty(name)) {
builder.setName(name);
}
if (!isNullOrEmpty(registryLockEmail)) {
builder.setRegistryLockEmailAddress(registryLockEmail);
}
if (phone != null) {
builder.setPhoneNumber(phone.orElse(null));
}
@@ -290,9 +266,6 @@ final class RegistrarPocCommand extends MutatingCommand {
if (visibleInDomainWhoisAsAbuse != null) {
builder.setVisibleInDomainWhoisAsAbuse(visibleInDomainWhoisAsAbuse);
}
if (allowedToSetRegistryLockPassword != null) {
builder.setAllowedToSetRegistryLockPassword(allowedToSetRegistryLockPassword);
}
return builder.build();
}
@@ -157,6 +157,9 @@ final class UpdateAllocationTokensCommand extends UpdateOrDeleteAllocationTokens
endToken = true;
}
GenerateAllocationTokensCommand.verifyAllRegistrarIdsExist(allowedClientIds);
GenerateAllocationTokensCommand.verifyAllTldsExist(allowedTlds);
tokensToSave =
tm().transact(
() ->
@@ -21,7 +21,6 @@ import static google.registry.util.DomainNameUtils.canonicalizeHostname;
import com.google.common.base.Ascii;
import com.google.common.base.Splitter;
import com.google.common.base.Strings;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.net.InternetDomainName;
@@ -29,7 +28,6 @@ import com.google.re2j.Pattern;
import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.RegistrarAddress;
import google.registry.model.registrar.RegistrarPoc;
import google.registry.ui.forms.FormException;
import google.registry.ui.forms.FormField;
import google.registry.ui.forms.FormFieldException;
import google.registry.ui.forms.FormFields;
@@ -38,7 +36,6 @@ import google.registry.util.X509Utils;
import java.security.cert.CertificateParsingException;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import java.util.Set;
import javax.annotation.Nullable;
@@ -192,14 +189,6 @@ public final class RegistrarFormFields {
public static final FormField<String, String> CONTACT_FAX_NUMBER_FIELD =
FormFields.PHONE_NUMBER.asBuilderNamed("faxNumber").build();
public static final FormField<Object, Boolean> CONTACT_ALLOWED_TO_SET_REGISTRY_LOCK_PASSWORD =
FormField.named("allowedToSetRegistryLockPassword", Object.class)
.transform(Boolean.class, b -> Boolean.valueOf(Objects.toString(b)))
.build();
public static final FormField<String, String> CONTACT_REGISTRY_LOCK_PASSWORD_FIELD =
FormFields.NAME.asBuilderNamed("registryLockPassword").build();
public static final FormField<String, Set<RegistrarPoc.Type>> CONTACT_TYPES =
FormField.named("types")
.uppercased()
@@ -369,8 +358,6 @@ public final class RegistrarFormFields {
private static void applyRegistrarContactArgs(RegistrarPoc.Builder builder, Map<String, ?> args) {
builder.setName(CONTACT_NAME_FIELD.extractUntyped(args).orElse(null));
builder.setEmailAddress(CONTACT_EMAIL_ADDRESS_FIELD.extractUntyped(args).orElse(null));
builder.setRegistryLockEmailAddress(
REGISTRY_LOCK_EMAIL_ADDRESS_FIELD.extractUntyped(args).orElse(null));
builder.setVisibleInWhoisAsAdmin(
CONTACT_VISIBLE_IN_WHOIS_AS_ADMIN_FIELD.extractUntyped(args).orElse(false));
builder.setVisibleInWhoisAsTech(
@@ -380,23 +367,5 @@ public final class RegistrarFormFields {
builder.setPhoneNumber(CONTACT_PHONE_NUMBER_FIELD.extractUntyped(args).orElse(null));
builder.setFaxNumber(CONTACT_FAX_NUMBER_FIELD.extractUntyped(args).orElse(null));
builder.setTypes(CONTACT_TYPES.extractUntyped(args).orElse(ImmutableSet.of()));
// The parser is inconsistent with whether it retrieves boolean values as strings or booleans.
// As a result, use a potentially-redundant converter that can deal with both.
builder.setAllowedToSetRegistryLockPassword(
CONTACT_ALLOWED_TO_SET_REGISTRY_LOCK_PASSWORD.extractUntyped(args).orElse(false));
// Registry lock password does not need to be set every time
CONTACT_REGISTRY_LOCK_PASSWORD_FIELD
.extractUntyped(args)
.ifPresent(
password -> {
if (!Strings.isNullOrEmpty(password)) {
if (password.length() < 8) {
throw new FormException(
"Registry lock password must be at least 8 characters long");
}
builder.setRegistryLockPassword(password);
}
});
}
}
@@ -15,15 +15,19 @@
package google.registry.ui.server.console;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.request.Action.Method.GET;
import static jakarta.servlet.http.HttpServletResponse.SC_BAD_REQUEST;
import static jakarta.servlet.http.HttpServletResponse.SC_OK;
import com.google.common.base.Strings;
import google.registry.config.RegistryConfig.Config;
import google.registry.model.console.ConsolePermission;
import google.registry.model.console.ConsoleUpdateHistory;
import google.registry.model.console.GlobalRole;
import google.registry.model.console.User;
import google.registry.model.console.UserRoles;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Action.GkeService;
@@ -43,8 +47,8 @@ public class ConsoleHistoryDataAction extends ConsoleApiAction {
private static final String SQL_USER_HISTORY =
"""
SELECT * FROM "ConsoleUpdateHistory"
WHERE acting_user = :actingUser
SELECT * FROM "ConsoleUpdateHistory"
WHERE acting_user = :actingUser
""";
private static final String SQL_REGISTRAR_HISTORY =
@@ -59,14 +63,18 @@ public class ConsoleHistoryDataAction extends ConsoleApiAction {
private final String registrarId;
private final Optional<String> consoleUserEmail;
private final String supportEmail;
@Inject
public ConsoleHistoryDataAction(
ConsoleApiParams consoleApiParams,
@Config("supportEmail") String supportEmail,
@Parameter("registrarId") String registrarId,
@Parameter("consoleUserEmail") Optional<String> consoleUserEmail) {
super(consoleApiParams);
this.registrarId = registrarId;
this.consoleUserEmail = consoleUserEmail;
this.supportEmail = supportEmail;
}
@Override
@@ -95,7 +103,9 @@ public class ConsoleHistoryDataAction extends ConsoleApiAction {
.setHint("org.hibernate.fetchSize", 1000)
.getResultList());
consoleApiParams.response().setPayload(consoleApiParams.gson().toJson(queryResult));
List<ConsoleUpdateHistory> formattedHistoryList =
replaceActiveUserIfNecessary(queryResult, user);
consoleApiParams.response().setPayload(consoleApiParams.gson().toJson(formattedHistoryList));
consoleApiParams.response().setStatus(SC_OK);
}
@@ -110,7 +120,39 @@ public class ConsoleHistoryDataAction extends ConsoleApiAction {
.setParameter("registrarId", registrarId)
.setHint("org.hibernate.fetchSize", 1000)
.getResultList());
consoleApiParams.response().setPayload(consoleApiParams.gson().toJson(queryResult));
List<ConsoleUpdateHistory> formattedHistoryList =
replaceActiveUserIfNecessary(queryResult, user);
consoleApiParams.response().setPayload(consoleApiParams.gson().toJson(formattedHistoryList));
consoleApiParams.response().setStatus(SC_OK);
}
/** Anonymizes support users in history logs if the user viewing the log is a registrar user. */
private List<ConsoleUpdateHistory> replaceActiveUserIfNecessary(
List<ConsoleUpdateHistory> historyList, User requestingUser) {
// Check if the user *viewing* the history is a registrar user (not a support user)
if (GlobalRole.NONE.equals(requestingUser.getUserRoles().getGlobalRole())) {
User genericSupportUser = // Fixed typo
new User.Builder()
.setEmailAddress(this.supportEmail)
.setUserRoles(new UserRoles.Builder().build()) // Simplified roles
.build();
return historyList.stream()
.map(
history -> {
// Check if the user who performed the action was a support user
if (!GlobalRole.NONE.equals(
history.getActingUser().getUserRoles().getGlobalRole())) {
return history.asBuilder().setActingUser(genericSupportUser).build();
}
// If acting user was a registrar user show them as-is
return history;
})
.collect(toImmutableList());
}
// If the viewing user is a support user, return the list unmodified
return historyList;
}
}
@@ -23,6 +23,7 @@ import static org.apache.http.HttpStatus.SC_OK;
import com.google.common.base.Strings;
import com.google.common.collect.ImmutableSet;
import com.google.common.flogger.FluentLogger;
import google.registry.model.console.ConsolePermission;
import google.registry.model.console.ConsoleUpdateHistory;
import google.registry.model.console.User;
@@ -47,6 +48,8 @@ import org.joda.time.DateTime;
method = {POST},
auth = Auth.AUTH_PUBLIC_LOGGED_IN)
public class ConsoleUpdateRegistrarAction extends ConsoleApiAction {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private static final String CHANGE_LOG_ENTRY = "%s updated on %s, old -> %s, new -> %s";
static final String PATH = "/console-api/registrar";
private final Optional<Registrar> registrar;
@@ -124,6 +127,9 @@ public class ConsoleUpdateRegistrarAction extends ConsoleApiAction {
new ConsoleUpdateHistory.Builder()
.setType(ConsoleUpdateHistory.Type.REGISTRAR_UPDATE)
.setDescription(updatedRegistrar.getRegistrarId()));
logConsoleChangesIfNecessary(updatedRegistrar, existingRegistrar.get());
sendExternalUpdatesIfNecessary(
EmailInfo.create(
existingRegistrar.get(),
@@ -134,4 +140,25 @@ public class ConsoleUpdateRegistrarAction extends ConsoleApiAction {
consoleApiParams.response().setStatus(SC_OK);
}
private void logConsoleChangesIfNecessary(
Registrar updatedRegistrar, Registrar existingRegistrar) {
if (!updatedRegistrar.getAllowedTlds().containsAll(existingRegistrar.getAllowedTlds())) {
logger.atInfo().log(
CHANGE_LOG_ENTRY,
"Allowed TLDs",
updatedRegistrar.getRegistrarId(),
existingRegistrar.getAllowedTlds(),
updatedRegistrar.getAllowedTlds());
}
if (updatedRegistrar.isRegistryLockAllowed() != existingRegistrar.isRegistryLockAllowed()) {
logger.atInfo().log(
CHANGE_LOG_ENTRY,
"Registry lock",
updatedRegistrar.getRegistrarId(),
existingRegistrar.isRegistryLockAllowed(),
updatedRegistrar.isRegistryLockAllowed());
}
}
}
@@ -17,6 +17,7 @@ package google.registry.ui.server.console;
import static com.google.common.base.Strings.isNullOrEmpty;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static google.registry.model.console.RegistrarRole.ACCOUNT_MANAGER;
import static google.registry.model.console.RegistrarRole.TECH_CONTACT;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.request.Action.Method.DELETE;
import static google.registry.request.Action.Method.GET;
@@ -152,7 +153,7 @@ public class ConsoleUsersAction extends ConsoleApiAction {
updateUserRegistrarRoles(
this.userData.get().emailAddress,
registrarId,
RegistrarRole.valueOf(this.userData.get().role));
requestRoleToAllowedRoles(this.userData.get().role));
sendConfirmationEmail(registrarId, this.userData.get().emailAddress, "Added existing user");
consoleApiParams.response().setStatus(SC_OK);
@@ -222,11 +223,9 @@ public class ConsoleUsersAction extends ConsoleApiAction {
throw e;
}
RegistrarRole newRole = requestRoleToAllowedRoles(userData.get().role);
UserRoles userRoles =
new UserRoles.Builder()
.setRegistrarRoles(
ImmutableMap.of(registrarId, RegistrarRole.valueOf(userData.get().role)))
.build();
new UserRoles.Builder().setRegistrarRoles(ImmutableMap.of(registrarId, newRole)).build();
User.Builder builder = new User.Builder().setUserRoles(userRoles).setEmailAddress(newEmail);
tm().put(builder.build());
@@ -238,9 +237,7 @@ public class ConsoleUsersAction extends ConsoleApiAction {
.setPayload(
consoleApiParams
.gson()
.toJson(
new UserData(
newEmail, null, ACCOUNT_MANAGER.toString(), newUser.getPassword())));
.toJson(new UserData(newEmail, null, newRole.toString(), newUser.getPassword())));
finishAndPersistConsoleUpdateHistory(
new ConsoleUpdateHistory.Builder()
.setType(ConsoleUpdateHistory.Type.USER_CREATE)
@@ -257,7 +254,7 @@ public class ConsoleUsersAction extends ConsoleApiAction {
updateUserRegistrarRoles(
this.userData.get().emailAddress,
registrarId,
RegistrarRole.valueOf(this.userData.get().role));
requestRoleToAllowedRoles(this.userData.get().role));
sendConfirmationEmail(registrarId, this.userData.get().emailAddress, "Updated user");
consoleApiParams.response().setStatus(SC_OK);
@@ -333,6 +330,11 @@ public class ConsoleUsersAction extends ConsoleApiAction {
.collect(toImmutableList()));
}
/** Maps a request role string to a RegistrarRole, using ACCOUNT_MANAGER as the default. */
private RegistrarRole requestRoleToAllowedRoles(String role) {
return TECH_CONTACT.name().equals(role) ? TECH_CONTACT : ACCOUNT_MANAGER;
}
private boolean sendConfirmationEmail(String registrarId, String emailAddress, String operation) {
Optional<Registrar> registrar = Registrar.loadByRegistrarId(registrarId);
if (registrar.isEmpty()) { // Shouldn't happen, but worth checking
@@ -53,8 +53,10 @@ import java.util.Optional;
public class RegistrarsAction extends ConsoleApiAction {
private static final int PASSWORD_LENGTH = 16;
private static final int PASSCODE_LENGTH = 5;
private static final ImmutableList<Registrar.Type> allowedRegistrarTypes =
ImmutableList.of(Registrar.Type.REAL, Registrar.Type.OTE);
private static final ImmutableSet<Registrar.Type> TYPES_ALLOWED_FOR_USERS =
ImmutableSet.of(Registrar.Type.REAL, Registrar.Type.OTE);
private static final ImmutableSet<Registrar.Type> TYPES_ALLOWED_FOR_ADMINS =
ImmutableSet.of(Registrar.Type.INTERNAL, Registrar.Type.REAL, Registrar.Type.OTE);
private static final String SQL_TEMPLATE =
"""
SELECT * FROM "Registrar"
@@ -80,6 +82,8 @@ public class RegistrarsAction extends ConsoleApiAction {
@Override
protected void getHandler(User user) {
if (user.getUserRoles().hasGlobalPermission(ConsolePermission.VIEW_REGISTRARS)) {
ImmutableSet<Registrar.Type> allowedRegistrarTypes =
user.getUserRoles().isAdmin() ? TYPES_ALLOWED_FOR_ADMINS : TYPES_ALLOWED_FOR_USERS;
ImmutableList<Registrar> registrars =
Streams.stream(Registrar.loadAll())
.filter(r -> allowedRegistrarTypes.contains(r.getType()))
@@ -94,6 +98,7 @@ public class RegistrarsAction extends ConsoleApiAction {
.map(Map.Entry::getKey)
.collect(toImmutableSet());
@SuppressWarnings("unchecked")
List<Registrar> registrars =
tm().transact(
() ->
@@ -45,7 +45,6 @@ import google.registry.ui.server.console.ConsoleApiAction;
import google.registry.ui.server.console.ConsoleApiParams;
import jakarta.inject.Inject;
import java.util.HashSet;
import java.util.Objects;
import java.util.Optional;
import java.util.Set;
import java.util.function.BiFunction;
@@ -246,7 +245,6 @@ public class ContactAction extends ConsoleApiAction {
throw new ContactRequirementException(
"An abuse contact visible in domain WHOIS query must be designated");
}
checkContactRegistryLockRequirements(existingContacts, updatedContacts);
}
private static void enforcePrimaryContactRestrictions(
@@ -266,69 +264,6 @@ public class ContactAction extends ConsoleApiAction {
}
}
private static void checkContactRegistryLockRequirements(
ImmutableSet<RegistrarPoc> existingContacts, ImmutableSet<RegistrarPoc> updatedContacts) {
// Any contact(s) with new passwords must be allowed to set them
for (RegistrarPoc updatedContact : updatedContacts) {
if (updatedContact.isRegistryLockAllowed()
|| updatedContact.isAllowedToSetRegistryLockPassword()) {
RegistrarPoc existingContact =
existingContacts.stream()
.filter(
contact -> contact.getEmailAddress().equals(updatedContact.getEmailAddress()))
.findFirst()
.orElseThrow(
() ->
new FormException(
"Cannot set registry lock password directly on new contact"));
// Can't modify registry lock email address
if (!Objects.equals(
updatedContact.getRegistryLockEmailAddress(),
existingContact.getRegistryLockEmailAddress())) {
throw new FormException("Cannot modify registryLockEmailAddress through the UI");
}
if (updatedContact.isRegistryLockAllowed()) {
// the password must have been set before or the user was allowed to set it now
if (!existingContact.isAllowedToSetRegistryLockPassword()
&& !existingContact.isRegistryLockAllowed()) {
throw new FormException("Registrar contact not allowed to set registry lock password");
}
}
if (updatedContact.isAllowedToSetRegistryLockPassword()) {
if (!existingContact.isAllowedToSetRegistryLockPassword()) {
throw new FormException(
"Cannot modify isAllowedToSetRegistryLockPassword through the UI");
}
}
}
}
// Any previously-existing contacts with registry lock enabled cannot be deleted
existingContacts.stream()
.filter(RegistrarPoc::isRegistryLockAllowed)
.forEach(
contact -> {
Optional<RegistrarPoc> updatedContactOptional =
updatedContacts.stream()
.filter(
updatedContact ->
updatedContact.getEmailAddress().equals(contact.getEmailAddress()))
.findFirst();
if (updatedContactOptional.isEmpty()) {
throw new FormException(
String.format(
"Cannot delete the contact %s that has registry lock enabled",
contact.getEmailAddress()));
}
if (!updatedContactOptional.get().isRegistryLockAllowed()) {
throw new FormException(
String.format(
"Cannot remove the ability to use registry lock on the contact %s",
contact.getEmailAddress()));
}
});
}
/**
* Retrieves the registrar contact whose phone number and email address is visible in domain WHOIS
* query as abuse contact (if any).
@@ -0,0 +1,22 @@
<epp xmlns="urn:ietf:params:xml:ns:epp-1.0">
<command>
<update>
<domain:update xmlns:domain="urn:ietf:params:xml:ns:domain-1.0">
<domain:name>%DOMAIN%</domain:name>
<domain:rem>
%CONTACTS%
</domain:rem>
<domain:chg>
<domain:registrant/>
</domain:chg>
</domain:update>
</update>
<extension>
<metadata:metadata xmlns:metadata="urn:google:params:xml:ns:metadata-1.0">
<metadata:reason>Registry minimum data set phase 3: Removed all contacts from domain.</metadata:reason>
<metadata:requestedByRegistrar>false</metadata:requestedByRegistrar>
</metadata:metadata>
</extension>
<clTRID>ABC-12345</clTRID>
</command>
</epp>
@@ -68,7 +68,7 @@ public class RelockDomainActionTest {
private static final String DOMAIN_NAME = "example.tld";
private static final String CLIENT_ID = "TheRegistrar";
private static final String POC_ID = "marla.singer@example.com";
private static final String LOCK_EMAIL_ADDRESS = "Marla.Singer.RegistryLock@crr.com";
private final FakeResponse response = new FakeResponse();
private final FakeClock clock = new FakeClock(DateTime.parse("2015-05-18T12:34:56Z"));
@@ -94,7 +94,9 @@ public class RelockDomainActionTest {
Host host = persistActiveHost("ns1.example.net");
domain = persistResource(DatabaseHelper.newDomain(DOMAIN_NAME, host));
oldLock = domainLockUtils.administrativelyApplyLock(DOMAIN_NAME, CLIENT_ID, POC_ID, false);
oldLock =
domainLockUtils.administrativelyApplyLock(
DOMAIN_NAME, CLIENT_ID, LOCK_EMAIL_ADDRESS, false);
assertThat(loadByEntity(domain).getStatusValues())
.containsAtLeastElementsIn(REGISTRY_LOCK_STATUSES);
oldLock =
@@ -255,9 +257,10 @@ public class RelockDomainActionTest {
.setSubject("Successful re-lock of domain example.tld")
.setBody(
"""
The domain example.tld was successfully re-locked.
The domain example.tld was successfully re-locked.
Please contact support at support@example.com if you have any questions.""")
Please contact support at support@example.com if you have any questions.\
""")
.setRecipients(
ImmutableSet.of(new InternetAddress("Marla.Singer.RegistryLock@crr.com")))
.build();
@@ -268,9 +271,10 @@ public class RelockDomainActionTest {
String expectedBody =
String.format(
"""
There was an error when automatically re-locking example.tld. Error message: %s
There was an error when automatically re-locking example.tld. Error message: %s
Please contact support at support@example.com if you have any questions.""",
Please contact support at support@example.com if you have any questions.\
""",
exceptionMessage);
assertFailureEmailWithBody(
expectedBody, ImmutableSet.of(new InternetAddress("Marla.Singer.RegistryLock@crr.com")));
@@ -0,0 +1,119 @@
// Copyright 2025 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.batch;
import static com.google.common.truth.Truth.assertThat;
import static google.registry.model.common.FeatureFlag.FeatureName.MINIMUM_DATASET_CONTACTS_PROHIBITED;
import static google.registry.model.common.FeatureFlag.FeatureStatus.ACTIVE;
import static google.registry.testing.DatabaseHelper.createTld;
import static google.registry.testing.DatabaseHelper.loadByEntity;
import static google.registry.testing.DatabaseHelper.newDomain;
import static google.registry.testing.DatabaseHelper.persistActiveContact;
import static google.registry.testing.DatabaseHelper.persistResource;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import static org.mockito.Mockito.mock;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedMap;
import com.google.common.util.concurrent.RateLimiter;
import google.registry.flows.DaggerEppTestComponent;
import google.registry.flows.EppController;
import google.registry.flows.EppTestComponent.FakesAndMocksModule;
import google.registry.model.common.FeatureFlag;
import google.registry.model.contact.Contact;
import google.registry.model.domain.Domain;
import google.registry.persistence.transaction.JpaTestExtensions;
import google.registry.persistence.transaction.JpaTestExtensions.JpaIntegrationTestExtension;
import google.registry.testing.FakeClock;
import google.registry.testing.FakeLockHandler;
import google.registry.testing.FakeResponse;
import java.util.Optional;
import org.joda.time.DateTime;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;
/** Unit tests for {@link RemoveAllDomainContactsAction}. */
class RemoveAllDomainContactsActionTest {
@RegisterExtension
final JpaIntegrationTestExtension jpa =
new JpaTestExtensions.Builder().buildIntegrationTestExtension();
private final FakeResponse response = new FakeResponse();
private final RateLimiter rateLimiter = mock(RateLimiter.class);
private RemoveAllDomainContactsAction action;
@BeforeEach
void beforeEach() {
createTld("tld");
persistResource(
new FeatureFlag.Builder()
.setFeatureName(MINIMUM_DATASET_CONTACTS_PROHIBITED)
.setStatusMap(ImmutableSortedMap.of(START_OF_TIME, ACTIVE))
.build());
EppController eppController =
DaggerEppTestComponent.builder()
.fakesAndMocksModule(FakesAndMocksModule.create(new FakeClock()))
.build()
.startRequest()
.eppController();
action =
new RemoveAllDomainContactsAction(
eppController, "NewRegistrar", new FakeLockHandler(true), rateLimiter, response);
}
@Test
void test_removesAllContactsFromMultipleDomains_andDoesntModifyDomainThatHasNoContacts() {
Contact c1 = persistActiveContact("contact12345");
Domain d1 = persistResource(newDomain("foo.tld", c1));
assertThat(d1.getAllContacts()).hasSize(3);
Contact c2 = persistActiveContact("contact23456");
Domain d2 = persistResource(newDomain("bar.tld", c2));
assertThat(d2.getAllContacts()).hasSize(3);
Domain d3 =
persistResource(
newDomain("baz.tld")
.asBuilder()
.setRegistrant(Optional.empty())
.setContacts(ImmutableSet.of())
.build());
assertThat(d3.getAllContacts()).isEmpty();
DateTime lastUpdate = d3.getUpdateTimestamp().getTimestamp();
action.run();
assertThat(loadByEntity(d1).getAllContacts()).isEmpty();
assertThat(loadByEntity(d2).getAllContacts()).isEmpty();
assertThat(loadByEntity(d3).getUpdateTimestamp().getTimestamp()).isEqualTo(lastUpdate);
}
@Test
void test_removesContacts_onDomainsThatOnlyPartiallyHaveContacts() {
Contact c1 = persistActiveContact("contact12345");
Domain d1 =
persistResource(
newDomain("foo.tld", c1).asBuilder().setContacts(ImmutableSet.of()).build());
assertThat(d1.getAllContacts()).hasSize(1);
Contact c2 = persistActiveContact("contact23456");
Domain d2 =
persistResource(
newDomain("bar.tld", c2).asBuilder().setRegistrant(Optional.empty()).build());
assertThat(d2.getAllContacts()).hasSize(2);
action.run();
assertThat(loadByEntity(d1).getAllContacts()).isEmpty();
assertThat(loadByEntity(d2).getAllContacts()).isEmpty();
}
}
@@ -20,13 +20,24 @@ import static google.registry.testing.DatabaseHelper.persistActiveDomain;
import static google.registry.testing.DatabaseHelper.persistDeletedDomain;
import static google.registry.testing.DatabaseHelper.persistReservedList;
import static google.registry.testing.DatabaseHelper.persistResource;
import static google.registry.testing.LogsSubject.assertAboutLogs;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import static google.registry.util.NetworkUtils.pickUnusedPort;
import static java.nio.charset.StandardCharsets.UTF_8;
import static java.util.concurrent.Executors.newSingleThreadExecutor;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;
import com.google.cloud.storage.BlobId;
import com.google.cloud.storage.contrib.nio.testing.LocalStorageHelper;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.flogger.FluentLogger;
import com.google.common.hash.Hashing;
import com.google.common.io.ByteStreams;
import com.google.common.net.HostAndPort;
import com.google.common.testing.TestLogHandler;
import com.google.gson.Gson;
import google.registry.bsa.api.BsaCredential;
import google.registry.gcs.GcsUtils;
import google.registry.model.tld.Tld;
@@ -35,9 +46,25 @@ import google.registry.model.tld.label.ReservedList;
import google.registry.persistence.transaction.JpaTestExtensions;
import google.registry.persistence.transaction.JpaTestExtensions.JpaIntegrationTestExtension;
import google.registry.request.UrlConnectionService;
import google.registry.server.Route;
import google.registry.server.TestServer;
import google.registry.testing.FakeClock;
import google.registry.testing.FakeResponse;
import jakarta.servlet.ServletException;
import jakarta.servlet.annotation.MultipartConfig;
import jakarta.servlet.http.HttpServlet;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import jakarta.servlet.http.Part;
import java.io.IOException;
import java.io.InputStream;
import java.io.PrintWriter;
import java.net.InetAddress;
import java.util.Map;
import java.util.Optional;
import java.util.logging.Level;
import java.util.logging.Logger;
import java.util.zip.GZIPInputStream;
import org.joda.time.DateTime;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
@@ -102,13 +129,112 @@ public class UploadBsaUnavailableDomainsActionTest {
BlobId existingFile =
BlobId.of(BUCKET, String.format("unavailable_domains_%s.txt", clock.nowUtc()));
String blockList = new String(gcsUtils.readBytesFrom(existingFile), UTF_8);
assertThat(blockList).isEqualTo("ace.tld\nflagrant.tld\nfoobar.tld\njimmy.tld\ntine.tld");
assertThat(blockList).isEqualTo("ace.tld\nflagrant.tld\nfoobar.tld\njimmy.tld\ntine.tld\n");
assertThat(blockList).doesNotContain("not-blocked.tld");
// This test currently fails in the upload-to-bsa step.
verify(emailSender, times(1))
.sendNotification("BSA daily upload completed with errors", "Please see logs for details.");
}
// TODO(mcilwain): Add test of BSA API upload as well.
@Test
void uploadToBsaTest() throws Exception {
TestLogHandler logHandler = new TestLogHandler();
Logger loggerToIntercept =
Logger.getLogger(UploadBsaUnavailableDomainsAction.class.getCanonicalName());
loggerToIntercept.addHandler(logHandler);
persistActiveDomain("foobar.tld");
persistActiveDomain("ace.tld");
persistDeletedDomain("not-blocked.tld", clock.nowUtc().minusDays(1));
var testServer = startTestServer();
action.apiUrl = testServer.getUrl("/upload").toURI().toString();
try {
action.run();
} finally {
testServer.stop();
}
String dataSent = "ace.tld\nflagrant.tld\nfoobar.tld\njimmy.tld\ntine.tld\n";
String checkSum = Hashing.sha512().hashString(dataSent, UTF_8).toString();
String expectedResponse =
"Received response with code 200 from server: "
+ String.format("Checksum: [%s]\n%s\n", checkSum, dataSent);
assertAboutLogs().that(logHandler).hasLogAtLevelWithMessage(Level.INFO, expectedResponse);
verify(emailSender, times(1)).sendNotification("BSA daily upload completed successfully", "");
}
private TestServer startTestServer() throws Exception {
TestServer testServer =
new TestServer(
HostAndPort.fromParts(InetAddress.getLocalHost().getHostAddress(), pickUnusedPort()),
ImmutableMap.of(),
ImmutableList.of(Route.route("/upload", Servelet.class)));
testServer.start();
newSingleThreadExecutor()
.execute(
() -> {
try {
while (true) {
testServer.process();
}
} catch (InterruptedException e) {
// Expected
}
});
return testServer;
}
@MultipartConfig(
location = "", // Directory for storing uploaded files. Use default when blank
maxFileSize = 10485760L, // 10MB
maxRequestSize = 20971520L, // 20MB
fileSizeThreshold = 1048576 // Save in memory if file size < 1MB
)
public static class Servelet extends HttpServlet {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
@Override
protected void doPost(HttpServletRequest req, HttpServletResponse resp)
throws ServletException, IOException {
String checkSum = null;
String content = null;
try {
for (Part part : req.getParts()) {
switch (part.getName()) {
case "zone" -> checkSum = readChecksum(part);
case "file" -> content = readGzipped(part);
}
}
} catch (Exception e) {
logger.atInfo().withCause(e).log("");
}
int status = checkSum == null || content == null ? 400 : 200;
resp.setStatus(status);
resp.setContentType("text/plain");
try (PrintWriter writer = resp.getWriter()) {
writer.printf("Checksum: [%s]\n%s\n", checkSum, content);
}
}
private String readChecksum(Part part) {
try (InputStream is = part.getInputStream()) {
return new Gson()
.fromJson(new String(ByteStreams.toByteArray(is), UTF_8), Map.class)
.getOrDefault("checkSum", "Not found")
.toString();
} catch (IOException e) {
throw new RuntimeException(e);
}
}
private String readGzipped(Part part) {
try (InputStream is = part.getInputStream();
GZIPInputStream gis = new GZIPInputStream(is)) {
return new String(ByteStreams.toByteArray(gis), UTF_8);
} catch (IOException e) {
throw new RuntimeException(e);
}
}
}
}
@@ -25,6 +25,7 @@ import google.registry.config.RegistryConfig.ConfigModule;
import google.registry.config.RegistryConfig.ConfigModule.TmchCaMode;
import google.registry.flows.custom.CustomLogicFactory;
import google.registry.flows.custom.TestCustomLogicFactory;
import google.registry.flows.domain.DomainDeletionTimeCache;
import google.registry.flows.domain.DomainFlowTmchUtils;
import google.registry.monitoring.whitebox.EppMetric;
import google.registry.request.RequestScope;
@@ -126,6 +127,11 @@ public interface EppTestComponent {
ServerTridProvider provideServerTridProvider() {
return new FakeServerTridProvider();
}
@Provides
DomainDeletionTimeCache provideDomainDeletionTimeCache() {
return DomainDeletionTimeCache.create();
}
}
class FakeServerTridProvider implements ServerTridProvider {
@@ -40,8 +40,9 @@ public class TestDomainCreateFlowCustomLogic extends DomainCreateFlowCustomLogic
.setMsg("Custom logic was triggered")
.build();
return EntityChanges.newBuilder()
.setSaves(parameters.entityChanges().getSaves())
.addSave(extraPollMessage)
.setInserts(parameters.entityChanges().getInserts())
.addInsert(extraPollMessage)
.setUpdates(parameters.entityChanges().getUpdates())
.setDeletes(parameters.entityChanges().getDeletes())
.build();
}
@@ -149,7 +149,6 @@ import google.registry.flows.domain.token.AllocationTokenFlowUtils.AlreadyRedeem
import google.registry.flows.domain.token.AllocationTokenFlowUtils.NonexistentAllocationTokenException;
import google.registry.flows.exceptions.ContactsProhibitedException;
import google.registry.flows.exceptions.OnlyToolCanPassMetadataException;
import google.registry.flows.exceptions.ResourceAlreadyExistsForThisClientException;
import google.registry.flows.exceptions.ResourceCreateContentionException;
import google.registry.model.billing.BillingBase;
import google.registry.model.billing.BillingBase.Flag;
@@ -1238,8 +1237,8 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
void testFailure_alreadyExists() throws Exception {
persistContactsAndHosts();
persistActiveDomain(getUniqueIdFromCommand());
ResourceAlreadyExistsForThisClientException thrown =
assertThrows(ResourceAlreadyExistsForThisClientException.class, this::runFlow);
ResourceCreateContentionException thrown =
assertThrows(ResourceCreateContentionException.class, this::runFlow);
assertAboutEppExceptions()
.that(thrown)
.marshalsToXml()
@@ -0,0 +1,97 @@
// Copyright 2025 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.flows.domain;
import static com.google.common.truth.Truth.assertThat;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.testing.DatabaseHelper.persistActiveDomain;
import static google.registry.testing.DatabaseHelper.persistDomainAsDeleted;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import google.registry.model.domain.Domain;
import google.registry.persistence.transaction.JpaTestExtensions;
import google.registry.testing.DatabaseHelper;
import google.registry.testing.FakeClock;
import java.util.Optional;
import org.joda.time.DateTime;
import org.joda.time.Duration;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;
/** Tests for {@link DomainDeletionTimeCache}. */
public class DomainDeletionTimeCacheTest {
private final FakeClock clock = new FakeClock(DateTime.parse("2025-10-01T00:00:00.000Z"));
private final DomainDeletionTimeCache cache = DomainDeletionTimeCache.create();
@RegisterExtension
final JpaTestExtensions.JpaIntegrationTestExtension jpa =
new JpaTestExtensions.Builder().withClock(clock).buildIntegrationTestExtension();
@BeforeEach
void beforeEach() {
DatabaseHelper.createTld("tld");
}
@Test
void testDomainAvailable_null() {
assertThat(getDeletionTimeFromCache("nonexistent.tld")).isEmpty();
}
@Test
void testDomainNotAvailable_notDeleted() {
persistActiveDomain("active.tld");
assertThat(getDeletionTimeFromCache("active.tld")).hasValue(END_OF_TIME);
}
@Test
void testDomainAvailable_deletedInFuture() {
persistDomainAsDeleted(persistActiveDomain("domain.tld"), clock.nowUtc().plusDays(1));
assertThat(getDeletionTimeFromCache("domain.tld")).hasValue(clock.nowUtc().plusDays(1));
}
@Test
void testCache_returnsOldData() {
Domain domain = persistActiveDomain("domain.tld");
assertThat(getDeletionTimeFromCache("domain.tld")).hasValue(END_OF_TIME);
persistDomainAsDeleted(domain, clock.nowUtc().plusDays(1));
// Without intervention, the cache should have the old data, even if a few minutes have passed
clock.advanceBy(Duration.standardMinutes(5));
assertThat(getDeletionTimeFromCache("domain.tld")).hasValue(END_OF_TIME);
}
@Test
void testCache_returnsNewDataAfterDomainCreate() {
// Null deletion dates (meaning an avilable domain) shouldn't be cached
assertThat(getDeletionTimeFromCache("domain.tld")).isEmpty();
persistDomainAsDeleted(persistActiveDomain("domain.tld"), clock.nowUtc().plusDays(1));
assertThat(getDeletionTimeFromCache("domain.tld")).hasValue(clock.nowUtc().plusDays(1));
}
@Test
void testCache_expires() {
Domain domain = persistActiveDomain("domain.tld");
assertThat(getDeletionTimeFromCache("domain.tld")).hasValue(END_OF_TIME);
DateTime elevenMinutesFromNow = clock.nowUtc().plusMinutes(11);
persistDomainAsDeleted(domain, elevenMinutesFromNow);
clock.advanceBy(Duration.standardMinutes(30));
assertThat(getDeletionTimeFromCache("domain.tld")).hasValue(elevenMinutesFromNow);
}
private Optional<DateTime> getDeletionTimeFromCache(String domainName) {
return tm().transact(() -> cache.getDeletionTimeForDomain(domainName));
}
}
@@ -14,17 +14,19 @@
package google.registry.model;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.testing.DatabaseHelper.createTld;
import static google.registry.testing.DatabaseHelper.persistActiveDomain;
import static google.registry.testing.DatabaseHelper.persistActiveHost;
import static google.registry.testing.DatabaseHelper.persistDeletedDomain;
import static google.registry.testing.DatabaseHelper.persistDeletedHost;
import static google.registry.testing.DatabaseHelper.persistDomainAsDeleted;
import static google.registry.testing.DatabaseHelper.persistPremiumList;
import static google.registry.testing.DatabaseHelper.persistResource;
import static google.registry.testing.TestDataHelper.loadBytes;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import static org.joda.money.CurrencyUnit.USD;
import google.registry.model.domain.Domain;
import google.registry.model.domain.DomainHistory;
import google.registry.model.eppcommon.Trid;
import google.registry.model.host.HostHistory;
@@ -49,7 +51,7 @@ public final class OteStatsTestHelper {
.build());
persistResource(
new DomainHistory.Builder()
.setDomain(persistActiveDomain("example.tld"))
.setDomain(persistActiveDomain("restored.tld"))
.setRegistrarId(oteAccount1)
.setType(Type.DOMAIN_RESTORE)
.setXmlBytes(getBytes("domain_restore.xml"))
@@ -84,7 +86,7 @@ public final class OteStatsTestHelper {
DateTime now = DateTime.now(DateTimeZone.UTC);
persistResource(
new DomainHistory.Builder()
.setDomain(persistActiveDomain("exampleone.tld"))
.setDomain(loadOrCreateDomain("exampleone.tld"))
.setRegistrarId(oteAccount1)
.setType(Type.DOMAIN_CREATE)
.setXmlBytes(getBytes("domain_create_sunrise.xml"))
@@ -92,15 +94,16 @@ public final class OteStatsTestHelper {
.build());
persistResource(
new DomainHistory.Builder()
.setDomain(persistActiveDomain("example-one.tld"))
.setDomain(loadOrCreateDomain("example-one.tld"))
.setRegistrarId(oteAccount1)
.setType(Type.DOMAIN_CREATE)
.setXmlBytes(getBytes("domain_create_claim_notice.xml"))
.setModificationTime(now)
.build());
Domain exampleDomain = loadOrCreateDomain("example.tld");
persistResource(
new DomainHistory.Builder()
.setDomain(persistActiveDomain("example.tld"))
.setDomain(exampleDomain)
.setRegistrarId(oteAccount1)
.setType(Type.DOMAIN_CREATE)
.setXmlBytes(getBytes("domain_create_anchor_tenant_fee_standard.xml"))
@@ -108,7 +111,7 @@ public final class OteStatsTestHelper {
.build());
persistResource(
new DomainHistory.Builder()
.setDomain(persistActiveDomain("example.tld"))
.setDomain(exampleDomain)
.setRegistrarId(oteAccount1)
.setType(Type.DOMAIN_CREATE)
.setXmlBytes(getBytes("domain_create_dsdata.xml"))
@@ -116,7 +119,7 @@ public final class OteStatsTestHelper {
.build());
persistResource(
new DomainHistory.Builder()
.setDomain(persistDeletedDomain("example.tld", now))
.setDomain(persistDomainAsDeleted(loadOrCreateDomain("deleted.tld"), now))
.setRegistrarId(oteAccount1)
.setType(Type.DOMAIN_DELETE)
.setXmlBytes(getBytes("domain_delete.xml"))
@@ -124,7 +127,7 @@ public final class OteStatsTestHelper {
.build());
persistResource(
new DomainHistory.Builder()
.setDomain(persistActiveDomain("example.tld"))
.setDomain(exampleDomain)
.setRegistrarId(oteAccount1)
.setType(Type.DOMAIN_TRANSFER_APPROVE)
.setXmlBytes(getBytes("domain_transfer_approve.xml"))
@@ -132,7 +135,7 @@ public final class OteStatsTestHelper {
.build());
persistResource(
new DomainHistory.Builder()
.setDomain(persistActiveDomain("example.tld"))
.setDomain(exampleDomain)
.setRegistrarId(oteAccount1)
.setType(Type.DOMAIN_TRANSFER_CANCEL)
.setXmlBytes(getBytes("domain_transfer_cancel.xml"))
@@ -140,7 +143,7 @@ public final class OteStatsTestHelper {
.build());
persistResource(
new DomainHistory.Builder()
.setDomain(persistActiveDomain("example.tld"))
.setDomain(exampleDomain)
.setRegistrarId(oteAccount1)
.setType(Type.DOMAIN_TRANSFER_REJECT)
.setXmlBytes(getBytes("domain_transfer_reject.xml"))
@@ -148,7 +151,7 @@ public final class OteStatsTestHelper {
.build());
persistResource(
new DomainHistory.Builder()
.setDomain(persistActiveDomain("example.tld"))
.setDomain(exampleDomain)
.setRegistrarId(oteAccount1)
.setType(Type.DOMAIN_TRANSFER_REQUEST)
.setXmlBytes(getBytes("domain_transfer_request.xml"))
@@ -156,7 +159,7 @@ public final class OteStatsTestHelper {
.build());
persistResource(
new DomainHistory.Builder()
.setDomain(persistActiveDomain("example.tld"))
.setDomain(exampleDomain)
.setRegistrarId(oteAccount1)
.setType(Type.DOMAIN_UPDATE)
.setXmlBytes(getBytes("domain_update_with_secdns.xml"))
@@ -189,4 +192,12 @@ public final class OteStatsTestHelper {
private static byte[] getBytes(String filename) throws IOException {
return loadBytes(OteStatsTestHelper.class, filename).read();
}
private static Domain loadOrCreateDomain(String domainName) {
return tm().transact(
() ->
EppResourceUtils.loadByForeignKey(
Domain.class, domainName, tm().getTransactionTime()))
.orElseGet(() -> persistActiveDomain(domainName));
}
}
@@ -20,6 +20,7 @@ import google.registry.config.CredentialModule;
import google.registry.config.RegistryConfig;
import google.registry.flows.ServerTridProviderModule;
import google.registry.flows.custom.CustomLogicFactoryModule;
import google.registry.flows.domain.DomainDeletionTimeCacheModule;
import google.registry.groups.GmailModule;
import google.registry.groups.GroupsModule;
import google.registry.groups.GroupssettingsModule;
@@ -43,6 +44,7 @@ import jakarta.inject.Singleton;
CredentialModule.class,
CustomLogicFactoryModule.class,
CloudTasksUtilsModule.class,
DomainDeletionTimeCacheModule.class,
FrontendRequestComponent.FrontendRequestComponentModule.class,
GmailModule.class,
GroupsModule.class,
@@ -434,11 +434,8 @@ public abstract class JpaTransactionManagerExtension
.setRegistrar(makeRegistrar2())
.setName("Marla Singer")
.setEmailAddress("Marla.Singer@crr.com")
.setRegistryLockEmailAddress("Marla.Singer.RegistryLock@crr.com")
.setPhoneNumber("+1.2128675309")
.setTypes(ImmutableSet.of(RegistrarPoc.Type.TECH))
.setAllowedToSetRegistryLockPassword(true)
.setRegistryLockPassword("hi")
.build();
}
@@ -112,6 +112,11 @@ class RdapJsonFormatterTest {
hostNotLinked =
makeAndPersistHost(
"ns5.cat.みんな", null, null, clock.nowUtc().minusYears(5), "unicoderegistrar");
// Create an unused domain that references hostBoth and hostNoAddresses so that
// they will have "associated" (ie, StatusValue.LINKED) status.
Domain dog =
persistResource(
makeDomain("dog.みんな", null, null, null, hostBoth, hostNoAddresses, registrar));
hostSuperordinatePendingTransfer =
persistResource(
makeAndPersistHost(
@@ -119,8 +124,7 @@ class RdapJsonFormatterTest {
.asBuilder()
.setSuperordinateDomain(
persistResource(
makeDomain("dog.みんな", null, null, null, null, null, registrar)
.asBuilder()
dog.asBuilder()
.addStatusValue(StatusValue.PENDING_TRANSFER)
.setTransferData(
new DomainTransferData.Builder()
@@ -150,9 +154,6 @@ class RdapJsonFormatterTest {
.setCreationTimeForTest(clock.nowUtc())
.setLastEppUpdateTime(null)
.build());
// Create an unused domain that references hostBoth and hostNoAddresses so that
// they will have "associated" (ie, StatusValue.LINKED) status.
persistResource(makeDomain("dog.みんな", null, null, null, hostBoth, hostNoAddresses, registrar));
// history entries
// We create 3 "transfer approved" entries, to make sure we only save the last one
@@ -253,7 +253,7 @@ class Spec11EmailUtilsTest {
// Create an inactive domain and an active domain with the same name.
persistResource(loadByEntity(aDomain).asBuilder().addStatusValue(SERVER_HOLD).build());
Host host = persistActiveHost("ns1.example.com");
aDomain = persistDomainWithHost("a.com", host);
aDomain = persistResource(aDomain.asBuilder().setNameservers(host.createVKey()).build());
emailUtils.emailSpec11Reports(
date,
@@ -54,6 +54,9 @@ public class OidcTokenAuthenticationMechanismTest {
private static final String rawToken = "this-token";
private static final String email = "user@email.test";
private static final String unknownEmail = "bad-guy@evil.real";
private static final String gaiaId = "gaia-id";
private static final ImmutableSet<String> serviceAccounts =
ImmutableSet.of("service@email.test", "email@service.goog");
@@ -148,13 +151,12 @@ public class OidcTokenAuthenticationMechanismTest {
payload.setEmail("service@email.test");
authResult = authenticationMechanism.authenticate(request);
assertThat(authResult.isAuthenticated()).isTrue();
assertThat(authResult.authLevel()).isEqualTo(AuthLevel.USER);
assertThat(authResult.user().get()).isEqualTo(serviceUser);
assertThat(authResult.authLevel()).isEqualTo(AuthLevel.APP);
}
@Test
void testAuthenticate_unknownEmailAddress() throws Exception {
payload.setEmail("bad-guy@evil.real");
payload.setEmail(unknownEmail);
authResult = authenticationMechanism.authenticate(request);
assertThat(authResult).isEqualTo(AuthResult.NOT_AUTHENTICATED);
}
@@ -50,7 +50,6 @@ class RegistrarPocTest {
.setRegistrar(testRegistrar)
.setName("Judith Registrar")
.setEmailAddress("judith.doe@example.com")
.setRegistryLockEmailAddress("judith.doe@external.com")
.setPhoneNumber("+1.2125650000")
.setFaxNumber("+1.2125650001")
.setTypes(ImmutableSet.of(WHOIS))
@@ -26,10 +26,16 @@ import com.google.common.net.HostAndPort;
import com.google.common.util.concurrent.SimpleTimeLimiter;
import google.registry.util.RegistryEnvironment;
import google.registry.util.UrlChecker;
import jakarta.servlet.MultipartConfigElement;
import jakarta.servlet.annotation.MultipartConfig;
import jakarta.servlet.http.HttpServlet;
import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URL;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.FutureTask;
@@ -49,6 +55,8 @@ import org.eclipse.jetty.server.ServerConnector;
* {@link #stop()} methods. However, a {@link #process()} method was added, which is used to process
* requests made to servlets (not static files) in the calling thread.
*
* <p>A servlet that expects multi-part requests should be annotated with {@link MultipartConfig}.
*
* <p><b>Note:</b> This server is intended for development purposes. For the love all that is good,
* do not make this public-facing.
*
@@ -70,6 +78,7 @@ public final class TestServer {
private final HostAndPort urlAddress;
private final Server server = new Server();
private final BlockingQueue<FutureTask<Void>> requestQueue = new LinkedBlockingDeque<>();
private List<Path> multiPartTmpDirs = new ArrayList<>();
/**
* Creates a new instance, but does not begin serving.
@@ -134,6 +143,13 @@ public final class TestServer {
},
SHUTDOWN_TIMEOUT_MS,
TimeUnit.MILLISECONDS);
for (var dir : multiPartTmpDirs) {
try {
Files.delete(dir);
} catch (Exception e) {
// Ignore
}
}
} catch (Exception e) {
throwIfUnchecked(e);
throw new RuntimeException(e);
@@ -161,7 +177,27 @@ public final class TestServer {
StaticResourceServlet.configureServletHolder(holder, runfile.getKey(), runfile.getValue());
}
for (Route route : routes) {
context.addServlet(wrapServlet(route.servletClass()), route.path());
holder = context.addServlet(wrapServlet(route.servletClass()), route.path());
MultipartConfig multipartConfig = route.servletClass().getAnnotation(MultipartConfig.class);
if (multipartConfig != null) {
try {
var location = multipartConfig.location();
if (location == null || location.isBlank()) {
Path tmpDir = Files.createTempDirectory("TestServer_");
multiPartTmpDirs.add(tmpDir);
location = tmpDir.toString();
}
MultipartConfigElement multipartConfigElement =
new MultipartConfigElement(
location,
multipartConfig.maxFileSize(),
multipartConfig.maxRequestSize(),
multipartConfig.fileSizeThreshold());
holder.getRegistration().setMultipartConfig(multipartConfigElement);
} catch (IOException e) {
throw new RuntimeException(e);
}
}
}
holder = context.addServlet(DefaultServlet.class, "/*");
holder.setInitParameter("aliases", "1");
@@ -76,11 +76,37 @@ class CreateCdnsTldTest extends CommandTestCase<CreateCdnsTld> {
@Test
@MockitoSettings(strictness = Strictness.LENIENT)
void testSandboxTldRestrictions() {
void testSandboxTldRestrictions_Disallowed() {
IllegalArgumentException thrown =
assertThrows(
IllegalArgumentException.class,
() -> runCommandInEnvironment(RegistryToolEnvironment.SANDBOX, "--dns_name=foobar."));
assertThat(thrown).hasMessageThat().contains("Sandbox TLDs must be of the form \"*.test.\"");
() ->
runCommandInEnvironment(
RegistryToolEnvironment.SANDBOX,
"--dns_name=foobar.",
"--description=test run",
"--force"));
assertThat(thrown)
.hasMessageThat()
.contains("Sandbox TLDs must be approved or in the form \"*.test.\"");
}
@Test
void testSandboxTldRestrictions_tldCheckSkipped() throws Exception {
runCommandInEnvironment(
RegistryToolEnvironment.SANDBOX,
"--dns_name=foobar.",
"--description=test run",
"--force",
"--skip_sandbox_tld_check");
}
@Test
void testSandboxTldRestrictions_testTld() throws Exception {
runCommandInEnvironment(
RegistryToolEnvironment.SANDBOX,
"--dns_name=abc.test.",
"--description=test run",
"--force");
}
}
@@ -43,10 +43,14 @@ class DeleteAllocationTokensCommandTest extends CommandTestCase<DeleteAllocation
private AllocationToken preNot2;
private AllocationToken othrRed;
private AllocationToken othrNot;
private Domain exampleDomain;
private Domain foobarDomain;
@BeforeEach
void beforeEach() {
createTlds("foo", "bar");
exampleDomain = persistActiveDomain("example.foo");
foobarDomain = persistActiveDomain("foo.bar");
preRed1 = persistToken("prefix12345AA", null, true);
preRed2 = persistToken("prefixgh8907a", null, true);
preNot1 = persistToken("prefix2978204", null, false);
@@ -97,8 +101,8 @@ class DeleteAllocationTokensCommandTest extends CommandTestCase<DeleteAllocation
@Test
void test_defaultOptions_doesntDeletePerDomainTokens() throws Exception {
AllocationToken preDom1 = persistToken("prefixasdfg897as", "foo.bar", false);
AllocationToken preDom2 = persistToken("prefix98HAZXadbn", "foo.bar", true);
AllocationToken preDom1 = persistToken("prefixasdfg897as", foobarDomain, false);
AllocationToken preDom2 = persistToken("prefix98HAZXadbn", foobarDomain, true);
runCommandForced("--prefix", "prefix");
assertNonexistent(preNot1, preNot2);
assertThat(reloadTokens(preRed1, preRed2, preDom1, preDom2, othrRed, othrNot))
@@ -107,8 +111,8 @@ class DeleteAllocationTokensCommandTest extends CommandTestCase<DeleteAllocation
@Test
void test_withDomains_doesDeletePerDomainTokens() throws Exception {
AllocationToken preDom1 = persistToken("prefixasdfg897as", "foo.bar", false);
AllocationToken preDom2 = persistToken("prefix98HAZXadbn", "foo.bar", true);
AllocationToken preDom1 = persistToken("prefixasdfg897as", foobarDomain, false);
AllocationToken preDom2 = persistToken("prefix98HAZXadbn", foobarDomain, true);
runCommandForced("--prefix", "prefix", "--with_domains");
assertNonexistent(preNot1, preNot2, preDom1);
assertThat(reloadTokens(preRed1, preRed2, preDom2, othrRed, othrNot))
@@ -164,17 +168,15 @@ class DeleteAllocationTokensCommandTest extends CommandTestCase<DeleteAllocation
assertThat(thrown).hasMessageThat().isEqualTo("Provided prefix should not be blank");
}
private static AllocationToken persistToken(
String token, @Nullable String domainName, boolean redeemed) {
private AllocationToken persistToken(String token, @Nullable Domain domain, boolean redeemed) {
AllocationToken.Builder builder =
new AllocationToken.Builder()
.setToken(token)
.setTokenType(SINGLE_USE)
.setDomainName(domainName);
.setDomainName(domain == null ? null : domain.getDomainName());
if (redeemed) {
String domainToPersist = domainName != null ? domainName : "example.foo";
Domain domain = persistActiveDomain(domainToPersist);
HistoryEntryId historyEntryId = new HistoryEntryId(domain.getRepoId(), 1051L);
String repoId = domain == null ? exampleDomain.getRepoId() : domain.getRepoId();
HistoryEntryId historyEntryId = new HistoryEntryId(repoId, 1051L);
builder.setRedemptionHistoryId(historyEntryId);
}
return persistResource(builder.build());
@@ -291,7 +291,7 @@ public final class DomainLockUtilsTest {
.setRegistrarId("TheRegistrar")
.setRepoId(domain.getRepoId())
.isSuperuser(false)
.setRegistrarPocId(POC_ID)
.setRegistryLockEmail(POC_ID)
.build());
clock.advanceOneMilli();
RegistryLock resultLock =
@@ -477,7 +477,7 @@ public final class DomainLockUtilsTest {
.setRepoId("repoId")
.setRelockDuration(standardHours(6))
.setRegistrarId("TheRegistrar")
.setRegistrarPocId("someone@example.com")
.setRegistryLockEmail("someone@example.com")
.setVerificationCode("hi")
.build());
domainLockUtils.enqueueDomainRelock(lock.getRelockDuration().get(), lock.getRevisionId(), 0);
@@ -504,7 +504,7 @@ public final class DomainLockUtilsTest {
.setDomainName("example.tld")
.setRepoId("repoId")
.setRegistrarId("TheRegistrar")
.setRegistrarPocId("someone@example.com")
.setRegistryLockEmail("someone@example.com")
.setVerificationCode("hi")
.build());
IllegalArgumentException thrown =
@@ -39,6 +39,7 @@ import google.registry.model.domain.fee.FeeQueryCommandExtensionItem.CommandName
import google.registry.model.domain.token.AllocationToken;
import google.registry.model.domain.token.AllocationToken.TokenStatus;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.testing.DatabaseHelper;
import google.registry.testing.DeterministicStringGenerator;
import google.registry.testing.DeterministicStringGenerator.Rule;
import google.registry.util.StringGenerator.Alphabets;
@@ -56,6 +57,7 @@ class GenerateAllocationTokensCommandTest extends CommandTestCase<GenerateAlloca
@BeforeEach
void beforeEach() {
DatabaseHelper.createTlds("tld", "example");
command.stringGenerator = new DeterministicStringGenerator(Alphabets.BASE_58);
}
@@ -531,6 +533,26 @@ class GenerateAllocationTokensCommandTest extends CommandTestCase<GenerateAlloca
.isEqualTo("For DEFAULT_PROMO tokens, must specify --token_status_transitions");
}
@Test
void testFailure_badTld() {
assertThat(
assertThrows(
IllegalArgumentException.class,
() -> runCommand("--number", "10", "--allowed_tlds", "badtld")))
.hasMessageThat()
.isEqualTo("Unknown REAL TLD(s) [badtld]");
}
@Test
void testFailure_badRegistrar() {
assertThat(
assertThrows(
IllegalArgumentException.class,
() -> runCommand("--number", "10", "--allowed_client_ids", "badregistrar")))
.hasMessageThat()
.isEqualTo("Unknown registrar ID(s) [badregistrar]");
}
private AllocationToken createToken(
String token,
@Nullable HistoryEntryId redemptionHistoryEntryId,
@@ -92,7 +92,6 @@ class RegistrarPocCommandTest extends CommandTestCase<RegistrarPocCommand> {
"--mode=UPDATE",
"--name=Judith Registrar",
"--email=judith.doe@example.com",
"--registry_lock_email=judith.doe@external.com",
"--phone=+1.2125650000",
"--fax=+1.2125650001",
"--contact_type=WHOIS",
@@ -108,7 +107,6 @@ class RegistrarPocCommandTest extends CommandTestCase<RegistrarPocCommand> {
.setRegistrar(registrar)
.setName("Judith Registrar")
.setEmailAddress("judith.doe@example.com")
.setRegistryLockEmailAddress("judith.doe@external.com")
.setPhoneNumber("+1.2125650000")
.setFaxNumber("+1.2125650001")
.setTypes(ImmutableSet.of(WHOIS))
@@ -255,7 +253,6 @@ class RegistrarPocCommandTest extends CommandTestCase<RegistrarPocCommand> {
"--mode=CREATE",
"--name=Jim Doe",
"--email=jim.doe@example.com",
"--registry_lock_email=jim.doe@external.com",
"--contact_type=ADMIN,ABUSE",
"--visible_in_whois_as_admin=true",
"--visible_in_whois_as_tech=false",
@@ -269,7 +266,6 @@ class RegistrarPocCommandTest extends CommandTestCase<RegistrarPocCommand> {
.setRegistrar(registrar)
.setName("Jim Doe")
.setEmailAddress("jim.doe@example.com")
.setRegistryLockEmailAddress("jim.doe@external.com")
.setTypes(ImmutableSet.of(ADMIN, ABUSE))
.setVisibleInWhoisAsAdmin(true)
.setVisibleInWhoisAsTech(false)
@@ -318,87 +314,6 @@ class RegistrarPocCommandTest extends CommandTestCase<RegistrarPocCommand> {
assertThat(loadRegistrar("NewRegistrar").getContactsRequireSyncing()).isTrue();
}
@Test
void testCreate_setAllowedToSetRegistryLockPassword() throws Exception {
runCommandForced(
"--mode=CREATE",
"--name=Jim Doe",
"--email=jim.doe@example.com",
"--registry_lock_email=jim.doe.registry.lock@example.com",
"--allowed_to_set_registry_lock_password=true",
"NewRegistrar");
RegistrarPoc registrarPoc = loadRegistrar("NewRegistrar").getContacts().asList().get(1);
assertThat(registrarPoc.isAllowedToSetRegistryLockPassword()).isTrue();
registrarPoc.asBuilder().setRegistryLockPassword("foo");
}
@Test
void testUpdate_setAllowedToSetRegistryLockPassword() throws Exception {
Registrar registrar = loadRegistrar("NewRegistrar");
RegistrarPoc registrarPoc =
persistResource(
new RegistrarPoc.Builder()
.setRegistrar(registrar)
.setName("Jim Doe")
.setEmailAddress("jim.doe@example.com")
.build());
assertThat(registrarPoc.isAllowedToSetRegistryLockPassword()).isFalse();
// First, try (and fail) to set the password directly
assertThrows(
IllegalArgumentException.class,
() -> registrarPoc.asBuilder().setRegistryLockPassword("foo"));
// Next, try (and fail) to allow registry lock without a registry lock email
assertThat(
assertThrows(
IllegalArgumentException.class,
() ->
runCommandForced(
"--mode=UPDATE",
"--email=jim.doe@example.com",
"--allowed_to_set_registry_lock_password=true",
"NewRegistrar")))
.hasMessageThat()
.isEqualTo("Registry lock email must not be null if allowing registry lock access");
// Next, include the email and it should succeed
runCommandForced(
"--mode=UPDATE",
"--email=jim.doe@example.com",
"--registry_lock_email=jim.doe.registry.lock@example.com",
"--allowed_to_set_registry_lock_password=true",
"NewRegistrar");
RegistrarPoc newContact = reloadResource(registrarPoc);
assertThat(newContact.isAllowedToSetRegistryLockPassword()).isTrue();
// should be allowed to set the password now
newContact.asBuilder().setRegistryLockPassword("foo");
}
@Test
void testUpdate_setAllowedToSetRegistryLockPassword_removesOldPassword() throws Exception {
Registrar registrar = loadRegistrar("NewRegistrar");
RegistrarPoc registrarPoc =
persistResource(
new RegistrarPoc.Builder()
.setRegistrar(registrar)
.setName("Jim Doe")
.setEmailAddress("jim.doe@example.com")
.setRegistryLockEmailAddress("jim.doe.registry.lock@example.com")
.setAllowedToSetRegistryLockPassword(true)
.setRegistryLockPassword("hi")
.build());
assertThat(registrarPoc.verifyRegistryLockPassword("hi")).isTrue();
assertThat(registrarPoc.verifyRegistryLockPassword("hello")).isFalse();
runCommandForced(
"--mode=UPDATE",
"--email=jim.doe@example.com",
"--allowed_to_set_registry_lock_password=true",
"NewRegistrar");
registrarPoc = reloadResource(registrarPoc);
assertThat(registrarPoc.verifyRegistryLockPassword("hi")).isFalse();
}
@Test
void testCreate_failure_badEmail() {
IllegalArgumentException thrown =
@@ -40,14 +40,21 @@ import google.registry.model.domain.fee.FeeQueryCommandExtensionItem.CommandName
import google.registry.model.domain.token.AllocationToken;
import google.registry.model.domain.token.AllocationToken.RegistrationBehavior;
import google.registry.model.domain.token.AllocationToken.TokenStatus;
import google.registry.testing.DatabaseHelper;
import org.joda.money.CurrencyUnit;
import org.joda.money.Money;
import org.joda.time.DateTime;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
/** Unit tests for {@link UpdateAllocationTokensCommand}. */
class UpdateAllocationTokensCommandTest extends CommandTestCase<UpdateAllocationTokensCommand> {
@BeforeEach
void beforeEach() {
DatabaseHelper.createTlds("tld", "example");
}
@Test
void testUpdateTlds_setTlds() throws Exception {
AllocationToken token =
@@ -64,14 +71,24 @@ class UpdateAllocationTokensCommandTest extends CommandTestCase<UpdateAllocation
assertThat(reloadResource(token).getAllowedTlds()).isEmpty();
}
@Test
void testUpdateTlds_badTlds() {
persistResource(builderWithPromo().build());
assertThat(
assertThrows(
IllegalArgumentException.class, () -> runCommandForced("--allowed_tlds=badtld")))
.hasMessageThat()
.isEqualTo("Unknown REAL TLD(s) [badtld]");
}
@Test
void testUpdateClientIds_setClientIds() throws Exception {
AllocationToken token =
persistResource(
builderWithPromo().setAllowedRegistrarIds(ImmutableSet.of("toRemove")).build());
runCommandForced("--prefix", "token", "--allowed_client_ids", "clientone,clienttwo");
runCommandForced("--prefix", "token", "--allowed_client_ids", "TheRegistrar,NewRegistrar");
assertThat(reloadResource(token).getAllowedRegistrarIds())
.containsExactly("clientone", "clienttwo");
.containsExactly("TheRegistrar", "NewRegistrar");
}
@Test
@@ -83,6 +100,17 @@ class UpdateAllocationTokensCommandTest extends CommandTestCase<UpdateAllocation
assertThat(reloadResource(token).getAllowedRegistrarIds()).isEmpty();
}
@Test
void testUpdateClientIds_badClientId() {
persistResource(builderWithPromo().build());
assertThat(
assertThrows(
IllegalArgumentException.class,
() -> runCommandForced("--allowed_client_ids=badregistrar")))
.hasMessageThat()
.isEqualTo("Unknown registrar ID(s) [badregistrar]");
}
@Test
void testUpdateEppActions_setEppActions() throws Exception {
AllocationToken token =
@@ -40,8 +40,10 @@ import org.junit.jupiter.api.Test;
class ConsoleHistoryDataActionTest extends ConsoleActionBaseTestCase {
private static final String SUPPORT_EMAIL = "supportEmailTest@test.com";
private static final Gson GSON = new Gson();
private User noPermissionUser;
private User registrarPrimaryUser;
@BeforeEach
void beforeEach() {
@@ -56,6 +58,17 @@ class ConsoleHistoryDataActionTest extends ConsoleActionBaseTestCase {
.build())
.build());
registrarPrimaryUser =
DatabaseHelper.persistResource(
new User.Builder()
.setEmailAddress("primary@example.com")
.setUserRoles(
new UserRoles.Builder()
.setRegistrarRoles(
ImmutableMap.of("TheRegistrar", RegistrarRole.PRIMARY_CONTACT))
.build())
.build());
DatabaseHelper.persistResources(
ImmutableList.of(
new ConsoleUpdateHistory.Builder()
@@ -85,7 +98,7 @@ class ConsoleHistoryDataActionTest extends ConsoleActionBaseTestCase {
}
@Test
void testSuccess_getByRegistrar() {
void testSuccess_getByRegistrar_notAnonymizedForSupportUser() {
ConsoleHistoryDataAction action =
createAction(AuthResult.createUser(fteUser), "TheRegistrar", Optional.empty());
action.run();
@@ -93,6 +106,35 @@ class ConsoleHistoryDataActionTest extends ConsoleActionBaseTestCase {
List<Map<String, Object>> payload = GSON.fromJson(response.getPayload(), List.class);
assertThat(payload.stream().map(record -> record.get("description")).collect(toImmutableList()))
.containsExactly("TheRegistrar|Some change", "TheRegistrar|Another change");
// Assert that the support user sees the real acting users
List<String> actingUserEmails =
payload.stream()
.map(record -> (Map<String, Object>) record.get("actingUser"))
.map(userMap -> (String) userMap.get("emailAddress"))
.collect(toImmutableList());
assertThat(actingUserEmails).containsExactly("fte@email.tld", "no.perms@example.com");
}
@Test
void testSuccess_getByRegistrar_anonymizedForRegistrarUser() {
ConsoleHistoryDataAction action =
createAction(AuthResult.createUser(registrarPrimaryUser), "TheRegistrar", Optional.empty());
action.run();
assertThat(response.getStatus()).isEqualTo(SC_OK);
List<Map<String, Object>> payload = GSON.fromJson(response.getPayload(), List.class);
assertThat(payload.stream().map(record -> record.get("description")).collect(toImmutableList()))
.containsExactly("TheRegistrar|Some change", "TheRegistrar|Another change");
// Assert that the registrar user sees the anonymized support user
List<String> actingUserEmails =
payload.stream()
.map(record -> (Map<String, Object>) record.get("actingUser"))
.map(userMap -> (String) userMap.get("emailAddress"))
.collect(toImmutableList());
assertThat(actingUserEmails).containsExactly(SUPPORT_EMAIL, "no.perms@example.com");
}
@Test
@@ -104,6 +146,13 @@ class ConsoleHistoryDataActionTest extends ConsoleActionBaseTestCase {
List<Map<String, Object>> payload = GSON.fromJson(response.getPayload(), List.class);
assertThat(payload.stream().map(record -> record.get("description")).collect(toImmutableList()))
.containsExactly("TheRegistrar|Some change", "OtherRegistrar|Some change");
List<String> actingUserEmails =
payload.stream()
.map(record -> (Map<String, Object>) record.get("actingUser"))
.map(userMap -> (String) userMap.get("emailAddress"))
.collect(toImmutableList());
assertThat(actingUserEmails).containsExactly("fte@email.tld", "fte@email.tld");
}
@Test
@@ -148,6 +197,7 @@ class ConsoleHistoryDataActionTest extends ConsoleActionBaseTestCase {
consoleApiParams = ConsoleApiParamsUtils.createFake(authResult);
when(consoleApiParams.request().getMethod()).thenReturn("GET");
response = (FakeResponse) consoleApiParams.response();
return new ConsoleHistoryDataAction(consoleApiParams, registrarId, consoleUserEmail);
return new ConsoleHistoryDataAction(
consoleApiParams, SUPPORT_EMAIL, registrarId, consoleUserEmail);
}
}
@@ -75,7 +75,8 @@ public class ConsoleRegistryLockActionTest extends ConsoleActionBaseTestCase {
Note: this code will expire in one hour.
https://registrarconsole.tld/console/#/registry-lock-verify?lockVerificationCode=\
123456789ABCDEFGHJKLMNPQRSTUVWXY""";
123456789ABCDEFGHJKLMNPQRSTUVWXY\
""";
@Mock GmailClient gmailClient;
private ConsoleRegistryLockAction action;
@@ -112,8 +113,8 @@ public class ConsoleRegistryLockActionTest extends ConsoleActionBaseTestCase {
assertThat(response.getStatus()).isEqualTo(SC_OK);
assertThat(response.getPayload())
.isEqualTo(
"""
[{"domainName":"example.test","registrarPocId":"johndoe@theregistrar.com","lockRequestTime":\
"""
[{"domainName":"example.test","registryLockEmail":"johndoe@theregistrar.com","lockRequestTime":\
{"creationTime":"2024-04-15T00:00:00.000Z"},"unlockRequestTime":"null","lockCompletionTime":\
"2024-04-15T00:00:00.000Z","unlockCompletionTime":"null","isSuperuser":false}]\
""");
@@ -127,7 +128,7 @@ public class ConsoleRegistryLockActionTest extends ConsoleActionBaseTestCase {
.setDomainName("expired.test")
.setRegistrarId("TheRegistrar")
.setVerificationCode("123456789ABCDEFGHJKLMNPQRSTUVWXY")
.setRegistrarPocId("johndoe@theregistrar.com")
.setRegistryLockEmail("johndoe@theregistrar.com")
.build();
saveRegistryLock(expiredLock);
RegistryLock expiredUnlock =
@@ -136,7 +137,7 @@ public class ConsoleRegistryLockActionTest extends ConsoleActionBaseTestCase {
.setDomainName("expiredunlock.test")
.setRegistrarId("TheRegistrar")
.setVerificationCode("123456789ABCDEFGHJKLMNPQRSTUVWXY")
.setRegistrarPocId("johndoe@theregistrar.com")
.setRegistryLockEmail("johndoe@theregistrar.com")
.setLockCompletionTime(clock.nowUtc())
.setUnlockRequestTime(clock.nowUtc())
.build();
@@ -149,7 +150,7 @@ public class ConsoleRegistryLockActionTest extends ConsoleActionBaseTestCase {
.setDomainName("example.test")
.setRegistrarId("TheRegistrar")
.setVerificationCode("123456789ABCDEFGHJKLMNPQRSTUVWXY")
.setRegistrarPocId("johndoe@theregistrar.com")
.setRegistryLockEmail("johndoe@theregistrar.com")
.setLockCompletionTime(clock.nowUtc())
.build();
clock.advanceOneMilli();
@@ -168,7 +169,7 @@ public class ConsoleRegistryLockActionTest extends ConsoleActionBaseTestCase {
.setDomainName("pending.test")
.setRegistrarId("TheRegistrar")
.setVerificationCode("111111111ABCDEFGHJKLMNPQRSTUVWXY")
.setRegistrarPocId("johndoe@theregistrar.com")
.setRegistryLockEmail("johndoe@theregistrar.com")
.build();
RegistryLock incompleteUnlock =
@@ -177,7 +178,7 @@ public class ConsoleRegistryLockActionTest extends ConsoleActionBaseTestCase {
.setDomainName("incompleteunlock.test")
.setRegistrarId("TheRegistrar")
.setVerificationCode("123456789ABCDEFGHJKLMNPQRSTUVWXY")
.setRegistrarPocId("johndoe@theregistrar.com")
.setRegistryLockEmail("johndoe@theregistrar.com")
.setLockCompletionTime(clock.nowUtc())
.setUnlockRequestTime(clock.nowUtc())
.build();
@@ -187,7 +188,7 @@ public class ConsoleRegistryLockActionTest extends ConsoleActionBaseTestCase {
.setRepoId("repoId")
.setDomainName("unlocked.test")
.setRegistrarId("TheRegistrar")
.setRegistrarPocId("johndoe@theregistrar.com")
.setRegistryLockEmail("johndoe@theregistrar.com")
.setVerificationCode("123456789ABCDEFGHJKLMNPQRSTUUUUU")
.setLockCompletionTime(clock.nowUtc())
.setUnlockRequestTime(clock.nowUtc())
@@ -206,26 +207,27 @@ public class ConsoleRegistryLockActionTest extends ConsoleActionBaseTestCase {
// locks or completed unlocks
assertThat(response.getPayload())
.isEqualTo(
"""
"""
[{"domainName":"adminexample.test","lockRequestTime":{"creationTime":"2024-04-16T00:00:00.001Z"},\
"unlockRequestTime":"null","lockCompletionTime":"2024-04-16T00:00:00.001Z","unlockCompletionTime":\
"null","isSuperuser":true},\
\
{"domainName":"example.test","registrarPocId":"johndoe@theregistrar.com","lockRequestTime":\
{"domainName":"example.test","registryLockEmail":"johndoe@theregistrar.com","lockRequestTime":\
{"creationTime":"2024-04-16T00:00:00.001Z"},"unlockRequestTime":"null","lockCompletionTime":\
"2024-04-16T00:00:00.000Z","unlockCompletionTime":"null","isSuperuser":false},\
\
{"domainName":"expiredunlock.test","registrarPocId":"johndoe@theregistrar.com","lockRequestTime":\
{"domainName":"expiredunlock.test","registryLockEmail":"johndoe@theregistrar.com","lockRequestTime":\
{"creationTime":"2024-04-15T00:00:00.000Z"},"unlockRequestTime":"2024-04-15T00:00:00.000Z",\
"lockCompletionTime":"2024-04-15T00:00:00.000Z","unlockCompletionTime":"null","isSuperuser":false},\
\
{"domainName":"incompleteunlock.test","registrarPocId":"johndoe@theregistrar.com","lockRequestTime":\
{"domainName":"incompleteunlock.test","registryLockEmail":"johndoe@theregistrar.com","lockRequestTime":\
{"creationTime":"2024-04-16T00:00:00.001Z"},"unlockRequestTime":"2024-04-16T00:00:00.001Z",\
"lockCompletionTime":"2024-04-16T00:00:00.001Z","unlockCompletionTime":"null","isSuperuser":false},\
\
{"domainName":"pending.test","registrarPocId":"johndoe@theregistrar.com","lockRequestTime":\
{"domainName":"pending.test","registryLockEmail":"johndoe@theregistrar.com","lockRequestTime":\
{"creationTime":"2024-04-16T00:00:00.001Z"},"unlockRequestTime":"null","lockCompletionTime":"null",\
"unlockCompletionTime":"null","isSuperuser":false}]""");
"unlockCompletionTime":"null","isSuperuser":false}]\
""");
}
@Test
@@ -500,7 +502,7 @@ public class ConsoleRegistryLockActionTest extends ConsoleActionBaseTestCase {
.setRepoId(defaultDomain.getRepoId())
.setDomainName(defaultDomain.getDomainName())
.setRegistrarId(defaultDomain.getCurrentSponsorRegistrarId())
.setRegistrarPocId("johndoe@theregistrar.com")
.setRegistryLockEmail("johndoe@theregistrar.com")
.setVerificationCode("123456789ABCDEFGHJKLMNPQRSTUUUUU");
}
@@ -187,7 +187,7 @@ public class ConsoleRegistryLockVerifyActionTest extends ConsoleActionBaseTestCa
.setRepoId(defaultDomain.getRepoId())
.setDomainName(defaultDomain.getDomainName())
.setRegistrarId(defaultDomain.getCurrentSponsorRegistrarId())
.setRegistrarPocId("johndoe@theregistrar.com")
.setRegistryLockEmail("johndoe@theregistrar.com")
.setVerificationCode(DEFAULT_CODE);
}
@@ -170,7 +170,26 @@ class ConsoleUsersActionTest extends ConsoleActionBaseTestCase {
createAction(
Optional.of(ConsoleApiParamsUtils.createFake(authResult)),
Optional.of("POST"),
Optional.of(new UserData("lol", null, RegistrarRole.ACCOUNT_MANAGER.toString(), null)));
Optional.of(new UserData("lol", null, RegistrarRole.TECH_CONTACT.name(), null)));
action.cloudTasksUtils = cloudTasksHelper.getTestCloudTasksUtils();
when(directory.users()).thenReturn(users);
when(users.insert(any(com.google.api.services.directory.model.User.class))).thenReturn(insert);
action.run();
assertThat(response.getStatus()).isEqualTo(SC_CREATED);
assertThat(response.getPayload())
.contains(
"{\"emailAddress\":\"lol.TheRegistrar@email.com\",\"role\":\"TECH_CONTACT\",\"password\":\"abcdefghijklmnop\"}");
}
@Test
void testSuccess_roleEnforcementCreate() throws IOException {
User user = DatabaseHelper.createAdminUser("email@email.com");
AuthResult authResult = AuthResult.createUser(user);
ConsoleUsersAction action =
createAction(
Optional.of(ConsoleApiParamsUtils.createFake(authResult)),
Optional.of("POST"),
Optional.of(new UserData("lol", null, RegistrarRole.PRIMARY_CONTACT.name(), null)));
action.cloudTasksUtils = cloudTasksHelper.getTestCloudTasksUtils();
when(directory.users()).thenReturn(users);
when(users.insert(any(com.google.api.services.directory.model.User.class))).thenReturn(insert);
@@ -284,39 +284,27 @@ class ContactActionTest extends ConsoleActionBaseTestCase {
"Registrar New Registrar (registrarId) updated in registry unittest"
+ " environment")
.setBody(
"The following changes were made in registry unittest environment to the"
+ " registrar registrarId by admin fte@email.tld:\n"
+ "\n"
+ "contacts:\n"
+ " ADDED:\n"
+ " {id="
+ id
+ ", name=Test Registrar 2,"
+ " emailAddress=incorrect@example.com, registrarId=registrarId,"
+ " registryLockEmailAddress=null, phoneNumber=+1.1234567890,"
+ " faxNumber=+1.1234567891, types=[TECH],"
+ " visibleInWhoisAsAdmin=false, visibleInWhoisAsTech=true,"
+ " visibleInDomainWhoisAsAbuse=false,"
+ " allowedToSetRegistryLockPassword=false}\n"
+ " REMOVED:\n"
+ " {id="
+ id
+ ", name=Test Registrar 2, emailAddress=test.registrar2@example.com,"
+ " registrarId=registrarId, registryLockEmailAddress=null,"
+ " phoneNumber=+1.1234567890, faxNumber=+1.1234567891, types=[TECH],"
+ " visibleInWhoisAsAdmin=false,"
+ " visibleInWhoisAsTech=true, visibleInDomainWhoisAsAbuse=false,"
+ " allowedToSetRegistryLockPassword=false}\n"
+ " FINAL CONTENTS:\n"
+ " {id="
+ id
+ ", name=Test Registrar 2,"
+ " emailAddress=incorrect@example.com, registrarId=registrarId,"
+ " registryLockEmailAddress=null, phoneNumber=+1.1234567890,"
+ " faxNumber=+1.1234567891, types=[TECH],"
+ " visibleInWhoisAsAdmin=false, visibleInWhoisAsTech=true,"
+ " visibleInDomainWhoisAsAbuse=false,"
+ " allowedToSetRegistryLockPassword=false}\n")
"""
The following changes were made in registry unittest environment to the registrar registrarId \
by admin fte@email.tld:
contacts:
ADDED:
{id=5, name=Test Registrar 2, emailAddress=incorrect@example.com, registrarId=registrarId, \
phoneNumber=+1.1234567890, faxNumber=+1.1234567891, types=[TECH], visibleInWhoisAsAdmin=false, \
visibleInWhoisAsTech=true, visibleInDomainWhoisAsAbuse=false, \
allowedToSetRegistryLockPassword=false}
REMOVED:
{id=5, name=Test Registrar 2, emailAddress=test.registrar2@example.com, \
registrarId=registrarId, phoneNumber=+1.1234567890, faxNumber=+1.1234567891, types=[TECH], \
visibleInWhoisAsAdmin=false, visibleInWhoisAsTech=true, visibleInDomainWhoisAsAbuse=false, \
allowedToSetRegistryLockPassword=false}
FINAL CONTENTS:
{id=5, name=Test Registrar 2, emailAddress=incorrect@example.com, registrarId=registrarId, \
phoneNumber=+1.1234567890, faxNumber=+1.1234567891, types=[TECH], visibleInWhoisAsAdmin=false, \
visibleInWhoisAsTech=true, visibleInDomainWhoisAsAbuse=false, \
allowedToSetRegistryLockPassword=false}
""")
.setRecipients(ImmutableList.of(new InternetAddress("notification@test.example")))
.build());
}
@@ -26,6 +26,7 @@ BACKEND /_dr/task/rdeUpload RdeUploadAction
BACKEND /_dr/task/readDnsRefreshRequests ReadDnsRefreshRequestsAction POST y APP ADMIN
BACKEND /_dr/task/refreshDnsOnHostRename RefreshDnsOnHostRenameAction POST n APP ADMIN
BACKEND /_dr/task/relockDomain RelockDomainAction POST y APP ADMIN
BACKEND /_dr/task/removeAllDomainContacts RemoveAllDomainContactsAction POST n APP ADMIN
BACKEND /_dr/task/resaveAllEppResourcesPipeline ResaveAllEppResourcesPipelineAction GET n APP ADMIN
BACKEND /_dr/task/resaveEntity ResaveEntityAction POST n APP ADMIN
BACKEND /_dr/task/sendExpiringCertificateNotificationEmail SendExpiringCertificateNotificationEmailAction GET n APP ADMIN
@@ -44,6 +44,7 @@ BACKEND /_dr/task/readDnsRefreshRequests ReadDnsRefreshReques
BACKEND /_dr/task/refreshDnsForAllDomains RefreshDnsForAllDomainsAction GET n APP ADMIN
BACKEND /_dr/task/refreshDnsOnHostRename RefreshDnsOnHostRenameAction POST n APP ADMIN
BACKEND /_dr/task/relockDomain RelockDomainAction POST y APP ADMIN
BACKEND /_dr/task/removeAllDomainContacts RemoveAllDomainContactsAction POST n APP ADMIN
BACKEND /_dr/task/resaveAllEppResourcesPipeline ResaveAllEppResourcesPipelineAction GET n APP ADMIN
BACKEND /_dr/task/resaveEntity ResaveEntityAction POST n APP ADMIN
BACKEND /_dr/task/sendExpiringCertificateNotificationEmail SendExpiringCertificateNotificationEmailAction GET n APP ADMIN
@@ -1,6 +1,6 @@
{
"objectClassName" : "nameserver",
"handle" : "C-ROID",
"handle" : "D-ROID",
"ldhName" : "ns1.dog.xn--q9jyb4c",
"unicodeName" : "ns1.dog.みんな",
"status" : ["active", "pending transfer"],

Some files were not shown because too many files have changed in this diff Show More