1
0
mirror of https://github.com/google/nomulus synced 2026-07-06 16:16:38 +00:00

Compare commits

...

21 Commits

Author SHA1 Message Date
Juan Celhay ff4c326ebe Delete step to push to release repo, trigger next release steps based on tag format (#2833)
* Change release cb file

* Add brackets around tag variable

* Redo tag matching

* Have tag matcher like the one in cb dev
2025-10-21 18:52:42 +00:00
Pavlo Tkach 51b579871a Anonymize support users in console history, add minor UI updates (#2851) 2025-10-17 18:57:40 +00:00
gbrodman b144aafb22 Use transaction time for deletion time cache ticker (#2848)
Basically, what happened is that the cache's expireAfterWrite was being
called some number of milliseconds (say, 50-100) after the transaction
was started. That method used the transaction time instead of the
current time, so as a result the entries were sticking around 50-100ms
longer in the cache than they should have been.

This fix contains two parts, each of which I believe would be sufficient
on their own to fix the issue:
1. Use the currentTime passed in in Expiry::expireAfterCreate
2. Use the transaction time in the cache's Ticker. This keeps everything
   on the same schedule.
2025-10-16 20:01:17 +00:00
Weimin Yu ddd955e156 Fix dependency of Gradle task for schema test (#2849)
Problem not showing up because all use cases run this test after
`build`.
2025-10-16 15:33:28 +00:00
gbrodman 6863f678f1 Allow Gradle to use more heap space (#2847)
During the release process, we are seeing the message "Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)" which seemingly can be caused by OOMs
2025-10-13 18:25:08 +00:00
gbrodman 6bd90e967b Add more hash indexes used during common flows (#2845)
I analyzed SQL statements run during the following flows and EXPLAIN
ANALYZEd each of them to figure out if there are any additional hash
indexes we could add that could be particularly helpful. Note: it's not
worth adding a hash index on the host_repo_id field in DomainHost
because so many rows (domains) use the same host.

- domain create
- domain delete
- domain info
- domain renew
- domain update
- host create
- host delete
- host update

I skipped the ones that use the read-only replica, as well as contact
flows (we're getting rid of them), and domain transfer/restore-related
flows as those are extremely infrequent.
2025-10-13 18:07:47 +00:00
gbrodman 5faf3d283c Differentiate between inserts and updates in flows (#2846)
Updates (AKA merges) run an extra SELECT statement to figure out if the
resource exists so that it can merge the entity into the existing object
in Hibernate's schema. When we're inserting new rows (such as new poll
messages or resource creates), we know that we don't need to do that
merge. Doing this should save us some SELECT statements (this has borne
out to be the truth in alpha)
2025-10-13 15:43:18 +00:00
gbrodman 149fb66ac5 Add cache for deletion times of existing domains (#2840)
This should help in instances of popular domains dropping, since we
won't need to do an additional two database loads every time (assuming
the deletion time is in the future).
2025-10-09 17:22:24 +00:00
gbrodman 8c96940a27 Only load from ClaimsList once when filling the cache (#2843) 2025-10-09 16:57:21 +00:00
Ben McIlwain 9c5510f05d Add a rate limiter to remove all domain contacts action (#2838)
The maximum QPS defaults to 10, but can also be specified at runtime through
use of a query-string parameter.

BUG = http://b/439636188
2025-10-02 22:15:19 +00:00
gbrodman 84884de77b Verify existence of TLDs and registrars for tokens (#2837)
Just in case someone makes a typo when running the commands
2025-10-02 20:10:58 +00:00
Ben McIlwain d6c35df9bc Ignore single domain failures in remove contacts from all domains action (#2836)
When running the action in sandbox on 1.5M domains, it failed a few times
updating individual domains (requiring a manual restart of the entire action).
It's better to just log the individual failures for manual inspection and then
otherwise continue running the action to process the vast majority of other
updates that won't fail.

BUG = http://b/439636188
2025-10-02 18:58:23 +00:00
Juan Celhay 7caa0ec9d6 Add environment configuration files to .gitignore (#2830)
* Add environment configuration files to .gitignore

* Delete config files from repo

* Refactor release cb file to delete config file lines from gitignore

* Reorder env files

* Add README for config files
2025-10-02 18:36:43 +00:00
Weimin Yu ee3866ec4a Allow top level tld creation in Sandbox (#2835)
Add a flag to the CreateCdnsTld command to bypass the dns name format
check in Sandbox (limiting names to `*.test.`). With this flag, we
can create TLDs for RST testing in Sandbox.

Note that if the new flag is wrongly set for a disallowed name, the
request to the Cloud DNS API will fail. The format check in the command
just provides a user-friendly error message.
2025-10-01 14:20:33 +00:00
gbrodman 97d0b7680f Add hash indexes for common use cases (#2834)
I went through all the SQL statements generated by some sample
DomainCreateFlow and DomainDeleteFlow cases to find situations where we
were either SELECTing from, or UPDATEing, tables with a direct "field =
value" format. These are the situations that I found where we can add
hash indexes. This does two things:

1. Makes these queries slight faster, since these are usually queries on
   columns that are either unique or very close to unique, and O(1) is
   faster than O(log(n))
2. Spreads around the optimistic predicate locks on the previously-used
   btree indexes. Many of our serialization errors came from the fact
   that we were autogenerating incrementing ID values for various
   tables, meaning that SELECTs, INSERTs, and UPDATEs would all try to
   take predicate locks out on the same page of the btree index. Using a
   hash index means that the page locks will be spread out to various
   index pages, rather than conflicting with each other.

Running load tests on alpha I see significant improvements in speed and
error rates. Speed is hard to quantify due to the nature of the way the
load tests distribute tasks among the queues but it could be more than
50% improvement, and serialization errors in the logs drop by more than
90%.
2025-09-29 22:16:24 +00:00
Pavlo Tkach 5700a008d6 Add console history frontend (#2832) 2025-09-26 21:25:03 +00:00
Ben McIlwain dc9f5b99bc Add a batch action to remove all contacts from domains (#2827)
This implements the first part of Minimum Data Set phase 3, wherein we delete
all contact data. This action is necessary to leave a permanent record on the
domain (in the form of a domain history entry) documenting when the contacts
were removed by the administrative user.

Then, after this has finished removing all contact assocations, we can simply
empty out or drop the Contact/ContactHistory tables and associated join tables.
2025-09-25 20:47:17 +00:00
Ben McIlwain d3c6de7a38 Modify the base Latin LGR with our intended changes to improve security (#2829) 2025-09-24 21:04:37 +00:00
Ben McIlwain 3c3303c16a Add ICANN's reference Latin LGR in RFC 7940 XML format (#2828)
In the next commit I will make changes to this file so it supports just the
basic Latin characters that we want, but it's good to check the base version in
so that we can see diffs.

This was downloaded from https://www.icann.org/sites/default/files/packages/lgr/lgr-second-level-latin-script-25oct24-en.xml
2025-09-19 16:42:46 +00:00
Nilay Shah 2a86a1bbe9 Skip user loading for proxy service account (#2825)
* Skip user loading for proxy service account

Reduces database load by skipping the User entity lookup for the proxy
service account during OIDC authentication.

The high volume of EPP "hello" and "login" commands from the proxy
service account results in a constant database load. These lookups
are unnecessary as the proxy service account is not expected to have a
corresponding User object.

This change optimizes the authentication flow by checking for the proxy
service account email *before* attempting to load a User from the
database. This bypasses the database transaction entirely for these
high-volume requests.

This approach is more efficient than caching, as it eliminates the
database lookup for the proxy service account altogether, rather than
just caching the result.

* comment added and service account llokup time improved

* comment updated for more clarity
2025-09-16 18:48:39 +00:00
gbrodman ea148ac13e Show success message on password reset (#2826) 2025-09-16 18:39:19 +00:00
97 changed files with 2902 additions and 382 deletions
+7
View File
@@ -18,6 +18,13 @@ gjf.out
# virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml
hs_err_pid*
# Environment-specific configuration files
core/src/main/java/google/registry/config/files/nomulus-config-alpha.yaml
core/src/main/java/google/registry/config/files/nomulus-config-crash.yaml
core/src/main/java/google/registry/config/files/nomulus-config-production.yaml
core/src/main/java/google/registry/config/files/nomulus-config-qa.yaml
core/src/main/java/google/registry/config/files/nomulus-config-sandbox.yaml
######################################################################
# Eclipse Ignores
+1 -1
View File
@@ -56,7 +56,7 @@ PROPERTIES_HEADER = """\
# nom_build), run ./nom_build --help.
#
# DO NOT EDIT THIS FILE BY HAND
org.gradle.jvmargs=-Xmx1024m
org.gradle.jvmargs=-Xmx2048m
org.gradle.caching=true
org.gradle.parallel=true
"""
+1 -1
View File
@@ -1,7 +1,7 @@
{
"/console-api":
{
"target": "http://localhost:8080",
"target": "http://[::1]:8080",
"secure": false,
"logLevel": "debug",
"changeOrigin": true
+7 -1
View File
@@ -26,6 +26,7 @@ import SecurityComponent from './settings/security/security.component';
import { SettingsComponent } from './settings/settings.component';
import { SupportComponent } from './support/support.component';
import RdapComponent from './settings/rdap/rdap.component';
import { HistoryComponent } from './history/history.component';
import { PasswordResetVerifyComponent } from './shared/components/passwordReset/passwordResetVerify.component';
export interface RouteWithIcon extends Route {
@@ -64,13 +65,18 @@ export const routes: RouteWithIcon[] = [
title: 'Dashboard',
iconName: 'view_comfy_alt',
},
// { path: 'tlds', component: TldsComponent, title: "TLDs", iconName: "event_list" },
{
path: DomainListComponent.PATH,
component: DomainListComponent,
title: 'Domains',
iconName: 'view_list',
},
{
path: HistoryComponent.PATH,
component: HistoryComponent,
// title: 'History',
// iconName: 'history',
},
{
path: SettingsComponent.PATH,
component: SettingsComponent,
+4 -2
View File
@@ -56,13 +56,14 @@ import { GlobalLoaderService } from './shared/services/globalLoader.service';
import { UserDataService } from './shared/services/userData.service';
import { SnackBarModule } from './snackbar.module';
import { SupportComponent } from './support/support.component';
import { TldsComponent } from './tlds/tlds.component';
import { ForceFocusDirective } from './shared/directives/forceFocus.directive';
import RdapComponent from './settings/rdap/rdap.component';
import RdapEditComponent from './settings/rdap/rdapEdit.component';
import { PocReminderComponent } from './shared/components/pocReminder/pocReminder.component';
import { PasswordResetVerifyComponent } from './shared/components/passwordReset/passwordResetVerify.component';
import { PasswordInputForm } from './shared/components/passwordReset/passwordInputForm.component';
import { HistoryComponent } from './history/history.component';
import { HistoryListComponent } from './history/historyList.component';
@NgModule({
declarations: [SelectedRegistrarWrapper],
@@ -81,6 +82,8 @@ export class SelectedRegistrarModule {}
EppPasswordEditComponent,
ForceFocusDirective,
HeaderComponent,
HistoryComponent,
HistoryListComponent,
HomeComponent,
LocationBackDirective,
NavigationComponent,
@@ -104,7 +107,6 @@ export class SelectedRegistrarModule {}
SettingsComponent,
SettingsContactComponent,
SupportComponent,
TldsComponent,
UserLevelVisibility,
],
bootstrap: [AppComponent],
@@ -0,0 +1,62 @@
<app-selected-registrar-wrapper>
<div class="history-log">
<h1 class="mat-headline-4" forceFocus>
Registrar Console Activity History
</h1>
<mat-tab-group
[elementId]="getElementIdForUserLog()"
class="history-log__tabs"
>
<mat-tab label="Registrar Activity">
<div class="spacer"></div>
<app-history-list
[historyRecords]="historyService.historyRecordsRegistrar()"
[isLoading]="isLoading"
/>
</mat-tab>
<mat-tab label="User Activity">
<div class="spacer"></div>
<form (ngSubmit)="loadHistory()" #form="ngForm">
<section>
<mat-form-field appearance="outline">
<mat-label>Console User Email: </mat-label>
<input
matInput
id="email"
type="email"
name="consoleUserEmail"
required
email
[(ngModel)]="consoleUserEmail"
#emailControl="ngModel"
/>
</mat-form-field>
</section>
<div class="spacer"></div>
<button
mat-flat-button
color="primary"
type="submit"
aria-label="Search user history"
[disabled]="!form.valid"
>
Search
</button>
</form>
<div class="spacer"></div>
<app-history-list
[historyRecords]="historyService.historyRecordsUser()"
[isLoading]="isLoading"
/>
</mat-tab>
</mat-tab-group>
</div>
<app-history-list
[elementId]="getElementIdForUserLog()"
[isReverse]="true"
[historyRecords]="historyService.historyRecordsUser()"
[isLoading]="isLoading"
/>
</app-selected-registrar-wrapper>
@@ -1,4 +1,4 @@
// Copyright 2024 The Nomulus Authors. All Rights Reserved.
// Copyright 2025 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
@@ -12,17 +12,11 @@
// See the License for the specific language governing permissions and
// limitations under the License.
.console-tlds {
&__cards {
display: flex;
border-top: 1px solid #ddd;
padding: 1rem;
}
&__card {
max-width: 300px;
}
&__card-links {
display: flex;
flex-direction: column;
.history-log {
font-family: "Roboto", sans-serif;
max-width: 760px;
.spacer {
margin: 20px 0;
}
}
@@ -0,0 +1,80 @@
// Copyright 2025 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { Component, effect } from '@angular/core';
import { UserDataService } from '../shared/services/userData.service';
import { BackendService } from '../shared/services/backend.service';
import { RegistrarService } from '../registrar/registrar.service';
import { HistoryService } from './history.service';
import { MatSnackBar } from '@angular/material/snack-bar';
import {
GlobalLoader,
GlobalLoaderService,
} from '../shared/services/globalLoader.service';
import { HttpErrorResponse } from '@angular/common/http';
import { RESTRICTED_ELEMENTS } from '../shared/directives/userLevelVisiblity.directive';
@Component({
selector: 'app-history',
templateUrl: './history.component.html',
styleUrls: ['./history.component.scss'],
providers: [HistoryService],
standalone: false,
})
export class HistoryComponent implements GlobalLoader {
public static PATH = 'history';
consoleUserEmail: string = '';
isLoading: boolean = false;
constructor(
private backendService: BackendService,
private registrarService: RegistrarService,
protected historyService: HistoryService,
protected globalLoader: GlobalLoaderService,
protected userDataService: UserDataService,
private _snackBar: MatSnackBar
) {
effect(() => {
if (registrarService.registrarId()) {
this.loadHistory();
}
});
}
getElementIdForUserLog() {
return RESTRICTED_ELEMENTS.ACTIVITY_PER_USER;
}
loadingTimeout() {
this._snackBar.open('Timeout loading records history');
}
loadHistory() {
this.globalLoader.startGlobalLoader(this);
this.isLoading = true;
this.historyService
.getHistoryLog(this.registrarService.registrarId(), this.consoleUserEmail)
.subscribe({
error: (err: HttpErrorResponse) => {
this._snackBar.open(err.error || err.message);
this.isLoading = false;
},
next: () => {
this.globalLoader.stopGlobalLoader(this);
this.isLoading = false;
},
});
}
}
@@ -0,0 +1,46 @@
// Copyright 2025 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { Injectable, signal } from '@angular/core';
import { BackendService } from '../shared/services/backend.service';
import { tap } from 'rxjs';
export interface HistoryRecord {
modificationTime: string;
type: string;
description: string;
actingUser: {
emailAddress: string;
};
}
@Injectable()
export class HistoryService {
historyRecordsRegistrar = signal<HistoryRecord[]>([]);
historyRecordsUser = signal<HistoryRecord[]>([]);
constructor(private backendService: BackendService) {}
getHistoryLog(registrarId: string, userEmail?: string) {
return this.backendService.getHistoryLog(registrarId, userEmail).pipe(
tap((historyRecords: HistoryRecord[]) => {
if (userEmail) {
this.historyRecordsUser.set(historyRecords);
} else {
this.historyRecordsRegistrar.set(historyRecords);
}
})
);
}
}
@@ -0,0 +1,50 @@
@if (!isLoading && historyRecords.length == 0) {
<div class="history-list__no-records">
<mat-icon class="history-list__no-records-icon secondary-text"
>apps_outage</mat-icon
>
<h1>No records found</h1>
</div>
} @else {
<mat-card>
<mat-card-content>
<mat-list role="list">
<ng-container *ngFor="let item of historyRecords; let last = last">
<mat-list-item class="history-list__item">
<mat-icon
[ngClass]="getIconClass(item.type)"
class="history-list__icon"
>
{{ getIconForType(item.type) }}
</mat-icon>
<div class="history-list__content">
<div class="history-list__description">
<span class="history-list__description--main">{{
item.type
}}</span>
<div>
<mat-chip
*ngIf="parseDescription(item.description).detail"
class="history-list__chip"
>
{{ parseDescription(item.description).detail }}
</mat-chip>
</div>
</div>
<div class="history-list__user">
<b>User - {{ item.actingUser.emailAddress }}</b>
</div>
</div>
<span class="history-list__timestamp">
{{ item.modificationTime | date : "MMM d, y, h:mm a" }}
</span>
</mat-list-item>
<mat-divider *ngIf="!last"></mat-divider>
</ng-container>
</mat-list>
</mat-card-content>
</mat-card>
}
@@ -0,0 +1,81 @@
// Copyright 2025 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
.history-list {
font-family: "Roboto", sans-serif;
&__item {
display: flex;
align-items: center;
// Override default mat-list-item height to fit content
height: auto !important;
padding: 16px 0;
}
&__no-records {
display: flex;
justify-content: center;
align-items: center;
flex-direction: column;
}
&__no-records-icon {
width: 4rem;
height: 4rem;
font-size: 4rem;
margin-top: 1.5rem;
}
&__icon {
margin-right: 16px;
&--update {
color: #1976d2;
}
&--security {
color: #d32f2f;
}
}
&__description {
&--main {
font-size: 1rem;
font-weight: 500;
color: rgba(0, 0, 0, 0.87);
margin-bottom: 1em;
}
}
&__content {
flex-grow: 1;
display: flex;
flex-direction: column;
gap: 4px;
margin-right: 16px;
}
&__chip {
margin: 0.5rem 0;
}
&__user {
font-size: 0.9rem;
color: rgba(0, 0, 0, 0.6);
}
&__timestamp {
color: rgba(0, 0, 0, 0.6);
white-space: nowrap;
text-align: right;
}
}
@@ -0,0 +1,66 @@
// Copyright 2025 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { ChangeDetectionStrategy, Component, Input } from '@angular/core';
import { HistoryRecord } from './history.service';
@Component({
selector: 'app-history-list',
templateUrl: './historyList.component.html',
styleUrls: ['./historyList.component.scss'],
changeDetection: ChangeDetectionStrategy.OnPush,
standalone: false,
})
export class HistoryListComponent {
@Input() historyRecords: HistoryRecord[] = [];
@Input() isLoading: boolean = false;
getIconForType(type: string): string {
switch (type) {
case 'REGISTRAR_UPDATE':
return 'edit';
case 'REGISTRAR_SECURITY_UPDATE':
return 'security';
default:
return 'history'; // A fallback icon
}
}
getIconClass(type: string): string {
switch (type) {
case 'REGISTRAR_UPDATE':
return 'history-log__icon--update';
case 'REGISTRAR_SECURITY_UPDATE':
return 'history-log__icon--security';
default:
return '';
}
}
parseDescription(description: string): {
main: string;
detail: string | null;
} {
if (!description) {
return { main: 'N/A', detail: null };
}
const parts = description.split('|');
const detail = parts.length > 1 ? parts[1].replace(/_/g, ' ') : parts[0];
return {
main: parts[0],
detail: detail,
};
}
}
@@ -57,6 +57,11 @@
[(ngModel)]="contactService.contactInEdit.emailAddress"
[ngModelOptions]="{ standalone: true }"
[disabled]="emailAddressIsDisabled()"
[matTooltip]="
emailAddressIsDisabled()
? 'Reach out to registry customer support to update email address'
: ''
"
/>
</mat-form-field>
@@ -84,6 +89,7 @@
<h1>Contact Type</h1>
<p class="console-app__contact-required">
<mat-icon color="accent">error</mat-icon>Required to select at least one
(primary contact can't be updated)
</p>
<div class="">
<ng-container
@@ -24,6 +24,7 @@ import {
PasswordResults,
} from './passwordInputForm.component';
import EppPasswordEditComponent from 'src/app/settings/security/eppPasswordEdit.component';
import { MatSnackBar } from '@angular/material/snack-bar';
export interface PasswordResetVerifyResponse {
registrarId: string;
@@ -54,7 +55,8 @@ export class PasswordResetVerifyComponent {
protected backendService: BackendService,
protected registrarService: RegistrarService,
private route: ActivatedRoute,
private router: Router
private router: Router,
private _snackBar: MatSnackBar
) {}
ngOnInit() {
@@ -99,7 +101,10 @@ export class PasswordResetVerifyComponent {
this.isLoading = false;
this.errorMessage = err.error;
},
next: (_) => this.router.navigate(['']),
next: (_) => {
this.router.navigate(['']);
this._snackBar.open('Password reset completed successfully');
},
});
}
}
@@ -16,6 +16,7 @@ import { Directive, ElementRef, Input, effect } from '@angular/core';
import { UserDataService } from '../services/userData.service';
export enum RESTRICTED_ELEMENTS {
ACTIVITY_PER_USER,
REGISTRAR_ELEMENT,
OTE,
USERS,
@@ -28,9 +29,10 @@ export const DISABLED_ELEMENTS_PER_ROLE = {
RESTRICTED_ELEMENTS.REGISTRAR_ELEMENT,
RESTRICTED_ELEMENTS.OTE,
RESTRICTED_ELEMENTS.SUSPEND,
RESTRICTED_ELEMENTS.ACTIVITY_PER_USER,
],
SUPPORT_LEAD: [],
SUPPORT_AGENT: [],
SUPPORT_AGENT: [RESTRICTED_ELEMENTS.ACTIVITY_PER_USER],
};
@Directive({
@@ -40,6 +42,8 @@ export const DISABLED_ELEMENTS_PER_ROLE = {
export class UserLevelVisibility {
@Input() elementId!: RESTRICTED_ELEMENTS | null;
@Input() isReverse: boolean = false;
constructor(
private userDataService: UserDataService,
private el: ElementRef
@@ -56,9 +60,9 @@ export class UserLevelVisibility {
// @ts-ignore
(DISABLED_ELEMENTS_PER_ROLE[globalRole] || []).includes(this.elementId)
) {
this.el.nativeElement.style.display = 'none';
this.el.nativeElement.style.display = this.isReverse ? '' : 'none';
} else {
this.el.nativeElement.style.display = '';
this.el.nativeElement.style.display = this.isReverse ? 'none' : '';
}
}
}
@@ -31,6 +31,7 @@ import { Contact } from '../../settings/contact/contact.service';
import { EppPasswordBackendModel } from '../../settings/security/security.service';
import { UserData } from './userData.service';
import { PasswordResetVerifyResponse } from '../components/passwordReset/passwordResetVerify.component';
import { HistoryRecord } from '../../history/history.service';
@Injectable()
export class BackendService {
@@ -123,6 +124,16 @@ export class BackendService {
.pipe(catchError((err) => this.errorCatcher<DomainListResult>(err)));
}
getHistoryLog(registrarId: string, userEmail?: string) {
return this.http
.get<HistoryRecord[]>(
userEmail
? `/console-api/history?registrarId=${registrarId}&consoleUserEmail=${userEmail}`
: `/console-api/history?registrarId=${registrarId}`
)
.pipe(catchError((err) => this.errorCatcher<HistoryRecord[]>(err)));
}
getRegistrars(): Observable<Registrar[]> {
return this.http
.get<Registrar[]>('/console-api/registrars')
@@ -1 +0,0 @@
<div class="console-tlds__cards"></div>
@@ -1,38 +0,0 @@
// Copyright 2024 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { ComponentFixture, TestBed } from '@angular/core/testing';
import { TldsComponent } from './tlds.component';
import { MaterialModule } from '../material.module';
describe('TldsComponent', () => {
let component: TldsComponent;
let fixture: ComponentFixture<TldsComponent>;
beforeEach(async () => {
await TestBed.configureTestingModule({
imports: [MaterialModule],
declarations: [TldsComponent],
}).compileComponents();
fixture = TestBed.createComponent(TldsComponent);
component = fixture.componentInstance;
fixture.detectChanges();
});
it('should create', () => {
expect(component).toBeTruthy();
});
});
@@ -29,9 +29,11 @@ import static google.registry.request.RequestParameters.extractRequiredParameter
import static google.registry.request.RequestParameters.extractSetOfDatetimeParameters;
import com.google.common.collect.ImmutableSet;
import com.google.common.util.concurrent.RateLimiter;
import dagger.Module;
import dagger.Provides;
import google.registry.request.Parameter;
import jakarta.inject.Named;
import jakarta.servlet.http.HttpServletRequest;
import java.util.Optional;
import org.joda.time.DateTime;
@@ -137,4 +139,18 @@ public class BatchModule {
static boolean provideIsFast(HttpServletRequest req) {
return extractBooleanParameter(req, PARAM_FAST);
}
private static final int DEFAULT_MAX_QPS = 10;
@Provides
@Parameter("maxQps")
static int provideMaxQps(HttpServletRequest req) {
return extractOptionalIntParameter(req, "maxQps").orElse(DEFAULT_MAX_QPS);
}
@Provides
@Named("removeAllDomainContacts")
static RateLimiter provideRemoveAllDomainContactsRateLimiter(@Parameter("maxQps") int maxQps) {
return RateLimiter.create(maxQps);
}
}
@@ -0,0 +1,246 @@
// Copyright 2025 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.batch;
import static com.google.common.base.Preconditions.checkState;
import static com.google.common.net.MediaType.PLAIN_TEXT_UTF_8;
import static google.registry.flows.FlowUtils.marshalWithLenientRetry;
import static google.registry.model.common.FeatureFlag.FeatureName.MINIMUM_DATASET_CONTACTS_PROHIBITED;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import static google.registry.util.ResourceUtils.readResourceUtf8;
import static jakarta.servlet.http.HttpServletResponse.SC_INTERNAL_SERVER_ERROR;
import static jakarta.servlet.http.HttpServletResponse.SC_NO_CONTENT;
import static jakarta.servlet.http.HttpServletResponse.SC_OK;
import static java.nio.charset.StandardCharsets.US_ASCII;
import com.google.common.base.Ascii;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.google.common.flogger.FluentLogger;
import com.google.common.util.concurrent.RateLimiter;
import google.registry.config.RegistryConfig.Config;
import google.registry.flows.EppController;
import google.registry.flows.EppRequestSource;
import google.registry.flows.PasswordOnlyTransportCredentials;
import google.registry.flows.StatelessRequestSessionMetadata;
import google.registry.model.common.FeatureFlag;
import google.registry.model.contact.Contact;
import google.registry.model.domain.DesignatedContact;
import google.registry.model.domain.Domain;
import google.registry.model.eppcommon.ProtocolDefinition;
import google.registry.model.eppoutput.EppOutput;
import google.registry.persistence.VKey;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.request.lock.LockHandler;
import jakarta.inject.Inject;
import jakarta.inject.Named;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.logging.Level;
import javax.annotation.Nullable;
import org.joda.time.Duration;
/**
* An action that removes all contacts from all active (non-deleted) domains.
*
* <p>This implements part 1 of phase 3 of the Minimum Dataset migration, wherein we remove all uses
* of contact objects in preparation for later removing all contact data from the system.
*
* <p>This runs as a singly threaded, resumable action that loads batches of domains still
* containing contacts, and runs a superuser domain update on each one to remove the contacts,
* leaving behind a record recording that update.
*/
@Action(
service = GaeService.BACKEND,
path = RemoveAllDomainContactsAction.PATH,
method = Action.Method.POST,
auth = Auth.AUTH_ADMIN)
public class RemoveAllDomainContactsAction implements Runnable {
public static final String PATH = "/_dr/task/removeAllDomainContacts";
private static final String LOCK_NAME = "Remove all domain contacts";
private static final String CONTACT_FMT = "<domain:contact type=\"%s\">%s</domain:contact>";
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private final EppController eppController;
private final String registryAdminClientId;
private final LockHandler lockHandler;
private final RateLimiter rateLimiter;
private final Response response;
private final String updateDomainXml;
private int successes = 0;
private int failures = 0;
private static final int BATCH_SIZE = 10000;
@Inject
RemoveAllDomainContactsAction(
EppController eppController,
@Config("registryAdminClientId") String registryAdminClientId,
LockHandler lockHandler,
@Named("removeAllDomainContacts") RateLimiter rateLimiter,
Response response) {
this.eppController = eppController;
this.registryAdminClientId = registryAdminClientId;
this.lockHandler = lockHandler;
this.rateLimiter = rateLimiter;
this.response = response;
this.updateDomainXml =
readResourceUtf8(RemoveAllDomainContactsAction.class, "domain_remove_contacts.xml");
}
@Override
public void run() {
checkState(
tm().transact(() -> FeatureFlag.isActiveNow(MINIMUM_DATASET_CONTACTS_PROHIBITED)),
"Minimum dataset migration must be completed prior to running this action");
response.setContentType(PLAIN_TEXT_UTF_8);
Callable<Void> runner =
() -> {
try {
runLocked();
response.setStatus(SC_OK);
} catch (Exception e) {
logger.atSevere().withCause(e).log("Errored out during execution.");
response.setStatus(SC_INTERNAL_SERVER_ERROR);
response.setPayload(String.format("Errored out with cause: %s", e));
}
return null;
};
if (!lockHandler.executeWithLocks(runner, null, Duration.standardHours(1), LOCK_NAME)) {
// Send a 200-series status code to prevent this conflicting action from retrying.
response.setStatus(SC_NO_CONTENT);
response.setPayload("Could not acquire lock; already running?");
}
}
private void runLocked() {
logger.atInfo().log("Removing contacts on all active domains.");
List<String> domainRepoIdsBatch;
do {
domainRepoIdsBatch =
tm().<List<String>>transact(
() ->
tm().getEntityManager()
.createQuery(
"""
SELECT repoId FROM Domain WHERE deletionTime = :end_of_time AND NOT (
adminContact IS NULL AND billingContact IS NULL
AND registrantContact IS NULL AND techContact IS NULL)
""")
.setParameter("end_of_time", END_OF_TIME)
.setMaxResults(BATCH_SIZE)
.getResultList());
for (String domainRepoId : domainRepoIdsBatch) {
rateLimiter.acquire();
runDomainUpdateFlow(domainRepoId);
}
} while (!domainRepoIdsBatch.isEmpty());
String msg =
String.format(
"Finished; %d domains were successfully updated and %d errored out.",
successes, failures);
logger.at(failures == 0 ? Level.INFO : Level.WARNING).log(msg);
response.setPayload(msg);
}
private void runDomainUpdateFlow(String repoId) {
// Create a new transaction that the flow's execution will be enlisted in that loads the domain
// transactionally. This way we can ensure that nothing else has modified the domain in question
// in the intervening period since the query above found it. If a single domain update fails
// permanently, log it and move on to not block processing all the other domains.
try {
boolean success = tm().transact(() -> runDomainUpdateFlowInner(repoId));
if (success) {
successes++;
} else {
failures++;
}
} catch (Throwable t) {
logger.atWarning().withCause(t).log(
"Failed updating domain with repoId %s; skipping.", repoId);
}
}
/**
* Runs the actual domain update flow and returns whether the contact removals were successful.
*/
private boolean runDomainUpdateFlowInner(String repoId) {
Domain domain = tm().loadByKey(VKey.create(Domain.class, repoId));
if (!domain.getDeletionTime().equals(END_OF_TIME)) {
// Domain has been deleted since the action began running; nothing further to be
// done here.
logger.atInfo().log("Nothing to process for deleted domain '%s'.", domain.getDomainName());
return false;
}
logger.atInfo().log("Attempting to remove contacts on domain '%s'.", domain.getDomainName());
StringBuilder sb = new StringBuilder();
ImmutableMap<VKey<? extends Contact>, Contact> contacts =
tm().loadByKeys(
domain.getContacts().stream()
.map(DesignatedContact::getContactKey)
.collect(ImmutableSet.toImmutableSet()));
// Collect all the (non-registrant) contacts referenced by the domain and compile an EPP XML
// string that removes each one.
for (DesignatedContact designatedContact : domain.getContacts()) {
@Nullable Contact contact = contacts.get(designatedContact.getContactKey());
if (contact == null) {
logger.atWarning().log(
"Domain '%s' referenced contact with repo ID '%s' that couldn't be" + " loaded.",
domain.getDomainName(), designatedContact.getContactKey().getKey());
continue;
}
sb.append(
String.format(
CONTACT_FMT,
Ascii.toLowerCase(designatedContact.getType().name()),
contact.getContactId()))
.append("\n");
}
String compiledXml =
updateDomainXml
.replace("%DOMAIN%", domain.getDomainName())
.replace("%CONTACTS%", sb.toString());
EppOutput output =
eppController.handleEppCommand(
new StatelessRequestSessionMetadata(
registryAdminClientId, ProtocolDefinition.getVisibleServiceExtensionUris()),
new PasswordOnlyTransportCredentials(),
EppRequestSource.BACKEND,
false,
true,
compiledXml.getBytes(US_ASCII));
if (output.isSuccess()) {
logger.atInfo().log(
"Successfully removed contacts from domain '%s'.", domain.getDomainName());
} else {
logger.atWarning().log(
"Failed removing contacts from domain '%s' with error %s.",
domain.getDomainName(), new String(marshalWithLenientRetry(output), US_ASCII));
}
return output.isSuccess();
}
}
@@ -1591,26 +1591,6 @@ public final class RegistryConfig {
return CONFIG_SETTINGS.get().caching.eppResourceMaxCachedEntries;
}
/** Returns if we have enabled caching for User Authentication */
public static boolean getUserAuthCachingEnabled() {
return CONFIG_SETTINGS.get().caching.userAuthCachingEnabled;
}
@VisibleForTesting
public static void overrideIsUserAuthCachingEnabledForTesting(boolean enabled) {
CONFIG_SETTINGS.get().caching.userAuthCachingEnabled = enabled;
}
/** Returns the expiry duration for the user authentication cache. */
public static java.time.Duration getUserAuthCachingDuration() {
return java.time.Duration.ofSeconds(CONFIG_SETTINGS.get().caching.userAuthCachingSeconds);
}
/** Returns the maximum number of entries in user authentication cache. */
public static int getUserAuthMaxCachedEntries() {
return CONFIG_SETTINGS.get().caching.userAuthMaxCachedEntries;
}
/** Returns the amount of time that a particular claims list should be cached. */
public static java.time.Duration getClaimsListCacheDuration() {
return java.time.Duration.ofSeconds(CONFIG_SETTINGS.get().caching.claimsListCachingSeconds);
@@ -161,9 +161,6 @@ public class RegistryConfigSettings {
public int eppResourceCachingSeconds;
public int eppResourceMaxCachedEntries;
public int claimsListCachingSeconds;
public boolean userAuthCachingEnabled;
public int userAuthCachingSeconds;
public int userAuthMaxCachedEntries;
}
/** Configuration for ICANN monthly reporting. */
@@ -0,0 +1,13 @@
# Nomulus Environment Configuration
The configuration files for the different Nomulus environments are not included in this repository. To configure and run a specific environment, you will need to create the corresponding YAML configuration file in this directory.
The following is a list of the environment configuration files that you may need to create:
* `nomulus-config-alpha.yaml`
* `nomulus-config-crash.yaml`
* `nomulus-config-qa.yaml`
* `nomulus-config-sandbox.yaml`
* `nomulus-config-production.yaml`
Please create the relevant file for the environment you intend to use and populate it with the necessary configuration details.
@@ -326,20 +326,6 @@ caching:
# long duration is acceptable because claims lists don't change frequently.
claimsListCachingSeconds: 21600 # six hours
#-- User Authentication Cache Settings --#
# Whether to cache User objects during OIDC token authentication to reduce database load.
# This helps mitigate high QPS from frequent hello commands and session-less requests.
userAuthCachingEnabled: true
# The duration in seconds for which a User object is cached after being loaded.
# A short duration is recommended to avoid stale data.
userAuthCachingSeconds: 60
# The maximum number of User objects to store in the cache per pod.
# This helps limit the memory footprint of the cache.
userAuthMaxCachedEntries: 200
# Note: Only allowedServiceAccountEmails and oauthClientId should be configured.
# Other fields are related to OAuth-based authentication and will be removed.
auth:
@@ -1 +0,0 @@
# Add environment-specific configuration here.
@@ -1 +0,0 @@
# Add environment-specific configuration here.
@@ -1 +0,0 @@
# Add environment-specific configuration here.
@@ -1 +0,0 @@
# Add environment-specific configuration here.
@@ -1 +0,0 @@
# Add environment-specific configuration here.
@@ -56,9 +56,10 @@ public final class FlowUtils {
}
}
/** Persists the saves and deletes in an {@link EntityChanges} to the DB. */
/** Persists the inserts, updates, and deletes in an {@link EntityChanges} to the DB. */
public static void persistEntityChanges(EntityChanges entityChanges) {
tm().putAll(entityChanges.getSaves());
tm().insertAll(entityChanges.getInserts());
tm().updateAll(entityChanges.getUpdates());
tm().delete(entityChanges.getDeletes());
}
@@ -19,25 +19,30 @@ import com.google.common.collect.ImmutableSet;
import google.registry.model.ImmutableObject;
import google.registry.persistence.VKey;
/** A record that encapsulates database entities to both save and delete. */
/** A record that encapsulates database entities to insert, update, and delete. */
public record EntityChanges(
ImmutableSet<ImmutableObject> saves, ImmutableSet<VKey<ImmutableObject>> deletes) {
ImmutableSet<ImmutableObject> inserts,
ImmutableSet<ImmutableObject> updates,
ImmutableSet<VKey<ImmutableObject>> deletes) {
public ImmutableSet<ImmutableObject> getSaves() {
return saves;
public ImmutableSet<ImmutableObject> getInserts() {
return inserts;
}
public ImmutableSet<ImmutableObject> getUpdates() {
return updates;
}
;
public ImmutableSet<VKey<ImmutableObject>> getDeletes() {
return deletes;
}
;
public static Builder newBuilder() {
// Default both entities to save and entities to delete to empty sets, so that the build()
// method won't subsequently throw an exception if one doesn't end up being applicable.
// Default inserts, updates, and deletes to empty sets, so that the build() method won't
// subsequently throw an exception if one doesn't end up being applicable.
return new AutoBuilder_EntityChanges_Builder()
.setSaves(ImmutableSet.of())
.setInserts(ImmutableSet.of())
.setUpdates(ImmutableSet.of())
.setDeletes(ImmutableSet.of());
}
@@ -45,12 +50,21 @@ public record EntityChanges(
@AutoBuilder
public interface Builder {
Builder setSaves(ImmutableSet<ImmutableObject> entitiesToSave);
Builder setInserts(ImmutableSet<ImmutableObject> entitiesToInsert);
ImmutableSet.Builder<ImmutableObject> savesBuilder();
ImmutableSet.Builder<ImmutableObject> insertsBuilder();
default Builder addSave(ImmutableObject entityToSave) {
savesBuilder().add(entityToSave);
default Builder addInsert(ImmutableObject entityToInsert) {
insertsBuilder().add(entityToInsert);
return this;
}
Builder setUpdates(ImmutableSet<ImmutableObject> entitiesToUpdate);
ImmutableSet.Builder<ImmutableObject> updatesBuilder();
default Builder addUpdate(ImmutableObject entityToUpdate) {
updatesBuilder().add(entityToUpdate);
return this;
}
@@ -18,7 +18,6 @@ import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.dns.DnsUtils.requestDomainDnsRefresh;
import static google.registry.flows.FlowUtils.persistEntityChanges;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.verifyResourceDoesNotExist;
import static google.registry.flows.domain.DomainFlowUtils.COLLISION_MESSAGE;
import static google.registry.flows.domain.DomainFlowUtils.checkAllowedAccessToTld;
import static google.registry.flows.domain.DomainFlowUtils.checkHasBillingAccount;
@@ -224,6 +223,7 @@ public final class DomainCreateFlow implements MutatingFlow {
@Inject DomainCreateFlowCustomLogic flowCustomLogic;
@Inject DomainFlowTmchUtils tmchUtils;
@Inject DomainPricingLogic pricingLogic;
@Inject DomainDeletionTimeCache domainDeletionTimeCache;
@Inject DomainCreateFlow() {}
@@ -239,13 +239,13 @@ public final class DomainCreateFlow implements MutatingFlow {
validateRegistrarIsLoggedIn(registrarId);
verifyRegistrarIsActive(registrarId);
extensionManager.validate();
verifyDomainDoesNotExist();
DateTime now = tm().getTransactionTime();
DomainCommand.Create command = cloneAndLinkReferences((Create) resourceCommand, now);
Period period = command.getPeriod();
verifyUnitIsYears(period);
int years = period.getValue();
validateRegistrationPeriod(years);
verifyResourceDoesNotExist(Domain.class, targetId, now, registrarId);
// Validate that this is actually a legal domain name on a TLD that the registrar has access to.
InternetDomainName domainName = validateDomainName(command.getDomainName());
String domainLabel = domainName.parts().getFirst();
@@ -357,11 +357,11 @@ public final class DomainCreateFlow implements MutatingFlow {
domainHistoryId, registrationExpirationTime, isAnchorTenant, allocationToken);
PollMessage.Autorenew autorenewPollMessage =
createAutorenewPollMessage(domainHistoryId, registrationExpirationTime);
ImmutableSet.Builder<ImmutableObject> entitiesToSave = new ImmutableSet.Builder<>();
entitiesToSave.add(createBillingEvent, autorenewBillingEvent, autorenewPollMessage);
ImmutableSet.Builder<ImmutableObject> entitiesToInsert = new ImmutableSet.Builder<>();
entitiesToInsert.add(createBillingEvent, autorenewBillingEvent, autorenewPollMessage);
// Bill for EAP cost, if any.
if (!feesAndCredits.getEapCost().isZero()) {
entitiesToSave.add(createEapBillingEvent(feesAndCredits, createBillingEvent));
entitiesToInsert.add(createEapBillingEvent(feesAndCredits, createBillingEvent));
}
ImmutableSet<ReservationType> reservationTypes = getReservationTypes(domainName);
@@ -404,12 +404,13 @@ public final class DomainCreateFlow implements MutatingFlow {
DomainHistory domainHistory =
buildDomainHistory(domain, tld, now, period, tld.getAddGracePeriodLength());
if (reservationTypes.contains(NAME_COLLISION)) {
entitiesToSave.add(
entitiesToInsert.add(
createNameCollisionOneTimePollMessage(targetId, domainHistory, registrarId, now));
}
entitiesToSave.add(domain, domainHistory);
entitiesToInsert.add(domain, domainHistory);
ImmutableSet.Builder<ImmutableObject> entitiesToUpdate = new ImmutableSet.Builder<>();
if (allocationToken.isPresent() && allocationToken.get().getTokenType().isOneTimeUse()) {
entitiesToSave.add(
entitiesToUpdate.add(
AllocationTokenFlowUtils.redeemToken(
allocationToken.get(), domainHistory.getHistoryEntryId()));
}
@@ -422,7 +423,10 @@ public final class DomainCreateFlow implements MutatingFlow {
.setNewDomain(domain)
.setHistoryEntry(domainHistory)
.setEntityChanges(
EntityChanges.newBuilder().setSaves(entitiesToSave.build()).build())
EntityChanges.newBuilder()
.setInserts(entitiesToInsert.build())
.setUpdates(entitiesToUpdate.build())
.build())
.setYears(years)
.build());
persistEntityChanges(entityChanges);
@@ -649,6 +653,15 @@ public final class DomainCreateFlow implements MutatingFlow {
.build();
}
private void verifyDomainDoesNotExist() throws ResourceCreateContentionException {
Optional<DateTime> previousDeletionTime =
domainDeletionTimeCache.getDeletionTimeForDomain(targetId);
if (previousDeletionTime.isPresent()
&& !tm().getTransactionTime().isAfter(previousDeletionTime.get())) {
throw new ResourceCreateContentionException(targetId);
}
}
private static BillingEvent createEapBillingEvent(
FeesAndCredits feesAndCredits, BillingEvent createBillingEvent) {
return new BillingEvent.Builder()
@@ -151,7 +151,7 @@ public final class DomainDeleteFlow implements MutatingFlow, SqlStatementLogging
verifyDeleteAllowed(existingDomain, tld, now);
flowCustomLogic.afterValidation(
AfterValidationParameters.newBuilder().setExistingDomain(existingDomain).build());
ImmutableSet.Builder<ImmutableObject> entitiesToSave = new ImmutableSet.Builder<>();
ImmutableSet.Builder<ImmutableObject> entitiesToInsert = new ImmutableSet.Builder<>();
Domain.Builder builder;
if (existingDomain.getStatusValues().contains(StatusValue.PENDING_TRANSFER)) {
builder =
@@ -221,7 +221,7 @@ public final class DomainDeleteFlow implements MutatingFlow, SqlStatementLogging
} else {
PollMessage.OneTime deletePollMessage =
createDeletePollMessage(existingDomain, domainHistoryId, deletionTime);
entitiesToSave.add(deletePollMessage);
entitiesToInsert.add(deletePollMessage);
builder.setDeletePollMessage(deletePollMessage.createVKey());
}
}
@@ -230,7 +230,7 @@ public final class DomainDeleteFlow implements MutatingFlow, SqlStatementLogging
// registrar other than the sponsoring registrar (which will necessarily be a superuser).
if (durationUntilDelete.isLongerThan(Duration.ZERO)
&& !registrarId.equals(existingDomain.getPersistedCurrentSponsorRegistrarId())) {
entitiesToSave.add(
entitiesToInsert.add(
createImmediateDeletePollMessage(existingDomain, domainHistoryId, now, deletionTime));
}
@@ -239,7 +239,7 @@ public final class DomainDeleteFlow implements MutatingFlow, SqlStatementLogging
for (GracePeriod gracePeriod : existingDomain.getGracePeriods()) {
// No cancellation is written if the grace period was not for a billable event.
if (gracePeriod.hasBillingEvent()) {
entitiesToSave.add(
entitiesToInsert.add(
BillingCancellation.forGracePeriod(gracePeriod, now, domainHistoryId, targetId));
if (gracePeriod.getBillingEvent() != null) {
// Take the amount of registration time being refunded off the expiration time.
@@ -271,7 +271,7 @@ public final class DomainDeleteFlow implements MutatingFlow, SqlStatementLogging
// ResourceDeleteFlow since it's listed in serverApproveEntities.
requestDomainDnsRefresh(existingDomain.getDomainName());
entitiesToSave.add(newDomain, domainHistory);
entitiesToInsert.add(domainHistory);
EntityChanges entityChanges =
flowCustomLogic.beforeSave(
BeforeSaveParameters.newBuilder()
@@ -279,7 +279,10 @@ public final class DomainDeleteFlow implements MutatingFlow, SqlStatementLogging
.setNewDomain(newDomain)
.setHistoryEntry(domainHistory)
.setEntityChanges(
EntityChanges.newBuilder().setSaves(entitiesToSave.build()).build())
EntityChanges.newBuilder()
.setInserts(entitiesToInsert.build())
.addUpdate(newDomain)
.build())
.build());
BeforeResponseReturnData responseData =
flowCustomLogic.beforeResponse(
@@ -0,0 +1,133 @@
// Copyright 2025 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.flows.domain;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.github.benmanes.caffeine.cache.CacheLoader;
import com.github.benmanes.caffeine.cache.Caffeine;
import com.github.benmanes.caffeine.cache.Expiry;
import com.github.benmanes.caffeine.cache.LoadingCache;
import com.github.benmanes.caffeine.cache.Ticker;
import com.google.common.collect.ImmutableSet;
import google.registry.model.ForeignKeyUtils;
import google.registry.model.domain.Domain;
import google.registry.util.DateTimeUtils;
import java.util.Optional;
import org.joda.time.DateTime;
/**
* Functionally-static loading cache that keeps track of deletion (AKA drop) times for domains.
*
* <p>Some domain names may have many create requests issued shortly before (and directly after) the
* name is released due to a previous registrant deleting it. In those cases, caching the deletion
* time of the existing domain allows us to short-circuit the request and avoid any load on the
* database checking the existing domain (at least, in cases where the request hits a particular
* node more than once).
*
* <p>The cache is fairly short-lived (as we're concerned about many requests at basically the same
* time), and entries also expire when the drop actually happens. If the domain is re-created after
* a drop, the next load attempt will populate the cache with a deletion time of END_OF_TIME, which
* will be read from the cache by subsequent attempts.
*
* <p>We take advantage of the fact that Caffeine caches don't store nulls returned from the
* CacheLoader, so a null result (meaning the domain doesn't exist) won't affect future calls (this
* avoids a stale-cache situation where the cache "thinks" the domain doesn't exist, but it does).
* Put another way, if a domain really doesn't exist, we'll re-attempt the database load every time.
*
* <p>We don't explicitly set the cache inside domain create/delete flows, in case the transaction
* fails at commit time. It's better to have stale data, or to require an additional database load,
* than to have incorrect data.
*
* <p>Note: this should be injected as a singleton -- it's essentially static, but we have it as a
* non-static object for concurrent testing purposes.
*/
public class DomainDeletionTimeCache {
// Max expiry time is ten minutes
private static final long NANOS_IN_ONE_MILLISECOND = 100000L;
private static final long MAX_EXPIRY_NANOS = 10L * 60L * 1000L * NANOS_IN_ONE_MILLISECOND;
private static final int MAX_ENTRIES = 500;
/**
* Expire after the max duration, or after the domain is set to drop (whichever comes first).
*
* <p>If the domain has already been deleted (the deletion time is <= now), the entry will
* immediately be expired/removed.
*
* <p>NB: the Expiry class requires the return value in <b>nanoseconds</b>, not milliseconds
*/
private static final Expiry<String, DateTime> EXPIRY_POLICY =
new Expiry<>() {
@Override
public long expireAfterCreate(String key, DateTime value, long currentTime) {
// Watch out for Long overflow
long deletionTimeNanos =
value.equals(DateTimeUtils.END_OF_TIME)
? Long.MAX_VALUE
: value.getMillis() * NANOS_IN_ONE_MILLISECOND;
long nanosUntilDeletion = deletionTimeNanos - currentTime;
return Math.max(0L, Math.min(MAX_EXPIRY_NANOS, nanosUntilDeletion));
}
/** Reset the time entirely on update, as if we were creating the entry anew. */
@Override
public long expireAfterUpdate(
String key, DateTime value, long currentTime, long currentDuration) {
return expireAfterCreate(key, value, currentTime);
}
/** Reads do not change the expiry duration. */
@Override
public long expireAfterRead(
String key, DateTime value, long currentTime, long currentDuration) {
return currentDuration;
}
};
/** Attempt to load the domain's deletion time if the domain exists. */
private static final CacheLoader<String, DateTime> CACHE_LOADER =
(domainName) -> {
ForeignKeyUtils.MostRecentResource mostRecentResource =
ForeignKeyUtils.loadMostRecentResources(
Domain.class, ImmutableSet.of(domainName), false)
.get(domainName);
return mostRecentResource == null ? null : mostRecentResource.deletionTime();
};
// Unfortunately, maintenance tasks aren't necessarily already in a transaction
private static final Ticker TRANSACTION_TIME_TICKER =
() -> tm().reTransact(() -> tm().getTransactionTime().getMillis() * NANOS_IN_ONE_MILLISECOND);
public static DomainDeletionTimeCache create() {
return new DomainDeletionTimeCache(
Caffeine.newBuilder()
.ticker(TRANSACTION_TIME_TICKER)
.expireAfter(EXPIRY_POLICY)
.maximumSize(MAX_ENTRIES)
.build(CACHE_LOADER));
}
private final LoadingCache<String, DateTime> cache;
private DomainDeletionTimeCache(LoadingCache<String, DateTime> cache) {
this.cache = cache;
}
/** Returns the domain's deletion time, or null if it doesn't currently exist. */
public Optional<DateTime> getDeletionTimeForDomain(String domainName) {
return Optional.ofNullable(cache.get(domainName));
}
}
@@ -1,4 +1,4 @@
// Copyright 2024 The Nomulus Authors. All Rights Reserved.
// Copyright 2025 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
@@ -12,12 +12,19 @@
// See the License for the specific language governing permissions and
// limitations under the License.
import { Component } from '@angular/core';
package google.registry.flows.domain;
@Component({
selector: 'app-tlds',
templateUrl: './tlds.component.html',
styleUrls: ['./tlds.component.scss'],
standalone: false,
})
export class TldsComponent {}
import dagger.Module;
import dagger.Provides;
import jakarta.inject.Singleton;
/** Dagger module to provide the {@link DomainDeletionTimeCache}. */
@Module
public class DomainDeletionTimeCacheModule {
@Provides
@Singleton
public static DomainDeletionTimeCache provideDomainDeletionTimeCache() {
return DomainDeletionTimeCache.create();
}
}
@@ -711,7 +711,7 @@ public class DomainFlowUtils {
BillingRecurrence newBillingRecurrence =
existingBillingRecurrence.asBuilder().setRecurrenceEndTime(newEndTime).build();
tm().put(newBillingRecurrence);
tm().update(newBillingRecurrence);
return newBillingRecurrence;
}
@@ -245,11 +245,13 @@ public final class DomainRenewFlow implements MutatingFlow {
.build();
DomainHistory domainHistory =
buildDomainHistory(newDomain, now, command.getPeriod(), tld.getRenewGracePeriodLength());
ImmutableSet.Builder<ImmutableObject> entitiesToSave = new ImmutableSet.Builder<>();
entitiesToSave.add(
newDomain, domainHistory, explicitRenewEvent, newAutorenewEvent, newAutorenewPollMessage);
ImmutableSet.Builder<ImmutableObject> entitiesToInsert = new ImmutableSet.Builder<>();
ImmutableSet.Builder<ImmutableObject> entitiesToUpdate = new ImmutableSet.Builder<>();
entitiesToInsert.add(
domainHistory, explicitRenewEvent, newAutorenewEvent, newAutorenewPollMessage);
entitiesToUpdate.add(newDomain);
if (allocationToken.isPresent() && allocationToken.get().getTokenType().isOneTimeUse()) {
entitiesToSave.add(
entitiesToUpdate.add(
AllocationTokenFlowUtils.redeemToken(
allocationToken.get(), domainHistory.getHistoryEntryId()));
}
@@ -262,7 +264,10 @@ public final class DomainRenewFlow implements MutatingFlow {
.setYears(years)
.setHistoryEntry(domainHistory)
.setEntityChanges(
EntityChanges.newBuilder().setSaves(entitiesToSave.build()).build())
EntityChanges.newBuilder()
.setInserts(entitiesToInsert.build())
.setUpdates(entitiesToUpdate.build())
.build())
.build());
BeforeResponseReturnData responseData =
flowCustomLogic.beforeResponse(
@@ -146,18 +146,18 @@ public final class DomainRestoreRequestFlow implements MutatingFlow {
verifyRestoreAllowed(command, existingDomain, feeUpdate, feesAndCredits, now);
HistoryEntryId domainHistoryId = createHistoryEntryId(existingDomain);
historyBuilder.setRevisionId(domainHistoryId.getRevisionId());
ImmutableSet.Builder<ImmutableObject> entitiesToSave = new ImmutableSet.Builder<>();
ImmutableSet.Builder<ImmutableObject> entitiesToInsert = new ImmutableSet.Builder<>();
DateTime newExpirationTime =
existingDomain.getRegistrationExpirationTime().plusYears(isExpired ? 1 : 0);
// Restore the expiration time on the deleted domain, except if that's already passed, then add
// a year and bill for it immediately, with no grace period.
if (isExpired) {
entitiesToSave.add(
entitiesToInsert.add(
createRenewBillingEvent(domainHistoryId, feesAndCredits.getRenewCost(), now));
}
// Always bill for the restore itself.
entitiesToSave.add(
entitiesToInsert.add(
createRestoreBillingEvent(domainHistoryId, feesAndCredits.getRestoreCost(), now));
BillingRecurrence autorenewEvent =
@@ -166,12 +166,14 @@ public final class DomainRestoreRequestFlow implements MutatingFlow {
.setRecurrenceEndTime(END_OF_TIME)
.setDomainHistoryId(domainHistoryId)
.build();
entitiesToInsert.add(autorenewEvent);
PollMessage.Autorenew autorenewPollMessage =
newAutorenewPollMessage(existingDomain)
.setEventTime(newExpirationTime)
.setAutorenewEndTime(END_OF_TIME)
.setDomainHistoryId(domainHistoryId)
.build();
entitiesToInsert.add(autorenewPollMessage);
Domain newDomain =
performRestore(
existingDomain,
@@ -181,8 +183,9 @@ public final class DomainRestoreRequestFlow implements MutatingFlow {
now,
registrarId);
DomainHistory domainHistory = buildDomainHistory(newDomain, now);
entitiesToSave.add(newDomain, domainHistory, autorenewEvent, autorenewPollMessage);
tm().putAll(entitiesToSave.build());
entitiesToInsert.add(domainHistory);
tm().update(newDomain);
tm().insertAll(entitiesToInsert.build());
if (existingDomain.getDeletePollMessage() != null) {
tm().delete(existingDomain.getDeletePollMessage());
}
@@ -172,7 +172,7 @@ public final class DomainTransferApproveFlow implements MutatingFlow {
.setDomainHistoryId(domainHistoryId)
.build());
ImmutableList.Builder<ImmutableObject> entitiesToSave = new ImmutableList.Builder<>();
ImmutableList.Builder<ImmutableObject> entitiesToInsert = new ImmutableList.Builder<>();
// If we are within an autorenew grace period, cancel the autorenew billing event and don't
// increase the registration time, since the transfer subsumes the autorenew's extra year.
GracePeriod autorenewGrace =
@@ -184,7 +184,7 @@ public final class DomainTransferApproveFlow implements MutatingFlow {
// then the gaining registrar is not charged for the one-year renewal and the losing registrar
// still needs to be charged for the auto-renew.
if (billingEvent.isPresent()) {
entitiesToSave.add(
entitiesToInsert.add(
BillingCancellation.forGracePeriod(autorenewGrace, now, domainHistoryId, targetId));
}
}
@@ -259,14 +259,11 @@ public final class DomainTransferApproveFlow implements MutatingFlow {
PollMessage gainingClientPollMessage =
createGainingTransferPollMessage(
targetId, newDomain.getTransferData(), newExpirationTime, now, domainHistoryId);
billingEvent.ifPresent(entitiesToSave::add);
entitiesToSave.add(
autorenewEvent,
gainingClientPollMessage,
gainingClientAutorenewPollMessage,
newDomain,
domainHistory);
tm().putAll(entitiesToSave.build());
billingEvent.ifPresent(entitiesToInsert::add);
entitiesToInsert.add(
autorenewEvent, gainingClientPollMessage, gainingClientAutorenewPollMessage, domainHistory);
tm().update(newDomain);
tm().insertAll(entitiesToInsert.build());
// Delete the billing event and poll messages that were written in case the transfer would have
// been implicitly server approved.
tm().delete(existingDomain.getTransferData().getServerApproveEntities());
@@ -110,8 +110,8 @@ public final class DomainTransferCancelFlow implements MutatingFlow {
Domain newDomain =
denyPendingTransfer(existingDomain, TransferStatus.CLIENT_CANCELLED, now, registrarId);
DomainHistory domainHistory = buildDomainHistory(newDomain, tld, now);
tm().putAll(
newDomain,
tm().update(newDomain);
tm().insertAll(
domainHistory,
createLosingTransferPollMessage(
targetId, newDomain.getTransferData(), null, domainHistoryId));
@@ -109,8 +109,8 @@ public final class DomainTransferRejectFlow implements MutatingFlow {
Domain newDomain =
denyPendingTransfer(existingDomain, TransferStatus.CLIENT_REJECTED, now, registrarId);
DomainHistory domainHistory = buildDomainHistory(newDomain, tld, now);
tm().putAll(
newDomain,
tm().update(newDomain);
tm().insertAll(
domainHistory,
createGainingTransferPollMessage(
targetId, newDomain.getTransferData(), null, now, domainHistoryId));
@@ -283,11 +283,9 @@ public final class DomainTransferRequestFlow implements MutatingFlow {
asyncTaskEnqueuer.enqueueAsyncResave(
newDomain.createVKey(), now, ImmutableSortedSet.of(automaticTransferTime));
tm().putAll(
new ImmutableSet.Builder<>()
.add(newDomain, domainHistory, requestPollMessage)
.addAll(serverApproveEntities)
.build());
tm().put(newDomain);
tm().putAll(serverApproveEntities);
tm().insertAll(domainHistory, requestPollMessage);
return responseBuilder
.setResultFromCode(SUCCESS_WITH_ACTION_PENDING)
.setResData(createResponse(period, existingDomain, newDomain, now))
@@ -190,14 +190,16 @@ public final class DomainUpdateFlow implements MutatingFlow {
if (requiresDnsUpdate(existingDomain, newDomain)) {
requestDomainDnsRefresh(targetId);
}
ImmutableSet.Builder<ImmutableObject> entitiesToSave = new ImmutableSet.Builder<>();
entitiesToSave.add(newDomain, domainHistory);
ImmutableSet.Builder<ImmutableObject> entitiesToInsert = new ImmutableSet.Builder<>();
ImmutableSet.Builder<ImmutableObject> entitiesToUpdate = new ImmutableSet.Builder<>();
entitiesToUpdate.add(newDomain);
entitiesToInsert.add(domainHistory);
Optional<BillingEvent> statusUpdateBillingEvent =
createBillingEventForStatusUpdates(existingDomain, newDomain, domainHistory, now);
statusUpdateBillingEvent.ifPresent(entitiesToSave::add);
statusUpdateBillingEvent.ifPresent(entitiesToInsert::add);
Optional<PollMessage.OneTime> serverStatusUpdatePollMessage =
createPollMessageForServerStatusUpdates(existingDomain, newDomain, domainHistory, now);
serverStatusUpdatePollMessage.ifPresent(entitiesToSave::add);
serverStatusUpdatePollMessage.ifPresent(entitiesToInsert::add);
EntityChanges entityChanges =
flowCustomLogic.beforeSave(
BeforeSaveParameters.newBuilder()
@@ -205,7 +207,10 @@ public final class DomainUpdateFlow implements MutatingFlow {
.setNewDomain(newDomain)
.setExistingDomain(existingDomain)
.setEntityChanges(
EntityChanges.newBuilder().setSaves(entitiesToSave.build()).build())
EntityChanges.newBuilder()
.setInserts(entitiesToInsert.build())
.setUpdates(entitiesToUpdate.build())
.build())
.build());
persistEntityChanges(entityChanges);
return responseBuilder.build();
@@ -126,7 +126,8 @@ public final class HostCreateFlow implements MutatingFlow {
.setSuperordinateDomain(superordinateDomain.map(Domain::createVKey).orElse(null))
.build();
historyBuilder.setType(HOST_CREATE).setHost(newHost);
ImmutableSet<ImmutableObject> entitiesToSave = ImmutableSet.of(newHost, historyBuilder.build());
ImmutableSet<ImmutableObject> entitiesToInsert =
ImmutableSet.of(newHost, historyBuilder.build());
if (superordinateDomain.isPresent()) {
tm().update(
superordinateDomain
@@ -138,7 +139,7 @@ public final class HostCreateFlow implements MutatingFlow {
// they are only written as NS records from the referencing domain.
requestHostDnsRefresh(targetId);
}
tm().insertAll(entitiesToSave);
tm().insertAll(entitiesToInsert);
return responseBuilder.setResData(HostCreateData.create(targetId, now)).build();
}
@@ -52,7 +52,6 @@ import google.registry.flows.annotations.ReportingSpec;
import google.registry.flows.exceptions.ResourceHasClientUpdateProhibitedException;
import google.registry.model.EppResource;
import google.registry.model.ForeignKeyUtils;
import google.registry.model.ImmutableObject;
import google.registry.model.domain.Domain;
import google.registry.model.domain.metadata.MetadataExtension;
import google.registry.model.eppcommon.StatusValue;
@@ -198,16 +197,12 @@ public final class HostUpdateFlow implements MutatingFlow {
.setPersistedCurrentSponsorRegistrarId(newPersistedRegistrarId)
.build();
verifyHasIpsIffIsExternal(command, existingHost, newHost);
ImmutableSet.Builder<ImmutableObject> entitiesToInsert = new ImmutableSet.Builder<>();
ImmutableSet.Builder<ImmutableObject> entitiesToUpdate = new ImmutableSet.Builder<>();
entitiesToUpdate.add(newHost);
if (isHostRename) {
updateSuperordinateDomains(existingHost, newHost);
}
enqueueTasks(existingHost, newHost);
entitiesToInsert.add(historyBuilder.setType(HOST_UPDATE).setHost(newHost).build());
tm().updateAll(entitiesToUpdate.build());
tm().insertAll(entitiesToInsert.build());
tm().update(newHost);
tm().insert(historyBuilder.setType(HOST_UPDATE).setHost(newHost).build());
return responseBuilder.build();
}
@@ -290,7 +285,7 @@ public final class HostUpdateFlow implements MutatingFlow {
&& newHost.isSubordinate()
&& Objects.equals(
existingHost.getSuperordinateDomain(), newHost.getSuperordinateDomain())) {
tm().put(
tm().update(
tm().loadByKey(existingHost.getSuperordinateDomain())
.asBuilder()
.removeSubordinateHost(existingHost.getHostName())
@@ -299,14 +294,14 @@ public final class HostUpdateFlow implements MutatingFlow {
return;
}
if (existingHost.isSubordinate()) {
tm().put(
tm().update(
tm().loadByKey(existingHost.getSuperordinateDomain())
.asBuilder()
.removeSubordinateHost(existingHost.getHostName())
.build());
}
if (newHost.isSubordinate()) {
tm().put(
tm().update(
tm().loadByKey(newHost.getSuperordinateDomain())
.asBuilder()
.addSubordinateHost(newHost.getHostName())
@@ -0,0 +1,661 @@
<?xml version="1.0" encoding="utf-8"?>
<lgr xmlns="urn:ietf:params:xml:ns:lgr-1.0">
<meta>
<version comment="Latin LGR">1</version>
<date>2025-10-01</date>
<language>und-Latn</language>
<unicode-version>2</unicode-version>
<description type="text/html"><![CDATA[
<div class="instructions">
<h2>INSTRUCTIONS</h2>
<ul>
<li>These instructions cover how to adopt an LGR based on this reference LGR for a given
zone and how to prepare the file for deposit in the IANA Repository of IDN Practices.</li>
<li>As described the IANA procedure (https://www.iana.org/help/idn-repository-procedure) an
LGR MUST contain the following elements in its header:
<ul style="list-style-type:square;">
<li>Script or Language Designator (see below for guidance) </li>
<li>Version Number (this must increase with each amendment to the LGR, even if the updates
are limited to the header itself) </li>
<li>Effective Date (the date at which the policy becomes applicable in operational use) </li>
<li>Registry Contact Details (contact name, email address, and/or phone number)</li>
</ul>
</li>
<li>The following information is optional:
<ul style="list-style-type:square;">
<li>Document creation date</li>
<li>Applicable Domain(s)</li>
<li>Changes made to the Reference LGR before adopting</li>
</ul>
</li>
</ul>
<p>Please add or modify the following items in the <b>XML source code for this file</b> before
depositing the document in the IANA Repository. (https://www.iana.org/domains/idn-tables)</p>
<h3>Meta Data</h3>
<p>Note: version numbers start at 1. RFC 7940 recommends using simple integers. The version comment is optional,
please replace or delete the default comment. Version comments may be used by some tools as part of the page header.</p>
<p><code>&lt;version comment=&quot;</code>[Please replace (or delete) the optional comment]<code>&quot;&gt;</code>[Please fill in version number, starting at 1]<code>&lt;/version&gt;</code></p>
<p><code>&lt;date&gt;</code>2025-10-01<code>&lt;/date&gt;</code></p>
<p><code>&lt;validity-start&gt;</code>2025-10-01<code>&lt;/validity-start&gt;</code></p>
<p>Note: the scope element may be repeated, so that the same document can serve for multiple domains.</p>
<p><code>&lt;scope type=&quot;domain&quot;&gt;</code>[Please provide, in &quot;.domain&quot; format]<code>&lt;/scope&gt;</code></p>
<p><strong>Registry Contact Information:</strong></p>
<p>Please fill in the <a href="#registry_contact_details">Registry Contact Details</a>.</p>
<p><strong>Change History</strong></p>
<p>If you made technical modifications to the LGR, please summarize them in the <a href="#change_history">Change History</a> (and also note the details in the appropriate section of the description).</p>
<section id="registry_contact_details">
<h2>Registry Contact Details</h2>
<ul style="list-style:none;">
<li><b>Contact Name:</b> Ben McIlwain</li>
<li><b>Email address:</b> nomulus-discuss@google.com</li>
</ul>
</section>
<h1>Label Generation Rules for the Latin Script</h1>
<h2>Overview</h2>
<p>This document specifies a set of Label Generation Rules (LGR) for the Latin script for the second level domain or domains identified above.
The starting point for the development of this LGR can be found in the related Root Zone LGR [RZ-LGR-Latn].
The format of this file follows [RFC 7940].
This LGR is adapted from the “Reference LGR for the Second Level for the Latin Script” [Ref-LGR-und-Latn], for details, see <a href="#change_history">Change History</a> below.</p>
<p>For details and additional background on the Latin script, see “Proposal for a Latin Script Root Zone Label Generation Rule-Set (LGR)” [Proposal-Latin].</p>
<h2>Repertoire</h2>
<p>The repertoire contains the 164 letters needed to write hundreds of languages in the Latin script.
The repertoire is a subset of [Unicode 11.0.0]. For details, see Section 5, “Repertoire” in [Proposal-Latin].
(The proposal cited has been adopted for the Latin script portion of the Root Zone LGR.)</p>
<p>For the second level, the repertoire has been augmented with the ASCII digits, U+0030 to U+0039, plus U+002D HYPHEN-MINUS, for a total of 175 repertoire elements.</p>
<p>Any code points outside the Latin Script repertoire that are targets for
out-of-repertoire variants would be included here only if the variant is listed
in this file. In this case they are identified as a reflexive (identity) variant
of type “out-of-repertoire-var”. Whether or not they are listed, they do not
form part of the repertoire.</p>
<p><b>Repertoire Listing:</b> Each code point or range is tagged with the script or scripts with which the code point is used and one or more other character categories. For each repertoire element,
one or more references document sufficient justification for inclusion in the repertoire; see the <a href="#ref_desc_sec_References">“References”</a> below.
For code points that are part of the repertoire, comments identify the languages using the code point along with their [EGIDS] level.</p>
<h3>Background on Script and Principal Languages Using It</h3>
<p>The Latin script is a major writing system of the world, and the most widely used in terms of
the number of languages and speakers, with circa 70% of the worlds readers and writers making use of
this script. From a list of 1,189 languages using the Latin script [Omniglot]
the 212 languages that were taken into consideration contain all 182 languages with [EGIDS]
level 1&ndash;4 together with many languages with EGIDS level 5, each spoken by more than
1 million estimated speakers. Altogether over 100 languages are cited here to justify
specific additions to the repertoire, but many other languages may also be written using
some subset of the repertoire of this LGR. In a few cases, code points were excluded in [MSR-5] due to
security concerns; for the affected languages, only a subrepertoire could be supported.
More details in Section 3, “Background on Script and Principal Languages Using It”
of [Proposal-Latin].</p>
<h2>Variants</h2>
<p>The variants defined in this LGR are limited to those required for use in zones not shared with any other script.
As such, this LGR does not define cross-script variants. However, using this LGR concurrently with any LGR for Armenian, Cyrillic, and Greek in the same zone will introduce some in-script variants due to cross-script variant transitivity. This will also create potential cross-script issues when used with the same LGRs.
For details, see Section 6, “Variants” in [Proposal-Latin].
Mitigation of these in-script and cross-script variants can be addressed by using the Common LGR.
For details, see Section 3, “Use of Multiple Reference LGRs in the Same Zone” in [Level-2-Overview].
In addition to variants defined by this LGR, the full variant information related to this script and added by concurrent use with the Armenian, Cyrillic, and Greek LGR(s) can be found
in the following LGR: [Ref-LGR-Latin-Full-Variant-Script]
</p>
<p>In particular, the Latin LGR contains a number of in-script variants resulting from transitivity with
variants needed if the LGR is used concurrently with other LGRs from the related scripts, primarily
due to variants with the Greek script.
These variant definitions are required when using this LGR together with the Common LGR in label processing,
but they also reflect a certain consistency in the approach to variants across typographically related scripts.
They can be removed if the LGR is used strictly as standalone.<p>
<p>All other in-script variants defined in this LGR largely follow the methodology defined in
Section 6, “Variants”, in [Proposal-Latin]. In a separate appendix that proposal identifies additional
candidates for visually confusable code points (see [Proposal-Latin-Appendices]). They provide data
on the basis of which additional variants, or fallback variants (see following) might be defined. However,
doing so would require additional review and analysis that has not been carried out for this version of the
LGR.</p>
<p>The LGR defines certain allocatable fallback variants as
described in Section 4.5.5 “Allocatable Fallback Variants” in [Level-2-Overview]. A fallback variant is a variant label that uses
substitute code points for code points or sequences not available (or not allowed) in some contexts, that would
otherwise be required for a linguistically accurate rendering of some label.</p>
<p>When “fallback” variants are defined, two labels may be allocated: a single label with the spelling preferred by the
applicant, plus a single fallback variant for that label.
The fallback exclusively uses the fallback characters for any characters for which fallbacks are defined, while the
“preferred” label may use any otherwise valid mix of code points.
If the fallback variant is the one applied for, no other variant label is allocatable.</p>
<p>An allocatable fallback variant exists for the following pairs where the second element of each pair is the fallback:</p>
<ul>
<h3>In-script Variant Mapping Types</h3>
<p>In each of the fallback variant pairs defined above, the mapping type from the first element to the second is of type
“fallback”, while the variant type for the other direction is “blocked”. In addition, the first element of each pair uses the
reflexive mapping “r-original”.
(By convention, the prefix “r-” marks a type used in a reflexive variant mapping, that is, it represents an instance
of the original code point at that location in a variant label, see Section 5.3.4 in [RFC 7940].)</p>
<p><b>Variant Disposition:</b> Except for limited exceptions for the fallback variants defined above, variants defined here result
in a variant label disposition of “blocked”.</p>
<p>The specification of variants in this LGR follows the guidelines in [RFC 8228].</p>
<h2>Character Classes</h2>
<p>This LGR does not define named character classes.</p>
<h2>Whole Label Evaluation (WLE) and Context rules</h2>
<h3>Default Whole Label Evaluation Rules and Actions</h3>
<p>By default, the LGR includes the rules and actions to implement the following restrictions mandated by the IDNA protocol. They are marked with &#x235F;.</p>
<ul>
<li><b>Hyphen Restrictions</b> &mdash; restrictions on the allowable placement of hyphens (no leading/ending hyphen
and no hyphen in positions 3 and 4). These restrictions are described in Section 4.2.3.1 of RFC 5891 [320].
They are implemented here as context rule on U+002D (-) HYPHEN-MINUS.</li>
<li><b>Leading Combining Marks</b> &mdash; restrictions on the allowable placement of combining marks
(no leading combining mark). This rule is described in Section 4.2.3.2 of RFC 5891 [320].</li>
</ul>
<h3>Latin-specific Rules</h3>
<h2>Actions</h2>
<h3>Default Actions</h3>
<p>This LGR includes the default actions for LGRs as well as the action needed to
invalidate labels with misplaced combining marks. They are marked with &#x235F;.
For a description see [RFC 7940].</p>
<p>Default actions that are
triggered by the LGR-specific variant types described above limit the “allocatable” variant
labels to those containing only “ss”, dotted “i” or hyphen variants, while
disallowing mixed use of “ss” and “ß”, Dotless i and “i”, or middle-dot and hyphen respectively, except
as in the original applied-for label.</p>
<p>Note that the mapping types for variants are not symmetric: they depend on which code point is considered
the source or the target in a given mapping. As specified in [RFC 7940], when mapping types are evaluated
code points in a label that are unchanged use the type of their “reflexive” mapping.
Per [RFC 7940] the actions are always applied one after the other, and the evaluation stops at the first
action that assigns a disposition to a given label.</p>
<h3>Script-specific Actions</h3>
<p>An action has been defined to invalidate labels containing the sequence U+00B7 U+006C U+00B7 which could otherwise result in two Ela Geminada sequences overlapping.</p>
<h2>Methodology and Contributors</h2>
<p>The LGR in this document has been adapted from the corresponding Reference LGR for the Second Level. The Second Level Reference LGR for the Latin Script was developed by Michel Suignard and Asmus Freytag, based on the Root Zone LGR for the Latin
script and information contained or referenced therein; see [RZ-LGR-Latn]. Suitable extensions for the second level have been applied according to the [Guidelines] and with community input.
The original proposal for a Root Zone LGR for the Latin script, that this LGR is based on, was developed by the Latin Generation Panel.
For more information on methodology and contributors to the underlying Root Zone LGR, see Sections 4 and 8 in [Proposal-Latin], as well as [RZ-LGR-Overview].</p>
<section id="change_history">
<h3>Changes from Version Dated 25 October 2024</h3>
<p>Adopted from the Second Level Reference LGR for the Latin Script [Ref-LGR-und-Latn] with security improvements implemented by removing confusable variants.</p>
</section>
<h2>References</h2>
<p>The following general references are cited in this document:</p>
<dl class="references">
<dt>[EGIDS]</dt>
<dd>Lewis and Simons, “EGIDS: Expanded Graded Intergenerational Disruption Scale,”
documented in [SIL-Ethnologue] and summarized here:
https://en.wikipedia.org/wiki/Expanded_Graded_Intergenerational_Disruption_Scale_(EGIDS)</dd>
<dt>[Guidelines]</dt>
<dd>ICANN, “Guidelines for Developing Reference LGRs for the Second Level”, (Los Angeles, California: ICANN, 27 May 2020), https://www.icann.org/en/system/files/files/lgr-guidelines-second-level-27may20-en.pdf</dd>
<dt>[Level-2-Overview]</dt>
<dd>Internet Corporation for Assigned Names and Numbers, (ICANN),“Reference Label Generation Rules (LGR) for the Second Level: Overview and Summary” (PDF),
(Los Angeles, California: ICANN, 25 October 2024), https://www.icann.org/en/system/files/files/level2-lgr-overview-summary-25oct24-en.pdf
</dd>
<dt>[MSR-5]</dt>
<dd>Integration Panel, “Maximal Starting Repertoire — MSR-5 Overview and Rationale”, 24 June 2021,
https://www.icann.org/en/system/files/files/msr-5-overview-24jun21-en.pdf</dd>
<dt>[Omniglot]</dt>
<dd>Omniglot, “Writing Systems by Language”, https://www.omniglot.com/writing/langalph.htm (accessed on 13 January 2022)</dd>
<dt>[Proposal-Latin]</dt>
<dd>Latin Generation Panel, “Proposal for a Latin Script Root Zone Label Generation Rule-Set (LGR)”, 27 January 2022 (PDF), https://www.icann.org/en/system/files/files/proposal-latin-lgr-27jan22-en.pdf</dd>
<dt>[Proposal-Latin-Appendices]</dt>
<dd>Appendices to “Proposal for a Latin Script Root Zone Label Generation Rule-Set (LGR)”,
27 January 2022 (ZIP), https://www.icann.org/en/system/files/files/proposal-latin-lgr-appendices-27jan22-en.zip</dd>
<dt>[Ref-LGR-und-Latn]</dt>
<dd>ICANN, Second Level Reference Label Generation Rules for the Latin Script (und-Latn), 25 October 2024 (XML)
https://www.icann.org/sites/default/files/packages/lgr/lgr-second-level-latin-script-25oct24-en.xml
non-normative HTML presentation: https://www.icann.org/sites/default/files/packages/lgr/lgr-second-level-latin-script-25oct24-en.html</dd>
<dt>[Ref-LGR-Latin-Full-Variant-Script]</dt>
<dd>ICANN, Second Level Reference Label Generation Rules for the Latin Script (und-Latn), 25 October 2024 (XML)
https://www.icann.org/sites/default/files/packages/lgr/lgr-second-level-latin-full-variant-script-25oct24-en.xml
non-normative HTML presentation: https://www.icann.org/sites/default/files/packages/lgr/lgr-second-level-latin-full-variant-script-25oct24-en.html</dd>
<dt>[RFC 7940]</dt>
<dd>Davies, K. and A. Freytag, “Representing Label Generation Rulesets Using XML”,
RFC 7940, August 2016, https://www.rfc-editor.org/info/rfc7940</dd>
<dt>[RFC 8228]</dt>
<dd>A. Freytag, “Guidance on Designing Label Generation Rulesets (LGRs) Supporting Variant Labels”, RFC 8228, August 2017,
https://www.rfc-editor.org/info/rfc8228</dd>
<dt>[RZ-LGR-Overview]</dt>
<dd>Integration Panel, “Root Zone Label Generation Rules (RZ LGR-5): Overview and Summary”, 26 May 2022 (PDF), https://www.icann.org/sites/default/files/lgr/rz-lgr-5-overview-26may22-en.pdf</dd>
<dt>[RZ-LGR-5]</dt>
<dd>Integration Panel, “Root Zone Label Generation Rules (RZ-LGR-5)”, 26 May 2022 (XML), https://www.icann.org/sites/default/files/lgr/rz-lgr-5-common-26may22-en.xml <br/>
<i>non-normative HTML presentation: https://www.icann.org/sites/default/files/lgr/rz-lgr-5-common-26may22-en.html</i></dd>
<dt>[RZ-LGR-Latn]</dt>
<dd>ICANN, Root Zone Label Generation Rules for the Latin Script (und-Latn), 26 May 2022 (XML)
https://www.icann.org/sites/default/files/lgr/rz-lgr-5-latin-script-26may22-en.xml</dd>
<dt>[SIL-Ethnologue]</dt>
<dd>David M. Eberhard, Gary F. Simons &amp; Charles D. Fennig (eds.). 2021.
Ethnologue: Languages of the World, Twenty fourth edition. Dallas, Texas: SIL
International. Online version available as https://www.ethnologue.com</dd>
<dt>[Unicode 11.0.0]</dt>
<dd>The Unicode Consortium. The Unicode Standard, Version 11.0.0, (Mountain View, CA: The Unicode Consortium, 2018. ISBN 978-1-936213-19-1)
https://www.unicode.org/versions/Unicode11.0.0/</dd>
</dl>
<p>For references consulted particularly in designing the repertoire for the Latin Script for the second level
please see details in the <a href="#table_of_references">Table of References</a> below.</p>
<p>References [0] and up refer to the Unicode Standard versions in which corresponding code points
were initially encoded. References [101] and up correspond to a source given in [Proposal-Latin] for justifying
the inclusion of the corresponding code points. In the listing of <a href="#whole_label_evaluation_and_context_rules">whole label evaluation and context rules</a>,
reference [320] indicates the source for common rules.</p>
]]></description>
<references>
<reference id="0" comment="Any code point originally encoded in Unicode Version 1.1">The Unicode Standard, Version 1.1</reference>
<reference id="3" comment="Any code point originally encoded in Unicode Version 3.0">The Unicode Standard, Version 3.0</reference>
<reference id="8" comment="Any code point originally encoded in Unicode Version 5.0">The Unicode Standard, Version 5.0</reference>
<reference id="99" comment="Any code point cited is part of the Basic Latin (ASCII) set">C0 Controls and Basic Latin, The Unicode Standard https://unicode.org/charts/PDF/U0000.pdf</reference>
<reference id="100">ICANN, Second Level Reference Label Generation Rules for Spanish https://www.icann.org/sites/default/files/packages/lgr/lgr-second-level-spanish-30aug16-en.html (Accessed on 31 August 2018)</reference>
<reference id="101">Omniglot, Czech (čeština) https://www.omniglot.com/writing/czech.htm (Accessed on 31 August 2018)</reference>
<reference id="102">Omniglot, Icelandic (Íslenska) https://www.omniglot.com/writing/icelandic.htm (Accessed on 31 August 2018)</reference>
<reference id="103">Omniglot, Faroese (føroyskt mál) https://www.omniglot.com/writing/faroese.htm (Accessed on 31 August 2018)</reference>
<reference id="105">Omniglot, Chuukese (Chuuk) https://www.omniglot.com/writing/chuukese.htm (Accessed on 31 August 2018)</reference>
<reference id="106">ScriptSource, Galician written with Latin script https://scriptsource.org/cms/scripts/page.php?item_id=wrSys_detail&amp;key=gl-Latn (Accessed on 1 May 2023)</reference>
<reference id="107">Omniglot, Lule Sámi (julevsámegiella) https://www.omniglot.com/writing/lulesami.htm (Accessed on 31 August 2018)</reference>
<reference id="108">Wikipedia, Northern Sami https://en.wikipedia.org/wiki/Northern_Sami (Accessed on 4 September 2018)</reference>
<reference id="109">Omniglot, Vietnamese (tiếng việt / 㗂越) https://www.omniglot.com/writing/vietnamese.htm (Accessed on 4 September 2018)</reference>
<reference id="110">Omniglot, Romanian (limba română) https://www.omniglot.com/writing/romanian.htm (Accessed on 4 September 2018)</reference>
<reference id="113">Omniglot, Skolt Sámi (Sääˊmǩiõll / Nuõrttsää’m) https://www.omniglot.com/writing/skoltsami.htm (Accessed on 4 September 2018)</reference>
<reference id="114">Omniglot, French (français) https://www.omniglot.com/writing/french.htm (Accessed on 4 September 2018)</reference>
<reference id="115">Omniglot, West Frisian (Frysk) https://www.omniglot.com/writing/westfrisian.htm (Accessed on 4 September 2018)</reference>
<reference id="116">Omniglot, Friulian (furlan/marilenghe) https://www.omniglot.com/writing/friulian.htm (Accessed on 4 September 2018)</reference>
<reference id="117">Summer Institute of Linguistics, Pequeno dicionário: Xavante-Português, Português-Xavante https://www.sil.org/resources/archives/17019 (Accessed on 1 October 2020)</reference>
<reference id="119">Omniglot, German (Deutsch) https://www.omniglot.com/writing/german.htm (Accessed on 4 September 2018)</reference>
<reference id="120">Omniglot, Finnish (suomi) https://www.omniglot.com/writing/finnish.htm (Accessed on 4 September 2018)</reference>
<reference id="121">Omniglot, Turkmen (Türkmen dili / Түркмен дили) https://www.omniglot.com/writing/turkmen.htm (Accessed on 4 September 2018)</reference>
<reference id="122">Omniglot, Estonian (eesti keel) https://www.omniglot.com/writing/estonian.htm (Accessed on 4 September 2018)</reference>
<reference id="123">Omniglot, Swedish (svenska) https://www.omniglot.com/writing/swedish.htm (Accessed on 4 September 2018)</reference>
<reference id="124">Omniglot, Yapese (Waab) https://www.omniglot.com/writing/yapese.htm (Accessed on 4 September 2018)</reference>
<reference id="125">Omniglot, Dinka (Thuɔŋjäŋ) https://www.omniglot.com/writing/dinka.php (Accessed on 4 September 2018)</reference>
<reference id="126">Omniglot, Kaqchikel (Kaqchikel Chab’äl) https://www.omniglot.com/writing/kaqchikel.htm (Accessed on 4 September 2018)</reference>
<reference id="127">Omniglot, Bashkir/Bashkort (Башҡорт теле / Başqort tele) https://www.omniglot.com/writing/bashkir.htm (Accessed on 4 September 2018)</reference>
<reference id="128">Omniglot, Alsatian (Ëlsässisch) https://www.omniglot.com/writing/alsatian.htm (Accessed on 4 September 2018)</reference>
<reference id="129">Wikipedia, Nuer language https://en.wikipedia.org/wiki/Nuer_language (Accessed on 4 September 2018)</reference>
<reference id="130">Omniglot, Italian (italiano) https://www.omniglot.com/writing/italian.htm (Accessed on 4 September 2018)</reference>
<reference id="131">Wikipedia, Italian orthography https://en.wikipedia.org/wiki/Italian_orthography (Accessed on 4 September 2018)</reference>
<reference id="132">Omniglot, Wolof (Wollof) https://www.omniglot.com/writing/wolof.htm (Accessed on 4 September 2018)</reference>
<reference id="133">Omniglot, Latvian (latviešu valoda) https://www.omniglot.com/writing/latvian.htm (Accessed on 4 September 2018)</reference>
<reference id="134">Omniglot, Tongan (Faka-Tonga) https://www.omniglot.com/writing/tongan.htm (Accessed on 4 September 2018)</reference>
<reference id="135">Omniglot, Hawaiian (ʻŌlelo Hawaiʻi) https://www.omniglot.com/writing/hawaiian.htm (Accessed on 4 September 2018)</reference>
<reference id="136">Omniglot, Marshallese (kajin m̧ajeļ) https://www.omniglot.com/writing/marshallese.php (Accessed on 4 September 2018)</reference>
<reference id="137">Omniglot, Polish (polski) https://www.omniglot.com/writing/polish.htm (Accessed on 4 September 2018)</reference>
<reference id="138">Omniglot, Lithuanian (lietuvių kalba) https://www.omniglot.com/writing/lithuanian.htm (Accessed on 4 September 2018)</reference>
<reference id="139">Omniglot, Danish (dansk) https://www.omniglot.com/writing/danish.htm (Accessed on 4 September 2018)</reference>
<reference id="140">Omniglot, Chamorro (chamoru) https://www.omniglot.com/writing/chamorro.htm (Accessed on 4 September 2018)</reference>
<reference id="141">Omniglot, Umbundu (Úmbúndú) https://www.omniglot.com/writing/umbundu.htm (Accessed on 4 September 2018)</reference>
<reference id="142">Omniglot, Guaraní (Avañe’ẽ) https://www.omniglot.com/writing/guarani.htm (Accessed on 4 September 2018)</reference>
<reference id="143">Wikipedia, Guarani alphabet https://en.wikipedia.org/wiki/Guarani_alphabet (Accessed on 4 September 2018)</reference>
<reference id="144">Omniglot, Nauruan (Ekaiairũ Naoero) https://www.omniglot.com/writing/nauruan.htm (Accessed on 4 September 2018)</reference>
<reference id="145">Omniglot, Khoekhoe (Khoekhoegowab) https://www.omniglot.com/writing/khoekhoe.htm (Accessed on 4 September 2018)</reference>
<reference id="146">Omniglot, Nuer (Naath) https://www.omniglot.com/writing/nuer.htm (Accessed on 4 September 2018)</reference>
<reference id="147">Omniglot, Hausa (Harshen Hausa / هَرْشَن هَوْسَ) https://www.omniglot.com/writing/hausa.htm (Accessed on 4 September 2018)</reference>
<reference id="148">Omniglot, Dagaare https://www.omniglot.com/writing/dagaare.htm (Accessed on 4 September 2018)</reference>
<reference id="149">Omniglot, Fula (Fulfulde, Pulaar, PularFulaare) https://www.omniglot.com/writing/fula.htm (Accessed on 4 September 2018)</reference>
<reference id="150">Omniglot, Croatian (Hrvatski) https://www.omniglot.com/writing/croatian.htm (Accessed on 4 September 2018)</reference>
<reference id="151">Omniglot, Serbian (српски / srpski) https://www.omniglot.com/writing/serbian.htm (Accessed on 4 September 2018)</reference>
<reference id="152">Wikipedia, Polish language https://en.wikipedia.org/wiki/Polish_language (Accessed on 4 September 2018)</reference>
<reference id="153">Omniglot, Slovak (slovenčina) https://www.omniglot.com/writing/slovak.htm (Accessed on 4 September 2018)</reference>
<reference id="154">Evertype Publishing, Lithuanian lietuvių kalba Version 1.1 https://www.evertype.com/alphabets/lithuanian.pdf (Accessed on 4 September 2018)</reference>
<reference id="157">Omniglot, Turkish (Türkçe) https://www.omniglot.com/writing/turkish.htm (Accessed on 4 September 2018)</reference>
<reference id="158">Omniglot, Kurdish (Kurdî / کوردی) https://www.omniglot.com/writing/kurdish.htm (Accessed on 4 September 2018)</reference>
<reference id="159">Omniglot, Azerbaijani (آذربايجانجا ديلي / Азәрбајҹан дили / Azərbaycan dili) https://www.omniglot.com/writing/azeri.htm (Accessed on 4 September 2018)</reference>
<reference id="160">Omniglot, Basque (euskara) https://www.omniglot.com/writing/basque.htm (Accessed on 4 September 2018)</reference>
<reference id="161">Wikipedia, Basque language https://en.wikipedia.org/wiki/Basque_language#Writing_system (Accessed on 4 September 2018)</reference>
<reference id="163">Omniglot, Maltese (Malti) https://www.omniglot.com/writing/maltese.htm (Accessed on 4 September 2018)</reference>
<reference id="164">Omniglot, Venda (Tshivenḓa / Luvenḓa) https://www.omniglot.com/writing/venda.htm (Accessed on 4 September 2018)</reference>
<reference id="168">Omniglot, Brahui (Bráhuí / براوی) https://www.omniglot.com/writing/brahui.htm (Accessed on 4 September 2018)</reference>
<reference id="169">Wikipedia, Fon language https://en.wikipedia.org/wiki/Fon_language (Accessed on 4 September 2018)</reference>
<reference id="170">Omniglot, Ewe (Eʋegbe) https://www.omniglot.com/writing/ewe.htm (Accessed on 4 September 2018)</reference>
<reference id="172">Omniglot, Sorbian (hornjoserbsce/dolnoserbski) https://www.omniglot.com/writing/sorbian.htm (Accessed on 4 September 2018)</reference>
<reference id="173">Peace corps, Botswana, An Introduction to Setswana Language https://files.peacecorps.gov/multimedia/audio/languagelessons/botswana/Bw_Setswana_Language_Lessons.pdf (Accessed on 4 September 2018)</reference>
<reference id="174">Omniglot, Tswana (Setswana) https://www.omniglot.com/writing/tswana.php (Accessed on 4 September 2018)</reference>
<reference id="175">Wikipedia, Afrikaans https://en.wikipedia.org/wiki/Afrikaans (Accessed on 4 September 2018)</reference>
<reference id="176">Omniglot, Albanian (shqip / gjuha shqipe) https://www.omniglot.com/writing/albanian.htm (Accessed on 4 September 2018)</reference>
<reference id="177">Wikipedia, Albanian alphabet https://en.wikipedia.org/wiki/Albanian_alphabet (Accessed on 4 September 2018)</reference>
<reference id="179">Wikipedia, Uyghur Latin alphabet https://en.wikipedia.org/wiki/Uyghur_Latin_alphabet (Accessed on 4 September 2018)</reference>
<reference id="180">Omniglot, Drehu (Deʼu) https://www.omniglot.com/writing/drehu.php (Accessed on 4 September 2018)</reference>
<reference id="182">Omniglot, Haitian Creole (Kreyòl ayisyen) https://www.omniglot.com/writing/haitiancreole.htm (Accessed on 4 September 2018)</reference>
<reference id="183">Wikipedia, Haitian Creole https://en.wikipedia.org/wiki/Haitian_Creole#Orthography (Accessed on 4 September 2018)</reference>
<reference id="184">Omniglot, Minangkabau (Baso Minangkabau / باسو مينڠكاباو) https://www.omniglot.com/writing/minangkabau.htm (Accessed on 4 September 2018)</reference>
<reference id="185">Omniglot, Palauan (a tekoi er a Belau) https://www.omniglot.com/writing/palauan.htm (Accessed on 4 September 2018)</reference>
<reference id="186">Omniglot, Cubeo (pãmié) https://www.omniglot.com/writing/cubeo.htm (Accessed on 4 September 2018)</reference>
<reference id="187">Editorial Alberto Lleras Camargo, Diccionario Ilustrado Bilingüe cubeo-español español-cubeo https://www.sil.org/system/files/reapdata/10/58/27/10582785843693992331766506069073895620/40337_01.pdf (Accessed on 4 September 2018)</reference>
<reference id="188">Omniglot, Inari Saami (Anarâškielâ) https://www.omniglot.com/writing/inarisami.htm (Accessed on 4 September 2018)</reference>
<reference id="189">Omniglot, Compiled by Wolfram Siegel, DAGBANI https://www.omniglot.com/charts/dagbani.pdf (Accessed on 4 September 2018)</reference>
<reference id="190">Omniglot, Ewondo https://www.omniglot.com/writing/ewondo.php (Accessed on 4 September 2018)</reference>
<reference id="191">Omniglot, Luganda (Oluganda) https://www.omniglot.com/writing/ganda.php (Accessed on 4 September 2018)</reference>
<reference id="192">Omniglot, Adzera https://www.omniglot.com/writing/adzera.htm (Accessed on 4 September 2018)</reference>
<reference id="193">Omniglot, Ga (Gã) https://www.omniglot.com/writing/ga.htm (Accessed on 4 September 2018)</reference>
<reference id="194">Omniglot, Duala (Duálá) https://www.omniglot.com/writing/duala.php (Accessed on 4 September 2018)</reference>
<reference id="195">Omniglot, Soga (Lusoga) https://www.omniglot.com/writing/soga.htm (Accessed on 4 September 2018)</reference>
<reference id="196">Omniglot, Alur (Lur) https://www.omniglot.com/writing/alur.htm (Accessed on 4 September 2018)</reference>
<reference id="197">Omniglot, Mandinka (Mandinka kango / لغة مندنكا) https://www.omniglot.com/writing/mandinka.htm (Accessed on 4 September 2018)</reference>
<reference id="198">Omniglot, Acholi (Lwo) https://www.omniglot.com/writing/acholi.htm (Accessed on 4 September 2018)</reference>
<reference id="199">Omniglot, Bambara (Bamanankan) https://www.omniglot.com/writing/bambara.htm (Accessed on 4 September 2018)</reference>
<reference id="200">Omniglot, Raga (Hano) https://www.omniglot.com/writing/raga.htm (Accessed on 4 September 2018)</reference>
<reference id="201">Omniglot, Tatar (tatarça / татарча / تاتارچا) https://www.omniglot.com/writing/tatar.htm (Accessed on 4 September 2018)</reference>
<reference id="202">Omniglot, Zaza (Zazaki / زازاکی) https://www.omniglot.com/writing/zazaki.htm (Accessed on 4 September 2018)</reference>
<reference id="203">Wikipedia, Turkish alphabet https://en.wikipedia.org/wiki/Turkish_alphabet (Accessed on 4 September 2018)</reference>
<reference id="204">School of English, Adam Michiewicz University, Poznań, Poland, Poznań Studies in Contemporary Linguistics 43(1),2007, pp. 169-180, A Demographic Igbo Orthography https://www.degruyter.com/downloadpdf/j/psicl.2007.43.issue-1/v10010-007-0009-0/v10010-007-0009-0.pdf (Accessed on 4 September 2018)</reference>
<reference id="205">Omniglot, Igbo (Asụsụ Igbo) https://www.omniglot.com/writing/igbo.htm (Accessed on 4 September 2018)</reference>
<reference id="206">ItalianPod101, Italian Accents and Proper Italian Pronunciation https://www.italianpod101.com/italian-accents (Accessed on 4 September 2018)</reference>
<reference id="208">Reverso Dictionary, venerdì translation | Italian-English dictionary https://dictionary.reverso.net/italian-english/venerd%C3%AC (Accessed on 4 September 2018)</reference>
<reference id="209">Omniglot, Kikuyu (Gĩkũyũ) https://www.omniglot.com/writing/kikuyu.htm (Accessed on 4 September 2018)</reference>
<reference id="210">Omniglot, Hixkaryána https://www.omniglot.com/writing/hixkaryana.htm (Accessed on 4 September 2018)</reference>
<reference id="211">Omniglot, Maasai (ɔl Maa) https://www.omniglot.com/writing/maasai.htm (Accessed on 4 September 2018)</reference>
<reference id="212">Omniglot, Mossi (Mòoré) https://www.omniglot.com/writing/mossi.htm (Accessed on 4 September 2018)</reference>
<reference id="213">Omniglot, Jenesis. The Bible in Marshallese, 2009., Contributed by Wolfgang Kuhl https://www.omniglot.com/babel/marshallese.htm (Accessed on 4 September 2018)</reference>
<reference id="214">Wikipedia, Cedilla https://en.wikipedia.org/wiki/Cedilla#Marshallese (Accessed on 4 September 2018)</reference>
<reference id="215">Wikipedia, Marshallese language https://en.wikipedia.org/wiki/Marshallese_language#Display_issues (Accessed on 4 September 2018)</reference>
<reference id="216">Trussel, Marshallese-English Online Dictionary https://www.trussel2.com/MOD/ (Accessed on 4 September 2018)</reference>
<reference id="218">Omniglot, Susu (Sosoxi) https://www.omniglot.com/writing/susu.htm (Accessed on 4 September 2018)</reference>
<reference id="219">Omniglot, Zarma (Zarmaciine) https://www.omniglot.com/writing/zarma.htm (Accessed on 4 September 2018)</reference>
<reference id="220">Omniglot, Pitjantjatjara https://www.omniglot.com/writing/pitjantjatjara.htm (Accessed on 4 September 2018)</reference>
<reference id="221">Omniglot, Spanish (español/castellano) https://www.omniglot.com/writing/spanish.htm (Accessed on 4 September 2018)</reference>
<reference id="222">Omniglot, Filipino (wikang Filipino) https://www.omniglot.com/writing/filipino.htm (Accessed on 4 September 2018)</reference>
<reference id="223">Omniglot, Chavacano https://www.omniglot.com/writing/chavacano.php (Accessed on 4 September 2018)</reference>
<reference id="224">Wikipedia, Ilocano language https://en.wikipedia.org/wiki/Ilocano_language#Modern_alphabet (Accessed on 4 September 2018)</reference>
<reference id="225">Omniglot, Quechua (Runasimi) https://www.omniglot.com/writing/quechua.htm (Accessed on 4 September 2018)</reference>
<reference id="226">Wikipedia, Quechua alphabet https://en.wikipedia.org/wiki/Quechua_alphabet (Accessed on 4 September 2018)</reference>
<reference id="227">Omniglot, Cape Verdean Creole (Kriolu) https://www.omniglot.com/writing/kriol.php (Accessed on 4 September 2018)</reference>
<reference id="228">Omniglot, Waray-Waray https://www.omniglot.com/writing/waray.php (Accessed on 4 September 2018)</reference>
<reference id="229">Omniglot, Lozi (siLozi) https://www.omniglot.com/writing/lozi.htm (Accessed on 4 September 2018)</reference>
<reference id="230">africanlanguages.com, Sesotho sa Leboa (Northern Sotho) https://africanlanguages.com/northern_sotho/ (Accessed on 4 September 2018)</reference>
<reference id="232">Wikipedia, Chechen language https://en.wikipedia.org/wiki/Chechen_language (Accessed on 4 September 2018)</reference>
<reference id="233">Omniglot, Hungarian (magyar) https://www.omniglot.com/writing/hungarian.htm (Accessed on 4 September 2018)</reference>
<reference id="234">Wikipedia, Hungarian alphabet https://en.wikipedia.org/wiki/Hungarian_alphabet (Accessed on 4 September 2018)</reference>
<reference id="236">Omniglot, Lingala https://www.omniglot.com/writing/lingala.htm (Accessed on 4 September 2018)</reference>
<reference id="237">Omniglot, Akan https://www.omniglot.com/writing/akan.htm (Accessed on 4 September 2018)</reference>
<reference id="238">Wikipedia, Mossi language https://en.wikipedia.org/wiki/Mossi_language (Accessed on 4 September 2018)</reference>
<reference id="239">SIL-Sudan, OCCASIONAL PAPERS in the study of SUDANESE LANGUAGES No. 9 (p. 75) https://www.sil.org/system/files/reapdata/10/06/46/100646256099282892829790816212446104791/OPSL_9.pdf (Accessed on 4 September 2018)</reference>
<reference id="240">Omniglot, Kanuri https://www.omniglot.com/writing/kanuri.htm (Accessed on 4 September 2018)</reference>
<reference id="241">Omniglot, Bugis (Basa Ugi ) https://www.omniglot.com/writing/bugis.htm (Accessed on 4 September 2018)</reference>
<reference id="242">Omniglot, Mizo (Mizo ṭawng) https://www.omniglot.com/writing/mizo.htm (Accessed on 4 September 2018)</reference>
<reference id="243">Omniglot, Miskito (Mískitu) https://www.omniglot.com/writing/miskito.htm (Accessed on 4 September 2018)</reference>
<reference id="245">Wikipedia, Papiamento https://en.wikipedia.org/wiki/Papiamento (Accessed on 4 September 2018)</reference>
<reference id="246">Omniglot, Papiamento (Papiamentu) https://www.omniglot.com/writing/papiamento.php (Accessed on 4 September 2018)</reference>
<reference id="247">Omniglot, Chichewa (Chicheŵa) https://www.omniglot.com/writing/chichewa.php (Accessed on 4 September 2018)</reference>
<reference id="248">Native Languages of the Americas website, Vocabulary in Native American Languages: Mam Words https://www.native-languages.org/mam_words.htm (Accessed on 4 September 2018)</reference>
<reference id="249">Omniglot, Mam (Qyol Mam) https://www.omniglot.com/writing/mam.htm (Accessed on 4 September 2018)</reference>
<reference id="250">Wikipedia, Pulaar language https://en.wikipedia.org/wiki/Pulaar_language (Accessed on 4 September 2018)</reference>
<reference id="251">Wikipedia, Fula language https://en.wikipedia.org/wiki/Fula_language#Writing_systems (Accessed on 4 September 2018)</reference>
<reference id="252">Wikipedia, Polish alphabet https://en.wikipedia.org/wiki/Polish_alphabet (Accessed on 4 September 2018)</reference>
<reference id="253">Wikipedia, French orthography https://en.wikipedia.org/wiki/French_orthography (Accessed on 4 September 2018)</reference>
<reference id="254">Omniglot, Yoruba (Èdè Yorùbá) https://www.omniglot.com/writing/yoruba.htm (Accessed on 4 September 2018)</reference>
<reference id="255">Omniglot, Esperanto https://www.omniglot.com/writing/esperanto.htm (Accessed on 4 September 2018)</reference>
<reference id="256">Omniglot, Welsh (Cymraeg) https://www.omniglot.com/writing/welsh.htm (Accessed on 4 September 2018)</reference>
<reference id="257">Wikipedia, List of Latin-script letters https://en.wikipedia.org/wiki/List_of_Latin-script_letters (Accessed on 4 September 2018)</reference>
<reference id="258">Omniglot, Montenegrin https://www.omniglot.com/writing/montenegrin.htm (Accessed on 20 March 2019)</reference>
<reference id="275">Omniglot, Shavante https://www.omniglot.com/writing/shavante.php (Accessed on 24 September 2020)</reference>
<reference id="276">Wikipedia, Malagasy Language https://en.wikipedia.org/wiki/Malagasy_language (Accessed on 24 September 2020)</reference>
<reference id="277">Wikipedia, Serer language, https://en.wikipedia.org/wiki/Serer_language, accessed on 6 April 2021</reference>
<reference id="278">Wikipedia, Kpelle language, https://en.wikipedia.org/wiki/Kpelle_language, accessed on 6 April 2021</reference>
<reference id="279">Wikipedia, Catalan Orthography, https://en.wikipedia.org/wiki/Catalan_orthography, accessed on 28 November 2022</reference>
<reference id="280">Fundacio PuntCAT registry (.CAT), IDN table for CAT_ca Version 1.0, 12 February 2006, https://www.iana.org/domains/idn-tables/tables/cat_ca_1.0.html</reference>
<reference id="281">Fundacio PuntCAT registry (.CAT), Rules of the .cat domain, https://domini.cat/en/rules-of-the-cat-domain/</reference>
<reference id="320">RFC 5891, Internationalized Domain Names in Applications (IDNA): Protocol https://tools.ietf.org/html/rfc5891</reference>
</references>
</meta>
<data>
<char cp="002D" not-when="hyphen-minus-disallowed" tag="sc:Zyyy" ref="0" comment="HYPHEN-MINUS; &#x235F;" />
<char cp="0030" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT ZERO; &#x235F;" />
<char cp="0031" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT ONE; &#x235F;" />
<char cp="0032" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT TWO; &#x235F;" />
<char cp="0033" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT THREE; &#x235F;" />
<char cp="0034" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT FOUR; &#x235F;" />
<char cp="0035" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT FIVE; &#x235F;" />
<char cp="0036" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT SIX; &#x235F;" />
<char cp="0037" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT SEVEN; &#x235F;" />
<char cp="0038" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT EIGHT; &#x235F;" />
<char cp="0039" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT NINE; &#x235F;" />
<char cp="0061" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0062" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0063" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0064" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0065" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0066" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0067" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0068" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0069" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="006A" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="006B" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="006C" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="006D" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="006E" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="006F" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0070" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0071" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0072" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0073" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0074" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0075" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0076" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0077" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0078" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0079" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="007A" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="00E0" tag="sc:Latn" ref="0 106 114 130 131 132" comment="Italian (1), French (1), Galician (2), Wolof (4)" />
<char cp="00E2" tag="sc:Latn" ref="0 106 109 110 113 114 115 116 117 275" comment="Vietnamese (1), Romanian (1), Skolt Sami (2), French (1), Galician (2), West Frisian (1), Friulian (4), Xavante (4)" />
<char cp="00E3" tag="sc:Latn" ref="0 141 142 143 144 145" comment="Umbundu (3), Guarani (1), Nauruan (3), Khoekhoe (4)" />
<char cp="00E4" tag="sc:Latn" ref="0 107 119 120 121 122 123 124 125 126 127 128 129" comment="German (1), Finnish (1), Turkmen (1), Estonian (1), Swedish (1), Lule Sami (2), Yapese (2), Dinka (4), Kaqchikel (4), Bashkir (4), Alsatian (5), Nuer (4)" />
<char cp="00E5" tag="sc:Latn" ref="0 107 120 123 139 140" comment="Danish (1), Finnish (1), Chamorro (1), Swedish (1), Lule Sami (2)" />
<char cp="00E6" tag="sc:Latn" ref="0 102 103 139" comment="Danish (1), Icelandic (1), Faroese (2)" />
<char cp="00E7" tag="sc:Latn" ref="0 106 114 116 121 127 157 158 159 160 161" comment="Turkish (1), Turkmen (1), Kurdish (2), French (1), Azerbaijani (1), Basque (1), Galician (2), Friulian (4), Bashkir (4)" />
<char cp="00E8" tag="sc:Latn" ref="0 114 130 175 182 183" comment="French (1), Italian (1), Afrikaans (1), Haitian Creole (1), French (1)" />
<char cp="00E9" tag="sc:Latn" ref="0 100 101 102 105 106 114 115 117 130 132 275" comment="French (1), Italian (1), Spanish (1), Czech (1), Icelandic (1), Chuukese (2), Galician (2), Wolof (4), Xavante (4), West Frisian (2)" />
<char cp="00EA" tag="sc:Latn" ref="0 109 114 115 116 158 173 174 175" comment="French (1), Tswana (1), Afrikaans (1), Vietnamese (1), Kurdish (2), West Frisian (2), Friulian (4)" />
<char cp="00EB" tag="sc:Latn" ref="0 114 115 124 126 129 132 175 176 177 179 180" comment="Afrikaans (1), Albanian (1), French (1), Uyghur (2), Yapese (2), Wolof (4), Drehu (4), Kaqchikel (4), West Frisian (2), Nuer (4)" />
<char cp="00EC" tag="sc:Latn" ref="0 130 206 208" comment="Italian (1)" />
<char cp="00EE" tag="sc:Latn" ref="0 110 114 116 158 175" comment="Afrikaans (1), Romanian (1), Kurdish (2), French (1), Friulian (4)" />
<char cp="00F0" tag="sc:Latn" ref="0 102 103" comment="Faroese (2), Icelandic (1)" />
<char cp="00F2" tag="sc:Latn" ref="0 130 182 183" comment="Italian (1), Haitian Creole (1)" />
<char cp="00F4" tag="sc:Latn" ref="0 106 109 114 115 116 117 173 174 175 230 275" comment="Tswana (1), Afrikaans (1), Vietnamese (1), French (1), Northern Sotho (1), West Frisian (2), Galician (2), Friulian (4), Xavante (4)" />
<char cp="00F5" tag="sc:Latn" ref="0 113 117 122 141 142 143 144 145 275" comment="Estonian (1), Skolt Sami (2), Umbundu (3), Guarani (1), Nauruan (3), Xavante (4), Khoekhoe (4)" />
<char cp="00F6" tag="sc:Latn" ref="0 115 119 120 123 124 125 126 127 129 157 175 179 180 232" comment="German (1), Finnish (1), Afrikaans (1), Turkish (1), Swedish (1), Uygur (2), Yapese (2), Drehu (4), Kaqchikel (4), Dinka (4), Bashkir (4), Chechen (2), 1992 Version, West Frisian (2), Nuer (4)" />
<char cp="00F8" tag="sc:Latn" ref="0 103 139" comment="Danish (1), Faroese (2)" />
<char cp="00F9" tag="sc:Latn" ref="0 114 130 206 245 246 253" comment="Italian (1), French (1), Papiamento (1)" />
<char cp="00FB" tag="sc:Latn" ref="0 114 115 116 158 175 202 243" comment="Afrikaans (1), Kurdish (2), French (1), Miskito (2), West Frisian (2), Friulian (4), Zazaki (4)" />
<char cp="00FD" tag="sc:Latn" ref="0 101 102 103 121 142 143" comment="Turkmen (1), Czech (1), Icelandic (1), Faroese (2), Guarani (1)" />
<char cp="00FE" tag="sc:Latn" ref="0 102" comment="Icelandic (1)" />
<char cp="00FF" tag="sc:Latn" ref="0 114 253 257" comment="French (1)" />
<char cp="0103" tag="sc:Latn" ref="0 109 110" comment="Vietnamese (1), Romanian (1)" />
<char cp="0105" tag="sc:Latn" ref="0 137 138" comment="Polish (1), Lithuanian (1)" />
<char cp="0107" tag="sc:Latn" ref="0 150 151 152" comment="Croatian (1), Serbian (1), Polish (1)" />
<char cp="0109" tag="sc:Latn" ref="0 255" comment="Esperanto (3)" />
<char cp="010D" tag="sc:Latn" ref="0 108 133 150 151 153 154" comment="Croatian (1), Serbian (1), Latvian (1), Slovak (1), Northern Sami (2), Lithuanian (1)" />
<char cp="010F" tag="sc:Latn" ref="0 101 153" comment="Czech (1), Slovak (1)" />
<char cp="0111" tag="sc:Latn" ref="0 108 109 150 151 168" comment="Croatian (1), Serbian (1), Vietnamese (1), Northern Sami (2), Brahui (5)" />
<char cp="0113" tag="sc:Latn" ref="0 133 134 135 184" comment="Latvian (1), Hawaiian (2), Tongan (1), Minangkabau (5)" />
<char cp="0117" tag="sc:Latn" ref="0 138 154" comment="Lithuanian (1)" />
<char cp="0119" tag="sc:Latn" ref="0 138 152 154 185" comment="Polish (1), Palauan (2), Lithuanian (1)" />
<char cp="011B" tag="sc:Latn" ref="0 101 172" comment="Czech (1), Sorbian (4)" />
<char cp="011D" tag="sc:Latn" ref="0 255" comment="Esperanto (3)" />
<char cp="011F" tag="sc:Latn" ref="0 127 157 159 201 202" comment="Turkish (1), Tatar (2), Azeri (1), Bashkir (4), Zaza (5)" />
<char cp="0123" tag="sc:Latn" ref="0 133 168" comment="Latvian (1), Brahui (5)" />
<char cp="0125" tag="sc:Latn" ref="0 255" comment="Esperanto (3)" />
<char cp="0127" tag="sc:Latn" ref="0 163" comment="Maltese (1)" />
<char cp="012B" tag="sc:Latn" ref="0 133 134 135 138" comment="Latvian (1), Lithuanian (1), Hawaiian (2), Tongan (1)" />
<char cp="012F" tag="sc:Latn" ref="0 154" comment="Lithuanian (1)" />
<char cp="0135" tag="sc:Latn" ref="0 255" comment="Esperanto (3)" />
<char cp="0137" tag="sc:Latn" ref="0 133" comment="Latvian (1)" />
<char cp="013A" tag="sc:Latn" ref="0 153" comment="Slovak (1)" />
<char cp="013C" tag="sc:Latn" ref="0 133 168 213 214" comment="Latvian (1), Marshallese (1), Brahui (5)" />
<char cp="013E" tag="sc:Latn" ref="0 153" comment="Slovak (1)" />
<char cp="0142" tag="sc:Latn" ref="0 152" comment="Polish (1)" />
<char cp="0146" tag="sc:Latn" ref="0 133 136" comment="Latvian (1), Marshallese (1)" />
<char cp="0148" tag="sc:Latn" ref="0 101 121 153" comment="Turkmen (1), Czech (1), Slovak (1)" />
<char cp="0151" tag="sc:Latn" ref="0 233 234" comment="Hungarian (1)" />
<char cp="0153" tag="sc:Latn" ref="0 114 253" comment="French (1)" />
<char cp="0155" tag="sc:Latn" ref="0 153 168" comment="Slovak (1), Brahui (5)" />
<char cp="0159" tag="sc:Latn" ref="0 101 172" comment="Czech (1), Sorbian (4)" />
<char cp="015B" tag="sc:Latn" ref="0 152 258" comment="Polish (1), Montenegrin (1)" />
<char cp="015D" tag="sc:Latn" ref="0 255" comment="Esperanto (3)" />
<char cp="015F" tag="sc:Latn" ref="0 121 127 157 158 159 168 201 202" comment="Turkish (1), Turkmen (1), Kurdish (2), Tatar (2), Azeri (1), Bashkir (4), Brahui (5), Zaza (5)" />
<char cp="0161" tag="sc:Latn" ref="0 108 133 150 151 154 174 230" comment="Tswana (1), Croatian (1), Serbian (1), Latvian (1), Northern Sotho (1), Northern Sami (2), Lithuanian (1)" />
<char cp="0165" tag="sc:Latn" ref="0 101 153" comment="Czech (1), Slovak (1)" />
<char cp="0167" tag="sc:Latn" ref="0 108 168" comment="Northern Sami (2), Brahui (5)" />
<char cp="016B" tag="sc:Latn" ref="0 133 134 135 136 138 154" comment="Latvian (1), Hawaiian (2), Lithuanian (1), Marshallese (1), Tongan (1)" />
<char cp="016D" tag="sc:Latn" ref="0 255" comment="Esperanto (3)" />
<char cp="016F" tag="sc:Latn" ref="0 101" comment="Czech (1)" />
<char cp="0171" tag="sc:Latn" ref="0 233 234" comment="Hungarian (1)" />
<char cp="0173" tag="sc:Latn" ref="0 138 154" comment="Lithuanian (1)" />
<char cp="0175" tag="sc:Latn" ref="0 247 256" comment="Chichewa (3), Welsh (2)" />
<char cp="0177" tag="sc:Latn" ref="0 256" comment="Welsh (2)" />
<char cp="017A" tag="sc:Latn" ref="0 152 168 172 252 258" comment="Polish (1), Brahui (5), Sorbian (4), Montenegrin (1)" />
<char cp="017E" tag="sc:Latn" ref="0 108 121 133 150 151 153 154 232" comment="Lithuanian (1), Croatian (1), Serbian (1), Turkmen (1), Latvian (1), Slovak (1), Northern Sami (2), Chechen (2) 1925 Version" />
<char cp="0188" tag="sc:Latn" ref="0 277" comment="Serer (5)" />
<char cp="0199" tag="sc:Latn" ref="0 147" comment="Hausa (2)" />
<char cp="01A1" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="01A5" tag="sc:Latn" ref="0 277" comment="Serer (5)" />
<char cp="01AD" tag="sc:Latn" ref="0 277" comment="Serer (5)" />
<char cp="01B0" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="01B4" tag="sc:Latn" ref="0 148 149 251" comment="Dagaare - Burkina Faso (4), Fula (3)" />
<char cp="01E9" tag="sc:Latn" ref="0 113" comment="Skolt Sami (2)" />
<char cp="01EF" tag="sc:Latn" ref="0 113" comment="Skolt Sami (2)" />
<char cp="0219" tag="sc:Latn" ref="3 110" comment="Romanian (1)" />
<char cp="021B" tag="sc:Latn" ref="3 110" comment="Romanian (1)" />
<char cp="024D" tag="sc:Latn" ref="8 240" comment="Kanuri (3)" />
<char cp="0253" tag="sc:Latn" ref="0 147 148 250" comment="Hausa (2), Dagaare - Burkina Faso (4), Pulaar (3)" />
<char cp="0254" tag="sc:Latn" ref="0 129 146 148 169 170 189 190 193 194 236 237" comment="Dagaare - Burkina Faso (4), Dagbani (Dagomba) (4), Lingala (2), Akan (3), Ewondo (3), Fon (3), Nuer (4), Ga (4), Duala (3), EWE (3), Nuer (4)" />
<char cp="0256" tag="sc:Latn" ref="0 169 170" comment="Fon (3), Ewe (3)" />
<char cp="0257" tag="sc:Latn" ref="0 147 149 250" comment="Hausa (2), Fula (3)" />
<char cp="0259" tag="sc:Latn" ref="0 159 170 190 241" comment="Azeri, Azerbaijani (1), Ewondo (3), Ewe (3), Bugis (3)" />
<char cp="025B" tag="sc:Latn" ref="0 129 148 169 170 189 190 193 194 199 212 236 237 238" comment="Dagaare - Burkina Faso (4), Lingala (2), Akan (3), Ewondo (3), Dagbani (Dagomba), (4), Fon (3), Mossi (3), Ga (4), Ewe (3), Duala (3), Bambara (4), Nuer (4)" />
<char cp="0260" tag="sc:Latn" ref="0 278" comment="Kpelle (5)" />
<char cp="0268" tag="sc:Latn" ref="0 186 189 210 211" comment="Cubeo (3), Dagbani (Dagomba) (4), HIxkaryána (4), Maasai (5)" />
<char cp="0272" tag="sc:Latn" ref="0 199 218 219" comment="Susu (4), Zarma (4), Bambara (4)" />
<char cp="0289" tag="sc:Latn" ref="0 186 187 211" comment="Cubeo (3), Maasai (5)" />
<char cp="0292" tag="sc:Latn" ref="0 113 189" comment="Skolt Sami (2), Dagbani (Dagomba) (4)" />
<char cp="1E13" tag="sc:Latn" ref="0 164 257" comment="Venda (1)" />
<char cp="1E3D" tag="sc:Latn" ref="0 164 257" comment="Venda (1)" />
<char cp="1E49" tag="sc:Latn" ref="0 220" comment="Pitjantjatjara (4)" />
<char cp="1E4B" tag="sc:Latn" ref="0 164 257" comment="Venda (1)" />
<char cp="1E63" tag="sc:Latn" ref="0 254" comment="Yoruba (2)" />
<char cp="1E6D" tag="sc:Latn" ref="0 242" comment="Mizo (4)" />
<char cp="1E71" tag="sc:Latn" ref="0 164 257" comment="Venda (1)" />
<char cp="1E8D" tag="sc:Latn" ref="0 248 249" comment="Mam (4)" />
<char cp="1EA1" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EA5" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EA7" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EA9" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EAB" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EAD" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EAF" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EB1" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EB3" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EB5" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EB7" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EB9" tag="sc:Latn" ref="0 254" comment="Yoruba (2)" />
<char cp="1EBB" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EBF" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EC1" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EC3" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EC5" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EC7" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1ECB" tag="sc:Latn" ref="0 205" comment="Igbo (2)" />
<char cp="1ECD" tag="sc:Latn" ref="0 136 204 205 215 216 254" comment="Igbo (2), Yoruba (2), Marshallese (1)" />
<char cp="1ED1" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1ED3" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1ED5" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1ED7" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1ED9" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EDB" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EDD" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EDF" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EE1" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EE3" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EE5" tag="sc:Latn" ref="0 109 204 205" comment="Vietnamese (1), Igbo (2)" />
<char cp="1EE9" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EEB" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EED" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EEF" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EF1" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EF5" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EF9" tag="sc:Latn" ref="0 109 142" comment="Vietnamese (1), Guarani (1)" />
</data>
<!--Rules section goes here-->
<rules>
<!--Character class definitions go here-->
<!--Whole label evaluation and context rules go here-->
<rule name="leading-combining-mark" ref="320" comment="RFC 5891 restrictions on placement of combining marks &#x235F;">
<start />
<union>
<class property="gc:Mn" />
<class property="gc:Mc" />
</union>
</rule>
<rule name="hyphen-minus-disallowed" ref="320" comment="RFC 5891 restrictions on placement of U+002D HYPHEN-MINUS &#x235F;">
<choice>
<rule comment="no leading hyphen">
<look-behind>
<start />
</look-behind>
<anchor />
</rule>
<rule comment="no trailing hyphen">
<anchor />
<look-ahead>
<end />
</look-ahead>
</rule>
<rule comment="no consecutive hyphens in third and fourth">
<look-behind>
<start />
<any />
<any />
<char cp="002D" comment="hyphen-minus" />
</look-behind>
<anchor />
</rule>
</choice>
</rule>
<!--Action elements go here - order defines precedence-->
<action disp="invalid" match="leading-combining-mark" comment="labels with leading combining marks are invalid &#x235F;" />
<action disp="invalid" any-variant="out-of-repertoire-var" comment="any variant label with a code point out of repertoire is invalid &#x235F;" />
<action disp="invalid" match="dot-L-dot" comment="labels with one L sharing two middle dots are invalid" />
<action disp="blocked" any-variant="blocked" comment="any variant label containing blocked variants is blocked &#x235F;" />
<action disp="allocatable" all-variants="allocatable" comment="variant labels with all variants allocatable are allocatable &#x235F;" />
<action disp="allocatable" all-variants="fallback" comment="any label with all variants of type fallback is allocatable &#x235F;" />
<action disp="blocked" any-variant="fallback" comment="any variant label with a mix of variant forms is blocked &#x235F;" />
<action disp="valid" all-variants="r-original" comment="any remaining label containing only original code points is valid &#x235F;" />
<action disp="valid" comment="catch all (default action) &#x235F;" />
</rules>
</lgr>
@@ -86,7 +86,7 @@ public final class ForeignKeyUtils {
*/
public static <E extends EppResource> ImmutableMap<String, VKey<E>> load(
Class<E> clazz, Collection<String> foreignKeys, final DateTime now) {
return load(clazz, foreignKeys, false).entrySet().stream()
return loadMostRecentResources(clazz, foreignKeys, false).entrySet().stream()
.filter(e -> now.isBefore(e.getValue().deletionTime()))
.collect(toImmutableMap(Entry::getKey, e -> VKey.create(clazz, e.getValue().repoId())));
}
@@ -104,8 +104,9 @@ public final class ForeignKeyUtils {
* same max {@code deleteTime}, usually {@code END_OF_TIME}, lest this method throws an error due
* to duplicate keys.
*/
private static <E extends EppResource> ImmutableMap<String, MostRecentResource> load(
Class<E> clazz, Collection<String> foreignKeys, boolean useReplicaTm) {
public static <E extends EppResource>
ImmutableMap<String, MostRecentResource> loadMostRecentResources(
Class<E> clazz, Collection<String> foreignKeys, boolean useReplicaTm) {
String fkProperty = RESOURCE_TYPE_TO_FK_PROPERTY.get(clazz);
JpaTransactionManager tmToUse = useReplicaTm ? replicaTm() : tm();
return tmToUse.reTransact(
@@ -148,7 +149,7 @@ public final class ForeignKeyUtils {
ImmutableList<String> foreignKeys =
keys.stream().map(key -> (String) key.getKey()).collect(toImmutableList());
ImmutableMap<String, MostRecentResource> existingKeys =
ForeignKeyUtils.load(clazz, foreignKeys, true);
ForeignKeyUtils.loadMostRecentResources(clazz, foreignKeys, true);
// The above map only contains keys that exist in the database, so we re-add the
// missing ones with Optional.empty() values for caching.
return Maps.asMap(
@@ -234,7 +235,7 @@ public final class ForeignKeyUtils {
e -> VKey.create(clazz, e.getValue().get().repoId())));
}
record MostRecentResource(String repoId, DateTime deletionTime) {
public record MostRecentResource(String repoId, DateTime deletionTime) {
static MostRecentResource create(String repoId, DateTime deletionTime) {
return new MostRecentResource(repoId, deletionTime);
@@ -15,7 +15,6 @@
package google.registry.model.tmch;
import static google.registry.config.RegistryConfig.getClaimsListCacheDuration;
import static google.registry.persistence.transaction.QueryComposer.Comparator.EQ;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
@@ -79,14 +78,11 @@ public class ClaimsListDao {
*/
private static ClaimsList getUncached() {
return tm().reTransact(
() -> {
Long revisionId =
tm().query("SELECT MAX(revisionId) FROM ClaimsList", Long.class)
.getSingleResult();
return tm().createQueryComposer(ClaimsList.class)
.where("revisionId", EQ, revisionId)
.first();
})
() ->
tm().query("FROM ClaimsList ORDER BY revisionId DESC", ClaimsList.class)
.setMaxResults(1)
.getResultStream()
.findFirst())
.orElse(ClaimsList.create(START_OF_TIME, ImmutableMap.of()));
}
@@ -30,6 +30,7 @@ import google.registry.export.DriveModule;
import google.registry.export.sheet.SheetsServiceModule;
import google.registry.flows.ServerTridProviderModule;
import google.registry.flows.custom.CustomLogicFactoryModule;
import google.registry.flows.domain.DomainDeletionTimeCacheModule;
import google.registry.groups.DirectoryModule;
import google.registry.groups.GmailModule;
import google.registry.groups.GroupsModule;
@@ -66,6 +67,7 @@ import jakarta.inject.Singleton;
CredentialModule.class,
CustomLogicFactoryModule.class,
DirectoryModule.class,
DomainDeletionTimeCacheModule.class,
DriveModule.class,
GmailModule.class,
GroupsModule.class,
@@ -23,6 +23,7 @@ import google.registry.batch.DeleteLoadTestDataAction;
import google.registry.batch.DeleteProberDataAction;
import google.registry.batch.ExpandBillingRecurrencesAction;
import google.registry.batch.RelockDomainAction;
import google.registry.batch.RemoveAllDomainContactsAction;
import google.registry.batch.ResaveAllEppResourcesPipelineAction;
import google.registry.batch.ResaveEntityAction;
import google.registry.batch.SendExpiringCertificateNotificationEmailAction;
@@ -270,6 +271,8 @@ interface RequestComponent {
ReadinessProbeActionFrontend readinessProbeActionFrontend();
RemoveAllDomainContactsAction removeAllDomainContactsAction();
RdapAutnumAction rdapAutnumAction();
RdapDomainAction rdapDomainAction();
@@ -26,6 +26,7 @@ import google.registry.export.DriveModule;
import google.registry.export.sheet.SheetsServiceModule;
import google.registry.flows.ServerTridProviderModule;
import google.registry.flows.custom.CustomLogicFactoryModule;
import google.registry.flows.domain.DomainDeletionTimeCacheModule;
import google.registry.groups.DirectoryModule;
import google.registry.groups.GmailModule;
import google.registry.groups.GroupsModule;
@@ -56,6 +57,7 @@ import jakarta.inject.Singleton;
CloudTasksUtilsModule.class,
CredentialModule.class,
CustomLogicFactoryModule.class,
DomainDeletionTimeCacheModule.class,
DirectoryModule.class,
DriveModule.class,
GmailModule.class,
@@ -23,6 +23,7 @@ import google.registry.batch.DeleteLoadTestDataAction;
import google.registry.batch.DeleteProberDataAction;
import google.registry.batch.ExpandBillingRecurrencesAction;
import google.registry.batch.RelockDomainAction;
import google.registry.batch.RemoveAllDomainContactsAction;
import google.registry.batch.ResaveAllEppResourcesPipelineAction;
import google.registry.batch.ResaveEntityAction;
import google.registry.batch.SendExpiringCertificateNotificationEmailAction;
@@ -153,6 +154,8 @@ public interface BackendRequestComponent {
RelockDomainAction relockDomainAction();
RemoveAllDomainContactsAction removeAllDomainContactsAction();
ResaveAllEppResourcesPipelineAction resaveAllEppResourcesPipelineAction();
ResaveEntityAction resaveEntityAction();
@@ -22,6 +22,7 @@ import google.registry.config.CredentialModule;
import google.registry.config.RegistryConfig.ConfigModule;
import google.registry.flows.ServerTridProviderModule;
import google.registry.flows.custom.CustomLogicFactoryModule;
import google.registry.flows.domain.DomainDeletionTimeCacheModule;
import google.registry.groups.DirectoryModule;
import google.registry.groups.GmailModule;
import google.registry.groups.GroupsModule;
@@ -50,6 +51,7 @@ import jakarta.inject.Singleton;
CustomLogicFactoryModule.class,
CloudTasksUtilsModule.class,
DirectoryModule.class,
DomainDeletionTimeCacheModule.class,
FrontendRequestComponentModule.class,
GmailModule.class,
GroupsModule.class,
@@ -23,6 +23,7 @@ import google.registry.config.RegistryConfig.ConfigModule;
import google.registry.export.DriveModule;
import google.registry.flows.ServerTridProviderModule;
import google.registry.flows.custom.CustomLogicFactoryModule;
import google.registry.flows.domain.DomainDeletionTimeCacheModule;
import google.registry.groups.DirectoryModule;
import google.registry.groups.GroupsModule;
import google.registry.groups.GroupssettingsModule;
@@ -47,6 +48,7 @@ import jakarta.inject.Singleton;
CustomLogicFactoryModule.class,
CloudTasksUtilsModule.class,
DirectoryModule.class,
DomainDeletionTimeCacheModule.class,
DriveModule.class,
GroupsModule.class,
GroupssettingsModule.class,
@@ -15,20 +15,14 @@
package google.registry.request.auth;
import static com.google.common.base.Preconditions.checkState;
import static google.registry.config.RegistryConfig.getUserAuthCachingDuration;
import static google.registry.config.RegistryConfig.getUserAuthCachingEnabled;
import static google.registry.config.RegistryConfig.getUserAuthMaxCachedEntries;
import static google.registry.model.CacheUtils.newCacheBuilder;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.github.benmanes.caffeine.cache.LoadingCache;
import com.google.api.client.json.webtoken.JsonWebSignature;
import com.google.auth.oauth2.TokenVerifier.VerificationException;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Splitter;
import com.google.common.collect.ImmutableSet;
import com.google.common.flogger.FluentLogger;
import google.registry.config.RegistryConfig;
import google.registry.config.RegistryConfig.Config;
import google.registry.model.console.User;
import google.registry.persistence.VKey;
@@ -77,40 +71,6 @@ public abstract class OidcTokenAuthenticationMechanism implements Authentication
this.tokenVerifier = tokenVerifier;
}
/**
* An in-memory cache for User entities, built using the project's standard utility.
*
* <p>This cache reduces database load by temporarily storing User objects after they are fetched.
* It is configured to cache negative results (i.e., when a user is not found) to prevent repeated
* lookups for invalid users. The cache's behavior (enabled, expiry, size) is controlled by
* settings in {@link RegistryConfig}.
*/
@VisibleForTesting
static LoadingCache<String, Optional<User>> userCache =
newCacheBuilder(getUserAuthCachingDuration())
.maximumSize(getUserAuthMaxCachedEntries())
.build(OidcTokenAuthenticationMechanism::loadUser);
/**
* A loader function that defines how to fetch a User from the database on a cache miss.
*
* <p>This is the single point of entry to the database for this authentication flow. It will only
* be invoked by the cache when a requested user is not already in memory.
*/
@VisibleForTesting
static Optional<User> loadUser(String email) {
VKey<User> userVKey = VKey.create(User.class, email);
return tm().transact(() -> tm().loadByKeyIfPresent(userVKey));
}
@VisibleForTesting
public static void setCacheForTesting(LoadingCache<String, Optional<User>> cache) {
checkState(
RegistryEnvironment.get() == RegistryEnvironment.UNITTEST,
"Cannot set cache outside of a test environment");
OidcTokenAuthenticationMechanism.userCache = cache;
}
@Override
public AuthResult authenticate(HttpServletRequest request) {
if (RegistryEnvironment.get().equals(RegistryEnvironment.UNITTEST)
@@ -152,24 +112,23 @@ public abstract class OidcTokenAuthenticationMechanism implements Authentication
logger.atInfo().log("No email address from the OIDC token:\n%s", token.getPayload());
return AuthResult.NOT_AUTHENTICATED;
}
Optional<User> maybeUser;
if (getUserAuthCachingEnabled()) {
// If caching is ON, use the cache.
maybeUser = userCache.get(email);
} else {
// If caching is OFF, fall back to the original direct database call.
maybeUser = loadUser(email);
}
// Short-circuit the user lookup for known service accounts.
// This check bypasses the database lookup for high-volume
// traffic from trusted system accounts to reduce database load.
if (serviceAccountEmails.contains(email)) {
return AuthResult.createApp(email);
}
logger.atInfo().log("No service account found for email address %s, loading the User", email);
Optional<User> maybeUser =
tm().transact(() -> tm().loadByKeyIfPresent(VKey.create(User.class, email)));
stopwatch.tick("OidcTokenAuthenticationMechanism maybeUser loaded");
if (maybeUser.isPresent()) {
return AuthResult.createUser(maybeUser.get());
}
logger.atInfo().log("No end user found for email address %s", email);
if (serviceAccountEmails.stream().anyMatch(e -> e.equals(email))) {
return AuthResult.createApp(email);
}
logger.atInfo().log("No service account found for email address %s", email);
logger.atWarning().log(
"The email address %s is not tied to a principal with access to Nomulus", email);
return AuthResult.NOT_AUTHENTICATED;
@@ -47,6 +47,11 @@ final class CreateCdnsTld extends ConfirmingCommand {
)
String name;
@Parameter(
names = "--skip_sandbox_tld_check",
description = "In Sandbox, skip the dns_name format check.")
boolean skipSandboxTldCheck;
@Inject
@Config("projectId")
String projectId;
@@ -61,10 +66,15 @@ final class CreateCdnsTld extends ConfirmingCommand {
protected void init() {
// Sandbox talks to production Cloud DNS. As a result, we can't configure any domains with a
// suffix that might be used by customers on the same nameserver set. Limit the user to setting
// up *.test TLDs.
if (RegistryToolEnvironment.get() == RegistryToolEnvironment.SANDBOX
&& !dnsName.endsWith(".test.")) {
throw new IllegalArgumentException("Sandbox TLDs must be of the form \"*.test.\"");
// up *.test TLDs unless the user declares that the name is approved.
//
// The name format check simply provides a user-friendly error message. If the user wrongly
// declares name approval, the request to the Cloud DNS API will still fail.
if (RegistryToolEnvironment.get() == RegistryToolEnvironment.SANDBOX) {
if (!skipSandboxTldCheck && !dnsName.endsWith(".test.")) {
throw new IllegalArgumentException(
"Sandbox TLDs must be approved or in the form \"*.test.\"");
}
}
managedZone =
@@ -15,6 +15,7 @@
package google.registry.tools;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static com.google.common.collect.Sets.difference;
import static google.registry.model.billing.BillingBase.RenewalPriceBehavior.DEFAULT;
@@ -46,6 +47,9 @@ import google.registry.model.domain.token.AllocationToken;
import google.registry.model.domain.token.AllocationToken.RegistrationBehavior;
import google.registry.model.domain.token.AllocationToken.TokenStatus;
import google.registry.model.domain.token.AllocationToken.TokenType;
import google.registry.model.registrar.Registrar;
import google.registry.model.tld.Tld;
import google.registry.model.tld.Tlds;
import google.registry.persistence.VKey;
import google.registry.tools.params.MoneyParameter;
import google.registry.tools.params.TransitionListParameter.TokenStatusTransitions;
@@ -291,10 +295,12 @@ class GenerateAllocationTokensCommand implements Command {
!ImmutableList.of("").equals(allowedClientIds),
"Either omit --allowed_client_ids if all registrars are allowed, or include a"
+ " comma-separated list");
verifyAllRegistrarIdsExist(allowedClientIds);
checkArgument(
!ImmutableList.of("").equals(allowedTlds),
"Either omit --allowed_tlds if all TLDs are allowed, or include a comma-separated list");
verifyAllTldsExist(allowedTlds);
if (ImmutableList.of("").equals(allowedEppActions)) {
allowedEppActions = ImmutableList.of();
@@ -326,6 +332,34 @@ class GenerateAllocationTokensCommand implements Command {
}
}
static void verifyAllRegistrarIdsExist(@Nullable List<String> allowedClientIds) {
// a null/empty list means that all registrars are allowed
if (isNullOrEmpty(allowedClientIds)) {
return;
}
ImmutableSet<String> allRegistrarIds =
Registrar.loadAllKeysCached().stream()
.map(VKey::getKey)
.map(Object::toString)
.collect(toImmutableSet());
ImmutableList<String> badRegistrarIds =
allowedClientIds.stream()
.filter(id -> !allRegistrarIds.contains(id))
.collect(toImmutableList());
checkArgument(badRegistrarIds.isEmpty(), "Unknown registrar ID(s) %s", badRegistrarIds);
}
static void verifyAllTldsExist(@Nullable List<String> allowedTlds) {
// a null/empty list means that all TLDs are allowed
if (isNullOrEmpty(allowedTlds)) {
return;
}
ImmutableSet<String> allTlds = Tlds.getTldsOfType(Tld.TldType.REAL);
ImmutableList<String> badTlds =
allowedTlds.stream().filter(tld -> !allTlds.contains(tld)).collect(toImmutableList());
checkArgument(badTlds.isEmpty(), "Unknown REAL TLD(s) %s", badTlds);
}
private void verifyTokenStringsDoNotExist() {
ImmutableSet<String> existingTokenStrings =
getExistingTokenStrings(ImmutableSet.copyOf(tokenStrings));
@@ -157,6 +157,9 @@ final class UpdateAllocationTokensCommand extends UpdateOrDeleteAllocationTokens
endToken = true;
}
GenerateAllocationTokensCommand.verifyAllRegistrarIdsExist(allowedClientIds);
GenerateAllocationTokensCommand.verifyAllTldsExist(allowedTlds);
tokensToSave =
tm().transact(
() ->
@@ -15,15 +15,19 @@
package google.registry.ui.server.console;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.request.Action.Method.GET;
import static jakarta.servlet.http.HttpServletResponse.SC_BAD_REQUEST;
import static jakarta.servlet.http.HttpServletResponse.SC_OK;
import com.google.common.base.Strings;
import google.registry.config.RegistryConfig.Config;
import google.registry.model.console.ConsolePermission;
import google.registry.model.console.ConsoleUpdateHistory;
import google.registry.model.console.GlobalRole;
import google.registry.model.console.User;
import google.registry.model.console.UserRoles;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Action.GkeService;
@@ -43,8 +47,8 @@ public class ConsoleHistoryDataAction extends ConsoleApiAction {
private static final String SQL_USER_HISTORY =
"""
SELECT * FROM "ConsoleUpdateHistory"
WHERE acting_user = :actingUser
SELECT * FROM "ConsoleUpdateHistory"
WHERE acting_user = :actingUser
""";
private static final String SQL_REGISTRAR_HISTORY =
@@ -59,14 +63,18 @@ public class ConsoleHistoryDataAction extends ConsoleApiAction {
private final String registrarId;
private final Optional<String> consoleUserEmail;
private final String supportEmail;
@Inject
public ConsoleHistoryDataAction(
ConsoleApiParams consoleApiParams,
@Config("supportEmail") String supportEmail,
@Parameter("registrarId") String registrarId,
@Parameter("consoleUserEmail") Optional<String> consoleUserEmail) {
super(consoleApiParams);
this.registrarId = registrarId;
this.consoleUserEmail = consoleUserEmail;
this.supportEmail = supportEmail;
}
@Override
@@ -95,7 +103,9 @@ public class ConsoleHistoryDataAction extends ConsoleApiAction {
.setHint("org.hibernate.fetchSize", 1000)
.getResultList());
consoleApiParams.response().setPayload(consoleApiParams.gson().toJson(queryResult));
List<ConsoleUpdateHistory> formattedHistoryList =
replaceActiveUserIfNecessary(queryResult, user);
consoleApiParams.response().setPayload(consoleApiParams.gson().toJson(formattedHistoryList));
consoleApiParams.response().setStatus(SC_OK);
}
@@ -110,7 +120,39 @@ public class ConsoleHistoryDataAction extends ConsoleApiAction {
.setParameter("registrarId", registrarId)
.setHint("org.hibernate.fetchSize", 1000)
.getResultList());
consoleApiParams.response().setPayload(consoleApiParams.gson().toJson(queryResult));
List<ConsoleUpdateHistory> formattedHistoryList =
replaceActiveUserIfNecessary(queryResult, user);
consoleApiParams.response().setPayload(consoleApiParams.gson().toJson(formattedHistoryList));
consoleApiParams.response().setStatus(SC_OK);
}
/** Anonymizes support users in history logs if the user viewing the log is a registrar user. */
private List<ConsoleUpdateHistory> replaceActiveUserIfNecessary(
List<ConsoleUpdateHistory> historyList, User requestingUser) {
// Check if the user *viewing* the history is a registrar user (not a support user)
if (GlobalRole.NONE.equals(requestingUser.getUserRoles().getGlobalRole())) {
User genericSupportUser = // Fixed typo
new User.Builder()
.setEmailAddress(this.supportEmail)
.setUserRoles(new UserRoles.Builder().build()) // Simplified roles
.build();
return historyList.stream()
.map(
history -> {
// Check if the user who performed the action was a support user
if (!GlobalRole.NONE.equals(
history.getActingUser().getUserRoles().getGlobalRole())) {
return history.asBuilder().setActingUser(genericSupportUser).build();
}
// If acting user was a registrar user show them as-is
return history;
})
.collect(toImmutableList());
}
// If the viewing user is a support user, return the list unmodified
return historyList;
}
}
@@ -23,6 +23,7 @@ import static org.apache.http.HttpStatus.SC_OK;
import com.google.common.base.Strings;
import com.google.common.collect.ImmutableSet;
import com.google.common.flogger.FluentLogger;
import google.registry.model.console.ConsolePermission;
import google.registry.model.console.ConsoleUpdateHistory;
import google.registry.model.console.User;
@@ -47,6 +48,8 @@ import org.joda.time.DateTime;
method = {POST},
auth = Auth.AUTH_PUBLIC_LOGGED_IN)
public class ConsoleUpdateRegistrarAction extends ConsoleApiAction {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private static final String CHANGE_LOG_ENTRY = "%s updated on %s, old -> %s, new -> %s";
static final String PATH = "/console-api/registrar";
private final Optional<Registrar> registrar;
@@ -124,6 +127,9 @@ public class ConsoleUpdateRegistrarAction extends ConsoleApiAction {
new ConsoleUpdateHistory.Builder()
.setType(ConsoleUpdateHistory.Type.REGISTRAR_UPDATE)
.setDescription(updatedRegistrar.getRegistrarId()));
logConsoleChangesIfNecessary(updatedRegistrar, existingRegistrar.get());
sendExternalUpdatesIfNecessary(
EmailInfo.create(
existingRegistrar.get(),
@@ -134,4 +140,25 @@ public class ConsoleUpdateRegistrarAction extends ConsoleApiAction {
consoleApiParams.response().setStatus(SC_OK);
}
private void logConsoleChangesIfNecessary(
Registrar updatedRegistrar, Registrar existingRegistrar) {
if (!updatedRegistrar.getAllowedTlds().containsAll(existingRegistrar.getAllowedTlds())) {
logger.atInfo().log(
CHANGE_LOG_ENTRY,
"Allowed TLDs",
updatedRegistrar.getRegistrarId(),
existingRegistrar.getAllowedTlds(),
updatedRegistrar.getAllowedTlds());
}
if (updatedRegistrar.isRegistryLockAllowed() != existingRegistrar.isRegistryLockAllowed()) {
logger.atInfo().log(
CHANGE_LOG_ENTRY,
"Registry lock",
updatedRegistrar.getRegistrarId(),
existingRegistrar.isRegistryLockAllowed(),
updatedRegistrar.isRegistryLockAllowed());
}
}
}
@@ -0,0 +1,22 @@
<epp xmlns="urn:ietf:params:xml:ns:epp-1.0">
<command>
<update>
<domain:update xmlns:domain="urn:ietf:params:xml:ns:domain-1.0">
<domain:name>%DOMAIN%</domain:name>
<domain:rem>
%CONTACTS%
</domain:rem>
<domain:chg>
<domain:registrant/>
</domain:chg>
</domain:update>
</update>
<extension>
<metadata:metadata xmlns:metadata="urn:google:params:xml:ns:metadata-1.0">
<metadata:reason>Registry minimum data set phase 3: Removed all contacts from domain.</metadata:reason>
<metadata:requestedByRegistrar>false</metadata:requestedByRegistrar>
</metadata:metadata>
</extension>
<clTRID>ABC-12345</clTRID>
</command>
</epp>
@@ -0,0 +1,119 @@
// Copyright 2025 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.batch;
import static com.google.common.truth.Truth.assertThat;
import static google.registry.model.common.FeatureFlag.FeatureName.MINIMUM_DATASET_CONTACTS_PROHIBITED;
import static google.registry.model.common.FeatureFlag.FeatureStatus.ACTIVE;
import static google.registry.testing.DatabaseHelper.createTld;
import static google.registry.testing.DatabaseHelper.loadByEntity;
import static google.registry.testing.DatabaseHelper.newDomain;
import static google.registry.testing.DatabaseHelper.persistActiveContact;
import static google.registry.testing.DatabaseHelper.persistResource;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import static org.mockito.Mockito.mock;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedMap;
import com.google.common.util.concurrent.RateLimiter;
import google.registry.flows.DaggerEppTestComponent;
import google.registry.flows.EppController;
import google.registry.flows.EppTestComponent.FakesAndMocksModule;
import google.registry.model.common.FeatureFlag;
import google.registry.model.contact.Contact;
import google.registry.model.domain.Domain;
import google.registry.persistence.transaction.JpaTestExtensions;
import google.registry.persistence.transaction.JpaTestExtensions.JpaIntegrationTestExtension;
import google.registry.testing.FakeClock;
import google.registry.testing.FakeLockHandler;
import google.registry.testing.FakeResponse;
import java.util.Optional;
import org.joda.time.DateTime;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;
/** Unit tests for {@link RemoveAllDomainContactsAction}. */
class RemoveAllDomainContactsActionTest {
@RegisterExtension
final JpaIntegrationTestExtension jpa =
new JpaTestExtensions.Builder().buildIntegrationTestExtension();
private final FakeResponse response = new FakeResponse();
private final RateLimiter rateLimiter = mock(RateLimiter.class);
private RemoveAllDomainContactsAction action;
@BeforeEach
void beforeEach() {
createTld("tld");
persistResource(
new FeatureFlag.Builder()
.setFeatureName(MINIMUM_DATASET_CONTACTS_PROHIBITED)
.setStatusMap(ImmutableSortedMap.of(START_OF_TIME, ACTIVE))
.build());
EppController eppController =
DaggerEppTestComponent.builder()
.fakesAndMocksModule(FakesAndMocksModule.create(new FakeClock()))
.build()
.startRequest()
.eppController();
action =
new RemoveAllDomainContactsAction(
eppController, "NewRegistrar", new FakeLockHandler(true), rateLimiter, response);
}
@Test
void test_removesAllContactsFromMultipleDomains_andDoesntModifyDomainThatHasNoContacts() {
Contact c1 = persistActiveContact("contact12345");
Domain d1 = persistResource(newDomain("foo.tld", c1));
assertThat(d1.getAllContacts()).hasSize(3);
Contact c2 = persistActiveContact("contact23456");
Domain d2 = persistResource(newDomain("bar.tld", c2));
assertThat(d2.getAllContacts()).hasSize(3);
Domain d3 =
persistResource(
newDomain("baz.tld")
.asBuilder()
.setRegistrant(Optional.empty())
.setContacts(ImmutableSet.of())
.build());
assertThat(d3.getAllContacts()).isEmpty();
DateTime lastUpdate = d3.getUpdateTimestamp().getTimestamp();
action.run();
assertThat(loadByEntity(d1).getAllContacts()).isEmpty();
assertThat(loadByEntity(d2).getAllContacts()).isEmpty();
assertThat(loadByEntity(d3).getUpdateTimestamp().getTimestamp()).isEqualTo(lastUpdate);
}
@Test
void test_removesContacts_onDomainsThatOnlyPartiallyHaveContacts() {
Contact c1 = persistActiveContact("contact12345");
Domain d1 =
persistResource(
newDomain("foo.tld", c1).asBuilder().setContacts(ImmutableSet.of()).build());
assertThat(d1.getAllContacts()).hasSize(1);
Contact c2 = persistActiveContact("contact23456");
Domain d2 =
persistResource(
newDomain("bar.tld", c2).asBuilder().setRegistrant(Optional.empty()).build());
assertThat(d2.getAllContacts()).hasSize(2);
action.run();
assertThat(loadByEntity(d1).getAllContacts()).isEmpty();
assertThat(loadByEntity(d2).getAllContacts()).isEmpty();
}
}
@@ -25,6 +25,7 @@ import google.registry.config.RegistryConfig.ConfigModule;
import google.registry.config.RegistryConfig.ConfigModule.TmchCaMode;
import google.registry.flows.custom.CustomLogicFactory;
import google.registry.flows.custom.TestCustomLogicFactory;
import google.registry.flows.domain.DomainDeletionTimeCache;
import google.registry.flows.domain.DomainFlowTmchUtils;
import google.registry.monitoring.whitebox.EppMetric;
import google.registry.request.RequestScope;
@@ -126,6 +127,11 @@ public interface EppTestComponent {
ServerTridProvider provideServerTridProvider() {
return new FakeServerTridProvider();
}
@Provides
DomainDeletionTimeCache provideDomainDeletionTimeCache() {
return DomainDeletionTimeCache.create();
}
}
class FakeServerTridProvider implements ServerTridProvider {
@@ -40,8 +40,9 @@ public class TestDomainCreateFlowCustomLogic extends DomainCreateFlowCustomLogic
.setMsg("Custom logic was triggered")
.build();
return EntityChanges.newBuilder()
.setSaves(parameters.entityChanges().getSaves())
.addSave(extraPollMessage)
.setInserts(parameters.entityChanges().getInserts())
.addInsert(extraPollMessage)
.setUpdates(parameters.entityChanges().getUpdates())
.setDeletes(parameters.entityChanges().getDeletes())
.build();
}
@@ -149,7 +149,6 @@ import google.registry.flows.domain.token.AllocationTokenFlowUtils.AlreadyRedeem
import google.registry.flows.domain.token.AllocationTokenFlowUtils.NonexistentAllocationTokenException;
import google.registry.flows.exceptions.ContactsProhibitedException;
import google.registry.flows.exceptions.OnlyToolCanPassMetadataException;
import google.registry.flows.exceptions.ResourceAlreadyExistsForThisClientException;
import google.registry.flows.exceptions.ResourceCreateContentionException;
import google.registry.model.billing.BillingBase;
import google.registry.model.billing.BillingBase.Flag;
@@ -1238,8 +1237,8 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
void testFailure_alreadyExists() throws Exception {
persistContactsAndHosts();
persistActiveDomain(getUniqueIdFromCommand());
ResourceAlreadyExistsForThisClientException thrown =
assertThrows(ResourceAlreadyExistsForThisClientException.class, this::runFlow);
ResourceCreateContentionException thrown =
assertThrows(ResourceCreateContentionException.class, this::runFlow);
assertAboutEppExceptions()
.that(thrown)
.marshalsToXml()
@@ -0,0 +1,97 @@
// Copyright 2025 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.flows.domain;
import static com.google.common.truth.Truth.assertThat;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.testing.DatabaseHelper.persistActiveDomain;
import static google.registry.testing.DatabaseHelper.persistDomainAsDeleted;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import google.registry.model.domain.Domain;
import google.registry.persistence.transaction.JpaTestExtensions;
import google.registry.testing.DatabaseHelper;
import google.registry.testing.FakeClock;
import java.util.Optional;
import org.joda.time.DateTime;
import org.joda.time.Duration;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;
/** Tests for {@link DomainDeletionTimeCache}. */
public class DomainDeletionTimeCacheTest {
private final FakeClock clock = new FakeClock(DateTime.parse("2025-10-01T00:00:00.000Z"));
private final DomainDeletionTimeCache cache = DomainDeletionTimeCache.create();
@RegisterExtension
final JpaTestExtensions.JpaIntegrationTestExtension jpa =
new JpaTestExtensions.Builder().withClock(clock).buildIntegrationTestExtension();
@BeforeEach
void beforeEach() {
DatabaseHelper.createTld("tld");
}
@Test
void testDomainAvailable_null() {
assertThat(getDeletionTimeFromCache("nonexistent.tld")).isEmpty();
}
@Test
void testDomainNotAvailable_notDeleted() {
persistActiveDomain("active.tld");
assertThat(getDeletionTimeFromCache("active.tld")).hasValue(END_OF_TIME);
}
@Test
void testDomainAvailable_deletedInFuture() {
persistDomainAsDeleted(persistActiveDomain("domain.tld"), clock.nowUtc().plusDays(1));
assertThat(getDeletionTimeFromCache("domain.tld")).hasValue(clock.nowUtc().plusDays(1));
}
@Test
void testCache_returnsOldData() {
Domain domain = persistActiveDomain("domain.tld");
assertThat(getDeletionTimeFromCache("domain.tld")).hasValue(END_OF_TIME);
persistDomainAsDeleted(domain, clock.nowUtc().plusDays(1));
// Without intervention, the cache should have the old data, even if a few minutes have passed
clock.advanceBy(Duration.standardMinutes(5));
assertThat(getDeletionTimeFromCache("domain.tld")).hasValue(END_OF_TIME);
}
@Test
void testCache_returnsNewDataAfterDomainCreate() {
// Null deletion dates (meaning an avilable domain) shouldn't be cached
assertThat(getDeletionTimeFromCache("domain.tld")).isEmpty();
persistDomainAsDeleted(persistActiveDomain("domain.tld"), clock.nowUtc().plusDays(1));
assertThat(getDeletionTimeFromCache("domain.tld")).hasValue(clock.nowUtc().plusDays(1));
}
@Test
void testCache_expires() {
Domain domain = persistActiveDomain("domain.tld");
assertThat(getDeletionTimeFromCache("domain.tld")).hasValue(END_OF_TIME);
DateTime elevenMinutesFromNow = clock.nowUtc().plusMinutes(11);
persistDomainAsDeleted(domain, elevenMinutesFromNow);
clock.advanceBy(Duration.standardMinutes(30));
assertThat(getDeletionTimeFromCache("domain.tld")).hasValue(elevenMinutesFromNow);
}
private Optional<DateTime> getDeletionTimeFromCache(String domainName) {
return tm().transact(() -> cache.getDeletionTimeForDomain(domainName));
}
}
@@ -20,6 +20,7 @@ import google.registry.config.CredentialModule;
import google.registry.config.RegistryConfig;
import google.registry.flows.ServerTridProviderModule;
import google.registry.flows.custom.CustomLogicFactoryModule;
import google.registry.flows.domain.DomainDeletionTimeCacheModule;
import google.registry.groups.GmailModule;
import google.registry.groups.GroupsModule;
import google.registry.groups.GroupssettingsModule;
@@ -43,6 +44,7 @@ import jakarta.inject.Singleton;
CredentialModule.class,
CustomLogicFactoryModule.class,
CloudTasksUtilsModule.class,
DomainDeletionTimeCacheModule.class,
FrontendRequestComponent.FrontendRequestComponentModule.class,
GmailModule.class,
GroupsModule.class,
@@ -16,20 +16,13 @@ package google.registry.request.auth;
import static com.google.common.net.HttpHeaders.AUTHORIZATION;
import static com.google.common.truth.Truth.assertThat;
import static google.registry.config.RegistryConfig.getUserAuthCachingDuration;
import static google.registry.config.RegistryConfig.getUserAuthMaxCachedEntries;
import static google.registry.request.auth.AuthModule.BEARER_PREFIX;
import static google.registry.request.auth.AuthModule.IAP_HEADER_NAME;
import static google.registry.testing.DatabaseHelper.createAdminUser;
import static google.registry.testing.DatabaseHelper.persistResource;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.never;
import static org.mockito.Mockito.spy;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import com.github.benmanes.caffeine.cache.LoadingCache;
import com.google.api.client.googleapis.auth.oauth2.GoogleIdToken.Payload;
import com.google.api.client.json.webtoken.JsonWebSignature;
import com.google.api.client.json.webtoken.JsonWebSignature.Header;
@@ -40,9 +33,7 @@ import dagger.Component;
import dagger.Module;
import dagger.Provides;
import google.registry.config.CredentialModule.ApplicationDefaultCredential;
import google.registry.config.RegistryConfig;
import google.registry.config.RegistryConfig.Config;
import google.registry.model.CacheUtils;
import google.registry.model.console.GlobalRole;
import google.registry.model.console.User;
import google.registry.model.console.UserRoles;
@@ -53,7 +44,6 @@ import google.registry.request.auth.OidcTokenAuthenticationMechanism.RegularOidc
import google.registry.util.GoogleCredentialsBundle;
import jakarta.inject.Singleton;
import jakarta.servlet.http.HttpServletRequest;
import java.util.Optional;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
@@ -88,12 +78,6 @@ public class OidcTokenAuthenticationMechanismTest {
@BeforeEach
void beforeEach() throws Exception {
// 1. Create a brand new cache.
LoadingCache<String, Optional<User>> testCache =
CacheUtils.newCacheBuilder(getUserAuthCachingDuration())
.maximumSize(getUserAuthMaxCachedEntries())
.build(OidcTokenAuthenticationMechanism::loadUser);
OidcTokenAuthenticationMechanism.setCacheForTesting(testCache);
payload.setEmail(email);
payload.setSubject(gaiaId);
user = createAdminUser(email);
@@ -167,8 +151,7 @@ public class OidcTokenAuthenticationMechanismTest {
payload.setEmail("service@email.test");
authResult = authenticationMechanism.authenticate(request);
assertThat(authResult.isAuthenticated()).isTrue();
assertThat(authResult.authLevel()).isEqualTo(AuthLevel.USER);
assertThat(authResult.user().get()).isEqualTo(serviceUser);
assertThat(authResult.authLevel()).isEqualTo(AuthLevel.APP);
}
@Test
@@ -208,62 +191,6 @@ public class OidcTokenAuthenticationMechanismTest {
authenticationMechanism = component.regularOidcAuthenticationMechanism();
}
@Test
void testAuthenticate_ExistentUser_isCached() {
// Arrange: Create a spy of the actual cache object.
// A spy calls the real methods of the object while allowing us to verify interactions.
LoadingCache<String, Optional<User>> spiedCache =
spy(OidcTokenAuthenticationMechanism.userCache);
OidcTokenAuthenticationMechanism.setCacheForTesting(spiedCache);
// Act: Call the authenticate method.
authenticationMechanism.authenticate(request);
// Assert: Verify that the cache's "get" method was called exactly once.
// This confirms the cache is being used without checking its internal stats.
verify(spiedCache).get(email);
}
@Test
void testAuthenticate_nonExistentUser_isCached() {
// Arrange: Use an email that is not in the test database.
payload.setEmail(unknownEmail);
LoadingCache<String, Optional<User>> spiedCache =
spy(OidcTokenAuthenticationMechanism.userCache);
OidcTokenAuthenticationMechanism.setCacheForTesting(spiedCache);
// Act: Call the authenticate method.
authenticationMechanism.authenticate(request);
// Assert: Verify that the cache's "get" method was called for the unverified email.
// This confirms that we attempted to look up the unknown user in the cache.
verify(spiedCache).get(unknownEmail);
}
@Test
void testAuthenticate_whenCacheIsDisabled_cacheIsNotUsed() {
// Arrange: Explicitly disable the cache and create a spy.
RegistryConfig.overrideIsUserAuthCachingEnabledForTesting(false);
LoadingCache<String, Optional<User>> spiedCache =
spy(OidcTokenAuthenticationMechanism.userCache);
OidcTokenAuthenticationMechanism.setCacheForTesting(spiedCache);
// Act: Authenticate the user.
AuthResult authResult = authenticationMechanism.authenticate(request);
// Assert: The authentication should still succeed because the code falls back
// to the direct database call.
assertThat(authResult.isAuthenticated()).isTrue();
// Assert: Crucially, verify that the cache's "get" method was NEVER called.
// This proves the cache was correctly bypassed.
verify(spiedCache, never()).get(any(String.class));
// Teardown: Restore the default setting for other tests.
RegistryConfig.overrideIsUserAuthCachingEnabledForTesting(true);
}
@Singleton
@Component(modules = {AuthModule.class, TestModule.class})
interface TestComponent {
@@ -309,12 +236,4 @@ public class OidcTokenAuthenticationMechanismTest {
return GoogleCredentialsBundle.create(GoogleCredentials.newBuilder().build());
}
}
private void reinitializeCache() {
OidcTokenAuthenticationMechanism.userCache =
CacheUtils.newCacheBuilder(getUserAuthCachingDuration())
.maximumSize(getUserAuthMaxCachedEntries())
.recordStats()
.build(OidcTokenAuthenticationMechanism::loadUser);
}
}
@@ -76,11 +76,37 @@ class CreateCdnsTldTest extends CommandTestCase<CreateCdnsTld> {
@Test
@MockitoSettings(strictness = Strictness.LENIENT)
void testSandboxTldRestrictions() {
void testSandboxTldRestrictions_Disallowed() {
IllegalArgumentException thrown =
assertThrows(
IllegalArgumentException.class,
() -> runCommandInEnvironment(RegistryToolEnvironment.SANDBOX, "--dns_name=foobar."));
assertThat(thrown).hasMessageThat().contains("Sandbox TLDs must be of the form \"*.test.\"");
() ->
runCommandInEnvironment(
RegistryToolEnvironment.SANDBOX,
"--dns_name=foobar.",
"--description=test run",
"--force"));
assertThat(thrown)
.hasMessageThat()
.contains("Sandbox TLDs must be approved or in the form \"*.test.\"");
}
@Test
void testSandboxTldRestrictions_tldCheckSkipped() throws Exception {
runCommandInEnvironment(
RegistryToolEnvironment.SANDBOX,
"--dns_name=foobar.",
"--description=test run",
"--force",
"--skip_sandbox_tld_check");
}
@Test
void testSandboxTldRestrictions_testTld() throws Exception {
runCommandInEnvironment(
RegistryToolEnvironment.SANDBOX,
"--dns_name=abc.test.",
"--description=test run",
"--force");
}
}
@@ -39,6 +39,7 @@ import google.registry.model.domain.fee.FeeQueryCommandExtensionItem.CommandName
import google.registry.model.domain.token.AllocationToken;
import google.registry.model.domain.token.AllocationToken.TokenStatus;
import google.registry.model.reporting.HistoryEntry.HistoryEntryId;
import google.registry.testing.DatabaseHelper;
import google.registry.testing.DeterministicStringGenerator;
import google.registry.testing.DeterministicStringGenerator.Rule;
import google.registry.util.StringGenerator.Alphabets;
@@ -56,6 +57,7 @@ class GenerateAllocationTokensCommandTest extends CommandTestCase<GenerateAlloca
@BeforeEach
void beforeEach() {
DatabaseHelper.createTlds("tld", "example");
command.stringGenerator = new DeterministicStringGenerator(Alphabets.BASE_58);
}
@@ -531,6 +533,26 @@ class GenerateAllocationTokensCommandTest extends CommandTestCase<GenerateAlloca
.isEqualTo("For DEFAULT_PROMO tokens, must specify --token_status_transitions");
}
@Test
void testFailure_badTld() {
assertThat(
assertThrows(
IllegalArgumentException.class,
() -> runCommand("--number", "10", "--allowed_tlds", "badtld")))
.hasMessageThat()
.isEqualTo("Unknown REAL TLD(s) [badtld]");
}
@Test
void testFailure_badRegistrar() {
assertThat(
assertThrows(
IllegalArgumentException.class,
() -> runCommand("--number", "10", "--allowed_client_ids", "badregistrar")))
.hasMessageThat()
.isEqualTo("Unknown registrar ID(s) [badregistrar]");
}
private AllocationToken createToken(
String token,
@Nullable HistoryEntryId redemptionHistoryEntryId,
@@ -40,14 +40,21 @@ import google.registry.model.domain.fee.FeeQueryCommandExtensionItem.CommandName
import google.registry.model.domain.token.AllocationToken;
import google.registry.model.domain.token.AllocationToken.RegistrationBehavior;
import google.registry.model.domain.token.AllocationToken.TokenStatus;
import google.registry.testing.DatabaseHelper;
import org.joda.money.CurrencyUnit;
import org.joda.money.Money;
import org.joda.time.DateTime;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
/** Unit tests for {@link UpdateAllocationTokensCommand}. */
class UpdateAllocationTokensCommandTest extends CommandTestCase<UpdateAllocationTokensCommand> {
@BeforeEach
void beforeEach() {
DatabaseHelper.createTlds("tld", "example");
}
@Test
void testUpdateTlds_setTlds() throws Exception {
AllocationToken token =
@@ -64,14 +71,24 @@ class UpdateAllocationTokensCommandTest extends CommandTestCase<UpdateAllocation
assertThat(reloadResource(token).getAllowedTlds()).isEmpty();
}
@Test
void testUpdateTlds_badTlds() {
persistResource(builderWithPromo().build());
assertThat(
assertThrows(
IllegalArgumentException.class, () -> runCommandForced("--allowed_tlds=badtld")))
.hasMessageThat()
.isEqualTo("Unknown REAL TLD(s) [badtld]");
}
@Test
void testUpdateClientIds_setClientIds() throws Exception {
AllocationToken token =
persistResource(
builderWithPromo().setAllowedRegistrarIds(ImmutableSet.of("toRemove")).build());
runCommandForced("--prefix", "token", "--allowed_client_ids", "clientone,clienttwo");
runCommandForced("--prefix", "token", "--allowed_client_ids", "TheRegistrar,NewRegistrar");
assertThat(reloadResource(token).getAllowedRegistrarIds())
.containsExactly("clientone", "clienttwo");
.containsExactly("TheRegistrar", "NewRegistrar");
}
@Test
@@ -83,6 +100,17 @@ class UpdateAllocationTokensCommandTest extends CommandTestCase<UpdateAllocation
assertThat(reloadResource(token).getAllowedRegistrarIds()).isEmpty();
}
@Test
void testUpdateClientIds_badClientId() {
persistResource(builderWithPromo().build());
assertThat(
assertThrows(
IllegalArgumentException.class,
() -> runCommandForced("--allowed_client_ids=badregistrar")))
.hasMessageThat()
.isEqualTo("Unknown registrar ID(s) [badregistrar]");
}
@Test
void testUpdateEppActions_setEppActions() throws Exception {
AllocationToken token =
@@ -40,8 +40,10 @@ import org.junit.jupiter.api.Test;
class ConsoleHistoryDataActionTest extends ConsoleActionBaseTestCase {
private static final String SUPPORT_EMAIL = "supportEmailTest@test.com";
private static final Gson GSON = new Gson();
private User noPermissionUser;
private User registrarPrimaryUser;
@BeforeEach
void beforeEach() {
@@ -56,6 +58,17 @@ class ConsoleHistoryDataActionTest extends ConsoleActionBaseTestCase {
.build())
.build());
registrarPrimaryUser =
DatabaseHelper.persistResource(
new User.Builder()
.setEmailAddress("primary@example.com")
.setUserRoles(
new UserRoles.Builder()
.setRegistrarRoles(
ImmutableMap.of("TheRegistrar", RegistrarRole.PRIMARY_CONTACT))
.build())
.build());
DatabaseHelper.persistResources(
ImmutableList.of(
new ConsoleUpdateHistory.Builder()
@@ -85,7 +98,7 @@ class ConsoleHistoryDataActionTest extends ConsoleActionBaseTestCase {
}
@Test
void testSuccess_getByRegistrar() {
void testSuccess_getByRegistrar_notAnonymizedForSupportUser() {
ConsoleHistoryDataAction action =
createAction(AuthResult.createUser(fteUser), "TheRegistrar", Optional.empty());
action.run();
@@ -93,6 +106,35 @@ class ConsoleHistoryDataActionTest extends ConsoleActionBaseTestCase {
List<Map<String, Object>> payload = GSON.fromJson(response.getPayload(), List.class);
assertThat(payload.stream().map(record -> record.get("description")).collect(toImmutableList()))
.containsExactly("TheRegistrar|Some change", "TheRegistrar|Another change");
// Assert that the support user sees the real acting users
List<String> actingUserEmails =
payload.stream()
.map(record -> (Map<String, Object>) record.get("actingUser"))
.map(userMap -> (String) userMap.get("emailAddress"))
.collect(toImmutableList());
assertThat(actingUserEmails).containsExactly("fte@email.tld", "no.perms@example.com");
}
@Test
void testSuccess_getByRegistrar_anonymizedForRegistrarUser() {
ConsoleHistoryDataAction action =
createAction(AuthResult.createUser(registrarPrimaryUser), "TheRegistrar", Optional.empty());
action.run();
assertThat(response.getStatus()).isEqualTo(SC_OK);
List<Map<String, Object>> payload = GSON.fromJson(response.getPayload(), List.class);
assertThat(payload.stream().map(record -> record.get("description")).collect(toImmutableList()))
.containsExactly("TheRegistrar|Some change", "TheRegistrar|Another change");
// Assert that the registrar user sees the anonymized support user
List<String> actingUserEmails =
payload.stream()
.map(record -> (Map<String, Object>) record.get("actingUser"))
.map(userMap -> (String) userMap.get("emailAddress"))
.collect(toImmutableList());
assertThat(actingUserEmails).containsExactly(SUPPORT_EMAIL, "no.perms@example.com");
}
@Test
@@ -104,6 +146,13 @@ class ConsoleHistoryDataActionTest extends ConsoleActionBaseTestCase {
List<Map<String, Object>> payload = GSON.fromJson(response.getPayload(), List.class);
assertThat(payload.stream().map(record -> record.get("description")).collect(toImmutableList()))
.containsExactly("TheRegistrar|Some change", "OtherRegistrar|Some change");
List<String> actingUserEmails =
payload.stream()
.map(record -> (Map<String, Object>) record.get("actingUser"))
.map(userMap -> (String) userMap.get("emailAddress"))
.collect(toImmutableList());
assertThat(actingUserEmails).containsExactly("fte@email.tld", "fte@email.tld");
}
@Test
@@ -148,6 +197,7 @@ class ConsoleHistoryDataActionTest extends ConsoleActionBaseTestCase {
consoleApiParams = ConsoleApiParamsUtils.createFake(authResult);
when(consoleApiParams.request().getMethod()).thenReturn("GET");
response = (FakeResponse) consoleApiParams.response();
return new ConsoleHistoryDataAction(consoleApiParams, registrarId, consoleUserEmail);
return new ConsoleHistoryDataAction(
consoleApiParams, SUPPORT_EMAIL, registrarId, consoleUserEmail);
}
}
@@ -26,6 +26,7 @@ BACKEND /_dr/task/rdeUpload RdeUploadAction
BACKEND /_dr/task/readDnsRefreshRequests ReadDnsRefreshRequestsAction POST y APP ADMIN
BACKEND /_dr/task/refreshDnsOnHostRename RefreshDnsOnHostRenameAction POST n APP ADMIN
BACKEND /_dr/task/relockDomain RelockDomainAction POST y APP ADMIN
BACKEND /_dr/task/removeAllDomainContacts RemoveAllDomainContactsAction POST n APP ADMIN
BACKEND /_dr/task/resaveAllEppResourcesPipeline ResaveAllEppResourcesPipelineAction GET n APP ADMIN
BACKEND /_dr/task/resaveEntity ResaveEntityAction POST n APP ADMIN
BACKEND /_dr/task/sendExpiringCertificateNotificationEmail SendExpiringCertificateNotificationEmailAction GET n APP ADMIN
@@ -44,6 +44,7 @@ BACKEND /_dr/task/readDnsRefreshRequests ReadDnsRefreshReques
BACKEND /_dr/task/refreshDnsForAllDomains RefreshDnsForAllDomainsAction GET n APP ADMIN
BACKEND /_dr/task/refreshDnsOnHostRename RefreshDnsOnHostRenameAction POST n APP ADMIN
BACKEND /_dr/task/relockDomain RelockDomainAction POST y APP ADMIN
BACKEND /_dr/task/removeAllDomainContacts RemoveAllDomainContactsAction POST n APP ADMIN
BACKEND /_dr/task/resaveAllEppResourcesPipeline ResaveAllEppResourcesPipelineAction GET n APP ADMIN
BACKEND /_dr/task/resaveEntity ResaveEntityAction POST n APP ADMIN
BACKEND /_dr/task/sendExpiringCertificateNotificationEmail SendExpiringCertificateNotificationEmailAction GET n APP ADMIN
+1 -1
View File
@@ -260,7 +260,7 @@ if (project.baseSchemaTag != '') {
// Checks if Flyway scripts can be deployed to an existing database with
// an older release. Please refer to SchemaTest.java for more information.
task schemaIncrementalDeployTest(dependsOn: processResources, type: Test) {
task schemaIncrementalDeployTest(dependsOn: processTestResources, type: Test) {
useJUnitPlatform()
include 'google/registry/sql/flyway/SchemaTest.*'
classpath = configurations.testRuntimeClasspath
@@ -261,11 +261,11 @@ td.section {
</tr>
<tr>
<td class="property_name">generated on</td>
<td class="property_value">2025-09-05 16:11:29</td>
<td class="property_value">2025-10-10 17:24:50</td>
</tr>
<tr>
<td class="property_name">last flyway file</td>
<td id="lastFlywayFile" class="property_value">V197__poc_rlock_drop_not_null.sql</td>
<td id="lastFlywayFile" class="property_value">V213__graceperiodhistory_history_revision_id_hash.sql</td>
</tr>
</tbody>
</table>
@@ -273,7 +273,7 @@ td.section {
<p>&nbsp;</p>
<svg viewBox="0.00 0.00 4903.00 3732.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" id="erDiagram" style="overflow: hidden; width: 100%; height: 800px">
<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 3728)">
<title>SchemaCrawler_Diagram</title> <polygon fill="white" stroke="transparent" points="-4,4 -4,-3728 4899,-3728 4899,4 -4,4" /> <text text-anchor="start" x="4655" y="-29.8" font-family="Helvetica,sans-Serif" font-size="14.00">generated by</text> <text text-anchor="start" x="4738" y="-29.8" font-family="Helvetica,sans-Serif" font-size="14.00">SchemaCrawler 16.27.1</text> <text text-anchor="start" x="4654" y="-10.8" font-family="Helvetica,sans-Serif" font-size="14.00">generated on</text> <text text-anchor="start" x="4738" y="-10.8" font-family="Helvetica,sans-Serif" font-size="14.00">2025-09-05 16:11:29</text> <polygon fill="none" stroke="#888888" points="4651,-4 4651,-44 4887,-44 4887,-4 4651,-4" /> <!-- allocationtoken_a08ccbef -->
<title>SchemaCrawler_Diagram</title> <polygon fill="white" stroke="transparent" points="-4,4 -4,-3728 4899,-3728 4899,4 -4,4" /> <text text-anchor="start" x="4655" y="-29.8" font-family="Helvetica,sans-Serif" font-size="14.00">generated by</text> <text text-anchor="start" x="4738" y="-29.8" font-family="Helvetica,sans-Serif" font-size="14.00">SchemaCrawler 16.27.1</text> <text text-anchor="start" x="4654" y="-10.8" font-family="Helvetica,sans-Serif" font-size="14.00">generated on</text> <text text-anchor="start" x="4738" y="-10.8" font-family="Helvetica,sans-Serif" font-size="14.00">2025-10-10 17:24:50</text> <polygon fill="none" stroke="#888888" points="4651,-4 4651,-44 4887,-44 4887,-4 4651,-4" /> <!-- allocationtoken_a08ccbef -->
<g id="node1" class="node">
<title>allocationtoken_a08ccbef</title> <polygon fill="#e9c2f2" stroke="transparent" points="481.5,-978 481.5,-997 667.5,-997 667.5,-978 481.5,-978" /> <text text-anchor="start" x="483.5" y="-984.8" font-family="Helvetica,sans-Serif" font-weight="bold" font-style="italic" font-size="14.00">public."AllocationToken"</text> <polygon fill="#e9c2f2" stroke="transparent" points="667.5,-978 667.5,-997 741.5,-997 741.5,-978 667.5,-978" /> <text text-anchor="start" x="702.5" y="-983.8" font-family="Helvetica,sans-Serif" font-size="14.00">[table]</text> <text text-anchor="start" x="483.5" y="-965.8" font-family="Helvetica,sans-Serif" font-weight="bold" font-style="italic" font-size="14.00">token</text> <text text-anchor="start" x="661.5" y="-964.8" font-family="Helvetica,sans-Serif" font-size="14.00"> </text> <text text-anchor="start" x="669.5" y="-964.8" font-family="Helvetica,sans-Serif" font-size="14.00">text not null</text> <text text-anchor="start" x="483.5" y="-945.8" font-family="Helvetica,sans-Serif" font-size="14.00">domain_name</text> <text text-anchor="start" x="661.5" y="-945.8" font-family="Helvetica,sans-Serif" font-size="14.00"> </text> <text text-anchor="start" x="669.5" y="-945.8" font-family="Helvetica,sans-Serif" font-size="14.00">text</text> <text text-anchor="start" x="483.5" y="-926.8" font-family="Helvetica,sans-Serif" font-size="14.00">redemption_domain_repo_id</text> <text text-anchor="start" x="661.5" y="-926.8" font-family="Helvetica,sans-Serif" font-size="14.00"> </text> <text text-anchor="start" x="669.5" y="-926.8" font-family="Helvetica,sans-Serif" font-size="14.00">text</text> <text text-anchor="start" x="483.5" y="-907.8" font-family="Helvetica,sans-Serif" font-size="14.00">token_type</text> <text text-anchor="start" x="661.5" y="-907.8" font-family="Helvetica,sans-Serif" font-size="14.00"> </text> <text text-anchor="start" x="669.5" y="-907.8" font-family="Helvetica,sans-Serif" font-size="14.00">text</text> <polygon fill="none" stroke="#888888" points="480.5,-901.5 480.5,-998.5 742.5,-998.5 742.5,-901.5 480.5,-901.5" />
</g>
File diff suppressed because one or more lines are too long
+16
View File
@@ -195,3 +195,19 @@ V194__password_reset_request_registrar.sql
V195__registrar_poc_id.sql
V196__tld_expiry_access_period_enabled.sql
V197__poc_rlock_drop_not_null.sql
V198__billing_cancellation_hash.sql
V199__billing_event_hash.sql
V200__billing_recurrence_hash.sql
V201__domain_hash.sql
V202__delegation_signer_data_hash.sql
V203__domain_history_hash.sql
V204__domain_host_hash.sql
V205__domain_transaction_record_hash.sql
V206__grace_period_hash.sql
V207__grace_period_history_hash.sql
V208__host_hash.sql
V209__poll_message_hash.sql
V210__allocationtoken_token_hash.sql
V211__domainhistoryhost_history_revision_id_hash.sql
V212__domaindsdatahistory_history_revision_id_hash.sql
V213__graceperiodhistory_history_revision_id_hash.sql
@@ -0,0 +1,16 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- Add hash indexes on columns that are commonly queried with a direct equals
CREATE INDEX CONCURRENTLY IF NOT EXISTS billingcancellation_billing_cancellation_id_hash ON "BillingCancellation" USING hash (billing_cancellation_id);
@@ -0,0 +1,16 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- Add hash indexes on columns that are commonly queried with a direct equals
CREATE INDEX CONCURRENTLY IF NOT EXISTS billingevent_billing_event_id_hash ON "BillingEvent" USING hash (billing_event_id);
@@ -0,0 +1,16 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- Add hash indexes on columns that are commonly queried with a direct equals
CREATE INDEX CONCURRENTLY IF NOT EXISTS billingrecurrence_billing_recurrence_id_hash ON "BillingRecurrence" USING hash (billing_recurrence_id);
@@ -0,0 +1,17 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- Add hash indexes on columns that are commonly queried with a direct equals
CREATE INDEX CONCURRENTLY IF NOT EXISTS domain_domain_name_hash ON "Domain" USING hash (domain_name);
CREATE INDEX CONCURRENTLY IF NOT EXISTS domain_domain_repo_id_hash ON "Domain" USING hash (repo_id);
@@ -0,0 +1,16 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- Add hash indexes on columns that are commonly queried with a direct equals
CREATE INDEX CONCURRENTLY IF NOT EXISTS delegationsignerdata_domain_repo_id_hash ON "DelegationSignerData" USING hash (domain_repo_id);
@@ -0,0 +1,17 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- Add hash indexes on columns that are commonly queried with a direct equals
CREATE INDEX CONCURRENTLY IF NOT EXISTS domainhistory_domain_repo_id_hash ON "DomainHistory" USING hash (domain_repo_id);
CREATE INDEX CONCURRENTLY IF NOT EXISTS domainhistory_history_revision_id_hash ON "DomainHistory" USING hash (history_revision_id);
@@ -0,0 +1,16 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- Add hash indexes on columns that are commonly queried with a direct equals
CREATE INDEX CONCURRENTLY IF NOT EXISTS domainhost_domain_repo_id_hash ON "DomainHost" USING hash (domain_repo_id);
@@ -0,0 +1,17 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- Add hash indexes on columns that are commonly queried with a direct equals
CREATE INDEX CONCURRENTLY IF NOT EXISTS domaintransactionrecord_id_hash ON "DomainTransactionRecord" USING hash (id);
CREATE INDEX CONCURRENTLY IF NOT EXISTS domaintransactionrecord_domain_history_revision_id_hash ON "DomainTransactionRecord" USING hash (history_revision_id);
@@ -0,0 +1,17 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- Add hash indexes on columns that are commonly queried with a direct equals
CREATE INDEX CONCURRENTLY IF NOT EXISTS graceperiod_grace_period_id_hash ON "GracePeriod" USING hash (grace_period_id);
CREATE INDEX CONCURRENTLY IF NOT EXISTS graceperiod_domain_repo_id_hash ON "GracePeriod" USING hash (domain_repo_id);
@@ -0,0 +1,16 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- Add hash indexes on columns that are commonly queried with a direct equals
CREATE INDEX CONCURRENTLY IF NOT EXISTS graceperiodhistory_grace_period_history_revision_id_hash ON "GracePeriodHistory" USING hash (grace_period_history_revision_id);
@@ -0,0 +1,17 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- Add hash indexes on columns that are commonly queried with a direct equals
CREATE INDEX CONCURRENTLY IF NOT EXISTS host_host_name_hash ON "Host" USING hash (host_name);
CREATE INDEX CONCURRENTLY IF NOT EXISTS host_repo_id_hash ON "Host" USING hash (repo_id);
@@ -0,0 +1,16 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- Add hash indexes on columns that are commonly queried with a direct equals
CREATE INDEX CONCURRENTLY IF NOT EXISTS pollmessage_poll_message_id_hash ON "PollMessage" USING hash (poll_message_id);
@@ -0,0 +1,16 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- Add hash indexes on columns that are commonly queried with a direct equals
CREATE INDEX CONCURRENTLY IF NOT EXISTS allocationtoken_token_hash ON "AllocationToken" USING hash (token);
@@ -0,0 +1,17 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- Add hash indexes on columns that are commonly queried with a direct equals
CREATE INDEX CONCURRENTLY IF NOT EXISTS domainhistoryhost_domain_history_history_revision_id_hash ON "DomainHistoryHost"
USING hash (domain_history_history_revision_id);
@@ -0,0 +1,17 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- Add hash indexes on columns that are commonly queried with a direct equals
CREATE INDEX CONCURRENTLY IF NOT EXISTS domaindsdatahistory_domain_history_revision_id_hash ON "DomainDsDataHistory"
USING hash (domain_history_revision_id);
@@ -0,0 +1,17 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- Add hash indexes on columns that are commonly queried with a direct equals
CREATE INDEX CONCURRENTLY IF NOT EXISTS graceperiodhistory_domain_history_revision_id_hash ON "GracePeriodHistory" USING
hash (domain_history_revision_id);
@@ -1924,6 +1924,55 @@ ALTER TABLE ONLY public."User"
CREATE INDEX allocation_token_domain_name_idx ON public."AllocationToken" USING btree (domain_name);
--
-- Name: allocationtoken_token_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX allocationtoken_token_hash ON public."AllocationToken" USING hash (token);
--
-- Name: billingcancellation_billing_cancellation_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX billingcancellation_billing_cancellation_id_hash ON public."BillingCancellation" USING hash (billing_cancellation_id);
--
-- Name: billingevent_billing_event_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX billingevent_billing_event_id_hash ON public."BillingEvent" USING hash (billing_event_id);
--
-- Name: billingrecurrence_billing_recurrence_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX billingrecurrence_billing_recurrence_id_hash ON public."BillingRecurrence" USING hash (billing_recurrence_id);
--
-- Name: delegationsignerdata_domain_repo_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX delegationsignerdata_domain_repo_id_hash ON public."DelegationSignerData" USING hash (domain_repo_id);
--
-- Name: domain_domain_name_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX domain_domain_name_hash ON public."Domain" USING hash (domain_name);
--
-- Name: domain_domain_repo_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX domain_domain_repo_id_hash ON public."Domain" USING hash (repo_id);
--
-- Name: domain_history_to_ds_data_history_idx; Type: INDEX; Schema: public; Owner: -
--
@@ -1938,6 +1987,97 @@ CREATE INDEX domain_history_to_ds_data_history_idx ON public."DomainDsDataHistor
CREATE INDEX domain_history_to_transaction_record_idx ON public."DomainTransactionRecord" USING btree (domain_repo_id, history_revision_id);
--
-- Name: domaindsdatahistory_domain_history_revision_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX domaindsdatahistory_domain_history_revision_id_hash ON public."DomainDsDataHistory" USING hash (domain_history_revision_id);
--
-- Name: domainhistory_domain_repo_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX domainhistory_domain_repo_id_hash ON public."DomainHistory" USING hash (domain_repo_id);
--
-- Name: domainhistory_history_revision_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX domainhistory_history_revision_id_hash ON public."DomainHistory" USING hash (history_revision_id);
--
-- Name: domainhistoryhost_domain_history_history_revision_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX domainhistoryhost_domain_history_history_revision_id_hash ON public."DomainHistoryHost" USING hash (domain_history_history_revision_id);
--
-- Name: domainhost_domain_repo_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX domainhost_domain_repo_id_hash ON public."DomainHost" USING hash (domain_repo_id);
--
-- Name: domaintransactionrecord_domain_history_revision_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX domaintransactionrecord_domain_history_revision_id_hash ON public."DomainTransactionRecord" USING hash (history_revision_id);
--
-- Name: domaintransactionrecord_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX domaintransactionrecord_id_hash ON public."DomainTransactionRecord" USING hash (id);
--
-- Name: graceperiod_domain_repo_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX graceperiod_domain_repo_id_hash ON public."GracePeriod" USING hash (domain_repo_id);
--
-- Name: graceperiod_grace_period_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX graceperiod_grace_period_id_hash ON public."GracePeriod" USING hash (grace_period_id);
--
-- Name: graceperiodhistory_domain_history_revision_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX graceperiodhistory_domain_history_revision_id_hash ON public."GracePeriodHistory" USING hash (domain_history_revision_id);
--
-- Name: graceperiodhistory_grace_period_history_revision_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX graceperiodhistory_grace_period_history_revision_id_hash ON public."GracePeriodHistory" USING hash (grace_period_history_revision_id);
--
-- Name: host_host_name_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX host_host_name_hash ON public."Host" USING hash (host_name);
--
-- Name: host_repo_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX host_repo_id_hash ON public."Host" USING hash (repo_id);
--
-- Name: idx1dyqmqb61xbnj7mt7bk27ds25; Type: INDEX; Schema: public; Owner: -
--
@@ -2617,6 +2757,13 @@ CREATE INDEX idxtmlqd31dpvvd2g1h9i7erw6aj ON public."AllocationToken" USING btre
CREATE INDEX idxy98mebut8ix1v07fjxxdkqcx ON public."Host" USING btree (creation_time);
--
-- Name: pollmessage_poll_message_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX pollmessage_poll_message_id_hash ON public."PollMessage" USING hash (poll_message_id);
--
-- Name: premiumlist_name_idx; Type: INDEX; Schema: public; Owner: -
--
+1 -1
View File
@@ -7,7 +7,7 @@
# nom_build), run ./nom_build --help.
#
# DO NOT EDIT THIS FILE BY HAND
org.gradle.jvmargs=-Xmx1024m
org.gradle.jvmargs=-Xmx2048m
org.gradle.caching=true
org.gradle.parallel=true
mavenUrl=
+28 -16
View File
@@ -42,6 +42,20 @@ steps:
rm -rf .git && rm -rf nomulus-internal/.git
cp -rf nomulus-internal/* .
rm -rf nomulus-internal
# Remove environment configs from .gitignore
- name: 'gcr.io/cloud-builders/git'
entrypoint: /bin/bash
args:
- -c
- |
set -e
sed -i \
-e '\#core/src/main/java/google/registry/config/files/nomulus-config-alpha.yaml#d' \
-e '\#core/src/main/java/google/registry/config/files/nomulus-config-crash.yaml#d' \
-e '\#core/src/main/java/google/registry/config/files/nomulus-config-production.yaml#d' \
-e '\#core/src/main/java/google/registry/config/files/nomulus-config-qa.yaml#d' \
-e '\#core/src/main/java/google/registry/config/files/nomulus-config-sandbox.yaml#d' \
.gitignore
# Build the builder image and pull the base images, them upload them to GCR.
- name: 'gcr.io/cloud-builders/docker'
entrypoint: /bin/bash
@@ -266,27 +280,25 @@ steps:
rm ${gradle_bin}
sed -i s%services.gradle.org/distributions%storage.googleapis.com/${gcs_loc}% \
gradle/wrapper/gradle-wrapper.properties
# Check out the release repo.
# Conditionally trigger the appropriate build based on the tag format.
- name: 'gcr.io/cloud-builders/gcloud'
args: ['source', 'repos', 'clone', 'nomulus-release']
# Tag and check in the release repo.
- name: 'gcr.io/cloud-builders/git'
entrypoint: /bin/bash
entrypoint: 'bash'
args:
- -c
- |
set -e
rm -rf gcompute-tools
cp -rf nomulus-release/.git .
rm -rf nomulus-release
git config --global user.name "Cloud Build"
git config --global user.email \
$(gcloud auth list --format='get(account)' --filter=active)
git add .
git commit -m "Release commit for tag ${TAG_NAME}"
git push -o nokeycheck origin master
git tag ${TAG_NAME}
git push -o nokeycheck origin ${TAG_NAME}
# Check for a nomulus release tag (e.g., "v1.2.3")
if [[ "${TAG_NAME}" =~ ^nomulus-20\d{2}[0-1]\d[0-3]\d-RC\d{2}$ ]]; then
echo "Tag format matches a nomulus release. Triggering nomulus build..."
gcloud builds submit . --config=release/cloudbuild-nomulus.yaml --substitutions=TAG_NAME=$TAG_NAME
# Check for a proxy release tag (e.g., "proxy-v1.2.3")
elif [[ "${TAG_NAME}" =~ ^proxy-20\d{2}[0-1]\d[0-3]\d-RC\d{2}$ ]]; then
echo "Tag format matches a proxy release. Triggering proxy build..."
gcloud builds submit . --config=release/cloudbuild-proxy.yaml --substitutions=TAG_NAME=$TAG_NAME
else
echo "Tag format '$TAG_NAME' does not match a known release type. Exiting."
exit 1
fi
timeout: 3600s
options:
machineType: 'E2_HIGHCPU_32'