1
0
mirror of https://github.com/google/nomulus synced 2026-05-25 17:20:32 +00:00

Compare commits

...

13 Commits

Author SHA1 Message Date
gbrodman
97d0b7680f Add hash indexes for common use cases (#2834)
I went through all the SQL statements generated by some sample
DomainCreateFlow and DomainDeleteFlow cases to find situations where we
were either SELECTing from, or UPDATEing, tables with a direct "field =
value" format. These are the situations that I found where we can add
hash indexes. This does two things:

1. Makes these queries slight faster, since these are usually queries on
   columns that are either unique or very close to unique, and O(1) is
   faster than O(log(n))
2. Spreads around the optimistic predicate locks on the previously-used
   btree indexes. Many of our serialization errors came from the fact
   that we were autogenerating incrementing ID values for various
   tables, meaning that SELECTs, INSERTs, and UPDATEs would all try to
   take predicate locks out on the same page of the btree index. Using a
   hash index means that the page locks will be spread out to various
   index pages, rather than conflicting with each other.

Running load tests on alpha I see significant improvements in speed and
error rates. Speed is hard to quantify due to the nature of the way the
load tests distribute tasks among the queues but it could be more than
50% improvement, and serialization errors in the logs drop by more than
90%.
2025-09-29 22:16:24 +00:00
Pavlo Tkach
5700a008d6 Add console history frontend (#2832) 2025-09-26 21:25:03 +00:00
Ben McIlwain
dc9f5b99bc Add a batch action to remove all contacts from domains (#2827)
This implements the first part of Minimum Data Set phase 3, wherein we delete
all contact data. This action is necessary to leave a permanent record on the
domain (in the form of a domain history entry) documenting when the contacts
were removed by the administrative user.

Then, after this has finished removing all contact assocations, we can simply
empty out or drop the Contact/ContactHistory tables and associated join tables.
2025-09-25 20:47:17 +00:00
Ben McIlwain
d3c6de7a38 Modify the base Latin LGR with our intended changes to improve security (#2829) 2025-09-24 21:04:37 +00:00
Ben McIlwain
3c3303c16a Add ICANN's reference Latin LGR in RFC 7940 XML format (#2828)
In the next commit I will make changes to this file so it supports just the
basic Latin characters that we want, but it's good to check the base version in
so that we can see diffs.

This was downloaded from https://www.icann.org/sites/default/files/packages/lgr/lgr-second-level-latin-script-25oct24-en.xml
2025-09-19 16:42:46 +00:00
Nilay Shah
2a86a1bbe9 Skip user loading for proxy service account (#2825)
* Skip user loading for proxy service account

Reduces database load by skipping the User entity lookup for the proxy
service account during OIDC authentication.

The high volume of EPP "hello" and "login" commands from the proxy
service account results in a constant database load. These lookups
are unnecessary as the proxy service account is not expected to have a
corresponding User object.

This change optimizes the authentication flow by checking for the proxy
service account email *before* attempting to load a User from the
database. This bypasses the database transaction entirely for these
high-volume requests.

This approach is more efficient than caching, as it eliminates the
database lookup for the proxy service account altogether, rather than
just caching the result.

* comment added and service account llokup time improved

* comment updated for more clarity
2025-09-16 18:48:39 +00:00
gbrodman
ea148ac13e Show success message on password reset (#2826) 2025-09-16 18:39:19 +00:00
Nilay Shah
06299ccb86 Add cache for User entities in OIDC auth flow (#2822)
* Add cache for User entities in OIDC auth flow

* refactor: Address review feedback

- Refactor database call into a single, reusable method
- Increase the default cache size to 200
- Remove .recordStats() and using spy for testing
- Split unit tests into separate implementation test that use Mockito spies instead of checking internal cache stats
2025-09-12 07:43:32 +00:00
gbrodman
732c30b359 Remove registry-lock-related fields from RegistrarPoc (#2818)
We've moved these over to the User class, so we should remove these for
clarity. In addition, we should make it clear (in Java at least) that
the field in the RegistryLock object refers to the email address used
for the lock in question.
2025-09-11 15:29:06 +00:00
gbrodman
ee5a2d3916 Include internal registrars in the console (#2821)
This allows us to also check / modify the CharlestonRoad registrar in
the console, and also allows us to test actions (like password reset)
using that registrar in the prod environment.
2025-09-05 20:37:23 +00:00
gbrodman
2b5643df4c Sort registrars list in console (#2820)
This was bugging me slightly
2025-09-05 18:44:17 +00:00
Pavlo Tkach
6bbd7a2290 Update proxy resources, increase ssl handshake timeout (#2819) 2025-09-05 18:09:55 +00:00
Weimin Yu
77ab80f3dc Fix OOM in UploadBsaUnavailableDomains action (#2817)
* Fix OOM in UploadBsaUnavailableDomains action

The action was using string concatenation to generate the upload content.
This causes an OOM when string length exceeds 25MB on our current VM.

This PR witches to streaming upload.

Also added an HTTP upload test.

* Fix OOM in UploadBsaUnavailableDomains action

The action was using string concatenation to generate the upload content.
This causes an OOM when string length exceeds 25MB on our current VM.

This PR witches to streaming upload.

Also added an HTTP upload test.
2025-09-03 18:25:56 +00:00
75 changed files with 3151 additions and 5990 deletions

View File

@@ -1,7 +1,7 @@
{
"/console-api":
{
"target": "http://localhost:8080",
"target": "http://[::1]:8080",
"secure": false,
"logLevel": "debug",
"changeOrigin": true

View File

@@ -26,6 +26,7 @@ import SecurityComponent from './settings/security/security.component';
import { SettingsComponent } from './settings/settings.component';
import { SupportComponent } from './support/support.component';
import RdapComponent from './settings/rdap/rdap.component';
import { HistoryComponent } from './history/history.component';
import { PasswordResetVerifyComponent } from './shared/components/passwordReset/passwordResetVerify.component';
export interface RouteWithIcon extends Route {
@@ -64,13 +65,18 @@ export const routes: RouteWithIcon[] = [
title: 'Dashboard',
iconName: 'view_comfy_alt',
},
// { path: 'tlds', component: TldsComponent, title: "TLDs", iconName: "event_list" },
{
path: DomainListComponent.PATH,
component: DomainListComponent,
title: 'Domains',
iconName: 'view_list',
},
{
path: HistoryComponent.PATH,
component: HistoryComponent,
// title: 'History',
// iconName: 'history',
},
{
path: SettingsComponent.PATH,
component: SettingsComponent,

View File

@@ -56,13 +56,14 @@ import { GlobalLoaderService } from './shared/services/globalLoader.service';
import { UserDataService } from './shared/services/userData.service';
import { SnackBarModule } from './snackbar.module';
import { SupportComponent } from './support/support.component';
import { TldsComponent } from './tlds/tlds.component';
import { ForceFocusDirective } from './shared/directives/forceFocus.directive';
import RdapComponent from './settings/rdap/rdap.component';
import RdapEditComponent from './settings/rdap/rdapEdit.component';
import { PocReminderComponent } from './shared/components/pocReminder/pocReminder.component';
import { PasswordResetVerifyComponent } from './shared/components/passwordReset/passwordResetVerify.component';
import { PasswordInputForm } from './shared/components/passwordReset/passwordInputForm.component';
import { HistoryComponent } from './history/history.component';
import { HistoryListComponent } from './history/historyList.component';
@NgModule({
declarations: [SelectedRegistrarWrapper],
@@ -81,6 +82,8 @@ export class SelectedRegistrarModule {}
EppPasswordEditComponent,
ForceFocusDirective,
HeaderComponent,
HistoryComponent,
HistoryListComponent,
HomeComponent,
LocationBackDirective,
NavigationComponent,
@@ -104,7 +107,6 @@ export class SelectedRegistrarModule {}
SettingsComponent,
SettingsContactComponent,
SupportComponent,
TldsComponent,
UserLevelVisibility,
],
bootstrap: [AppComponent],

View File

@@ -0,0 +1,62 @@
<app-selected-registrar-wrapper>
<div class="history-log">
<h1 class="mat-headline-4" forceFocus>
Registrar Console Activity History
</h1>
<mat-tab-group
[elementId]="getElementIdForUserLog()"
class="history-log__tabs"
>
<mat-tab label="Registrar Activity">
<div class="spacer"></div>
<app-history-list
[historyRecords]="historyService.historyRecordsRegistrar()"
[isLoading]="isLoading"
/>
</mat-tab>
<mat-tab label="User Activity">
<div class="spacer"></div>
<form (ngSubmit)="loadHistory()" #form="ngForm">
<section>
<mat-form-field appearance="outline">
<mat-label>Console User Email: </mat-label>
<input
matInput
id="email"
type="email"
name="consoleUserEmail"
required
email
[(ngModel)]="consoleUserEmail"
#emailControl="ngModel"
/>
</mat-form-field>
</section>
<div class="spacer"></div>
<button
mat-flat-button
color="primary"
type="submit"
aria-label="Search user history"
[disabled]="!form.valid"
>
Search
</button>
</form>
<div class="spacer"></div>
<app-history-list
[historyRecords]="historyService.historyRecordsUser()"
[isLoading]="isLoading"
/>
</mat-tab>
</mat-tab-group>
</div>
<app-history-list
[elementId]="getElementIdForUserLog()"
[isReverse]="true"
[historyRecords]="historyService.historyRecordsUser()"
[isLoading]="isLoading"
/>
</app-selected-registrar-wrapper>

View File

@@ -1,4 +1,4 @@
// Copyright 2024 The Nomulus Authors. All Rights Reserved.
// Copyright 2025 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
@@ -12,17 +12,11 @@
// See the License for the specific language governing permissions and
// limitations under the License.
.console-tlds {
&__cards {
display: flex;
border-top: 1px solid #ddd;
padding: 1rem;
}
&__card {
max-width: 300px;
}
&__card-links {
display: flex;
flex-direction: column;
.history-log {
font-family: "Roboto", sans-serif;
max-width: 760px;
.spacer {
margin: 20px 0;
}
}

View File

@@ -0,0 +1,80 @@
// Copyright 2025 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { Component, effect } from '@angular/core';
import { UserDataService } from '../shared/services/userData.service';
import { BackendService } from '../shared/services/backend.service';
import { RegistrarService } from '../registrar/registrar.service';
import { HistoryService } from './history.service';
import { MatSnackBar } from '@angular/material/snack-bar';
import {
GlobalLoader,
GlobalLoaderService,
} from '../shared/services/globalLoader.service';
import { HttpErrorResponse } from '@angular/common/http';
import { RESTRICTED_ELEMENTS } from '../shared/directives/userLevelVisiblity.directive';
@Component({
selector: 'app-history',
templateUrl: './history.component.html',
styleUrls: ['./history.component.scss'],
providers: [HistoryService],
standalone: false,
})
export class HistoryComponent implements GlobalLoader {
public static PATH = 'history';
consoleUserEmail: string = '';
isLoading: boolean = false;
constructor(
private backendService: BackendService,
private registrarService: RegistrarService,
protected historyService: HistoryService,
protected globalLoader: GlobalLoaderService,
protected userDataService: UserDataService,
private _snackBar: MatSnackBar
) {
effect(() => {
if (registrarService.registrarId()) {
this.loadHistory();
}
});
}
getElementIdForUserLog() {
return RESTRICTED_ELEMENTS.ACTIVITY_PER_USER;
}
loadingTimeout() {
this._snackBar.open('Timeout loading records history');
}
loadHistory() {
this.globalLoader.startGlobalLoader(this);
this.isLoading = true;
this.historyService
.getHistoryLog(this.registrarService.registrarId(), this.consoleUserEmail)
.subscribe({
error: (err: HttpErrorResponse) => {
this._snackBar.open(err.error || err.message);
this.isLoading = false;
},
next: () => {
this.globalLoader.stopGlobalLoader(this);
this.isLoading = false;
},
});
}
}

View File

@@ -0,0 +1,46 @@
// Copyright 2025 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { Injectable, signal } from '@angular/core';
import { BackendService } from '../shared/services/backend.service';
import { tap } from 'rxjs';
export interface HistoryRecord {
modificationTime: string;
type: string;
description: string;
actingUser: {
emailAddress: string;
};
}
@Injectable()
export class HistoryService {
historyRecordsRegistrar = signal<HistoryRecord[]>([]);
historyRecordsUser = signal<HistoryRecord[]>([]);
constructor(private backendService: BackendService) {}
getHistoryLog(registrarId: string, userEmail?: string) {
return this.backendService.getHistoryLog(registrarId, userEmail).pipe(
tap((historyRecords: HistoryRecord[]) => {
if (userEmail) {
this.historyRecordsUser.set(historyRecords);
} else {
this.historyRecordsRegistrar.set(historyRecords);
}
})
);
}
}

View File

@@ -0,0 +1,50 @@
@if (!isLoading && historyRecords.length == 0) {
<div class="history-list__no-records">
<mat-icon class="history-list__no-records-icon secondary-text"
>apps_outage</mat-icon
>
<h1>No records found</h1>
</div>
} @else {
<mat-card>
<mat-card-content>
<mat-list role="list">
<ng-container *ngFor="let item of historyRecords; let last = last">
<mat-list-item class="history-list__item">
<mat-icon
[ngClass]="getIconClass(item.type)"
class="history-list__icon"
>
{{ getIconForType(item.type) }}
</mat-icon>
<div class="history-list__content">
<div class="history-list__description">
<span class="history-list__description--main">{{
item.type
}}</span>
<div>
<mat-chip
*ngIf="parseDescription(item.description).detail"
class="history-list__chip"
>
{{ parseDescription(item.description).detail }}
</mat-chip>
</div>
</div>
<div class="history-list__user">
<b>User - {{ item.actingUser.emailAddress }}</b>
</div>
</div>
<span class="history-list__timestamp">
{{ item.modificationTime | date : "MMM d, y, h:mm a" }}
</span>
</mat-list-item>
<mat-divider *ngIf="!last"></mat-divider>
</ng-container>
</mat-list>
</mat-card-content>
</mat-card>
}

View File

@@ -0,0 +1,81 @@
// Copyright 2025 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
.history-list {
font-family: "Roboto", sans-serif;
&__item {
display: flex;
align-items: center;
// Override default mat-list-item height to fit content
height: auto !important;
padding: 16px 0;
}
&__no-records {
display: flex;
justify-content: center;
align-items: center;
flex-direction: column;
}
&__no-records-icon {
width: 4rem;
height: 4rem;
font-size: 4rem;
margin-top: 1.5rem;
}
&__icon {
margin-right: 16px;
&--update {
color: #1976d2;
}
&--security {
color: #d32f2f;
}
}
&__description {
&--main {
font-size: 1rem;
font-weight: 500;
color: rgba(0, 0, 0, 0.87);
margin-bottom: 1em;
}
}
&__content {
flex-grow: 1;
display: flex;
flex-direction: column;
gap: 4px;
margin-right: 16px;
}
&__chip {
margin: 0.5rem 0;
}
&__user {
font-size: 0.9rem;
color: rgba(0, 0, 0, 0.6);
}
&__timestamp {
color: rgba(0, 0, 0, 0.6);
white-space: nowrap;
text-align: right;
}
}

View File

@@ -0,0 +1,66 @@
// Copyright 2025 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { ChangeDetectionStrategy, Component, Input } from '@angular/core';
import { HistoryRecord } from './history.service';
@Component({
selector: 'app-history-list',
templateUrl: './historyList.component.html',
styleUrls: ['./historyList.component.scss'],
changeDetection: ChangeDetectionStrategy.OnPush,
standalone: false,
})
export class HistoryListComponent {
@Input() historyRecords: HistoryRecord[] = [];
@Input() isLoading: boolean = false;
getIconForType(type: string): string {
switch (type) {
case 'REGISTRAR_UPDATE':
return 'edit';
case 'REGISTRAR_SECURITY_UPDATE':
return 'security';
default:
return 'history'; // A fallback icon
}
}
getIconClass(type: string): string {
switch (type) {
case 'REGISTRAR_UPDATE':
return 'history-log__icon--update';
case 'REGISTRAR_SECURITY_UPDATE':
return 'history-log__icon--security';
default:
return '';
}
}
parseDescription(description: string): {
main: string;
detail: string | null;
} {
if (!description) {
return { main: 'N/A', detail: null };
}
const parts = description.split('|');
const detail = parts.length > 1 ? parts[1].replace(/_/g, ' ') : null;
return {
main: parts[0],
detail: detail,
};
}
}

View File

@@ -25,7 +25,10 @@ export class RegistrarSelectorComponent {
registrarInput = signal<string>(this.registrarService.registrarId());
filteredOptions?: string[];
allRegistrarIds = computed(() =>
this.registrarService.registrars().map((r) => r.registrarId)
this.registrarService
.registrars()
.map((r) => r.registrarId)
.sort()
);
constructor(protected registrarService: RegistrarService) {

View File

@@ -24,6 +24,7 @@ import {
PasswordResults,
} from './passwordInputForm.component';
import EppPasswordEditComponent from 'src/app/settings/security/eppPasswordEdit.component';
import { MatSnackBar } from '@angular/material/snack-bar';
export interface PasswordResetVerifyResponse {
registrarId: string;
@@ -54,7 +55,8 @@ export class PasswordResetVerifyComponent {
protected backendService: BackendService,
protected registrarService: RegistrarService,
private route: ActivatedRoute,
private router: Router
private router: Router,
private _snackBar: MatSnackBar
) {}
ngOnInit() {
@@ -99,7 +101,10 @@ export class PasswordResetVerifyComponent {
this.isLoading = false;
this.errorMessage = err.error;
},
next: (_) => this.router.navigate(['']),
next: (_) => {
this.router.navigate(['']);
this._snackBar.open('Password reset completed successfully');
},
});
}
}

View File

@@ -16,6 +16,7 @@ import { Directive, ElementRef, Input, effect } from '@angular/core';
import { UserDataService } from '../services/userData.service';
export enum RESTRICTED_ELEMENTS {
ACTIVITY_PER_USER,
REGISTRAR_ELEMENT,
OTE,
USERS,
@@ -28,9 +29,10 @@ export const DISABLED_ELEMENTS_PER_ROLE = {
RESTRICTED_ELEMENTS.REGISTRAR_ELEMENT,
RESTRICTED_ELEMENTS.OTE,
RESTRICTED_ELEMENTS.SUSPEND,
RESTRICTED_ELEMENTS.ACTIVITY_PER_USER,
],
SUPPORT_LEAD: [],
SUPPORT_AGENT: [],
SUPPORT_AGENT: [RESTRICTED_ELEMENTS.ACTIVITY_PER_USER],
};
@Directive({
@@ -40,6 +42,8 @@ export const DISABLED_ELEMENTS_PER_ROLE = {
export class UserLevelVisibility {
@Input() elementId!: RESTRICTED_ELEMENTS | null;
@Input() isReverse: boolean = false;
constructor(
private userDataService: UserDataService,
private el: ElementRef
@@ -56,9 +60,9 @@ export class UserLevelVisibility {
// @ts-ignore
(DISABLED_ELEMENTS_PER_ROLE[globalRole] || []).includes(this.elementId)
) {
this.el.nativeElement.style.display = 'none';
this.el.nativeElement.style.display = this.isReverse ? '' : 'none';
} else {
this.el.nativeElement.style.display = '';
this.el.nativeElement.style.display = this.isReverse ? 'none' : '';
}
}
}

View File

@@ -31,6 +31,7 @@ import { Contact } from '../../settings/contact/contact.service';
import { EppPasswordBackendModel } from '../../settings/security/security.service';
import { UserData } from './userData.service';
import { PasswordResetVerifyResponse } from '../components/passwordReset/passwordResetVerify.component';
import { HistoryRecord } from '../../history/history.service';
@Injectable()
export class BackendService {
@@ -123,6 +124,16 @@ export class BackendService {
.pipe(catchError((err) => this.errorCatcher<DomainListResult>(err)));
}
getHistoryLog(registrarId: string, userEmail?: string) {
return this.http
.get<HistoryRecord[]>(
userEmail
? `/console-api/history?registrarId=${registrarId}&consoleUserEmail=${userEmail}`
: `/console-api/history?registrarId=${registrarId}`
)
.pipe(catchError((err) => this.errorCatcher<HistoryRecord[]>(err)));
}
getRegistrars(): Observable<Registrar[]> {
return this.http
.get<Registrar[]>('/console-api/registrars')

View File

@@ -1 +0,0 @@
<div class="console-tlds__cards"></div>

View File

@@ -1,38 +0,0 @@
// Copyright 2024 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { ComponentFixture, TestBed } from '@angular/core/testing';
import { TldsComponent } from './tlds.component';
import { MaterialModule } from '../material.module';
describe('TldsComponent', () => {
let component: TldsComponent;
let fixture: ComponentFixture<TldsComponent>;
beforeEach(async () => {
await TestBed.configureTestingModule({
imports: [MaterialModule],
declarations: [TldsComponent],
}).compileComponents();
fixture = TestBed.createComponent(TldsComponent);
component = fixture.componentInstance;
fixture.detectChanges();
});
it('should create', () => {
expect(component).toBeTruthy();
});
});

View File

@@ -1,23 +0,0 @@
// Copyright 2024 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { Component } from '@angular/core';
@Component({
selector: 'app-tlds',
templateUrl: './tlds.component.html',
styleUrls: ['./tlds.component.scss'],
standalone: false,
})
export class TldsComponent {}

View File

@@ -58,6 +58,8 @@ def fragileTestPatterns = [
// Changes cache timeouts and for some reason appears to have contention
// with other tests.
"google/registry/whois/WhoisCommandFactoryTest.*",
// Breaks random other tests when running with standardTests.
"google/registry/bsa/UploadBsaUnavailableDomainsActionTest.*",
// Currently changes a global configuration parameter that for some reason
// results in timestamp inversions for other tests. TODO(mmuller): fix.
"google/registry/flows/host/HostInfoFlowTest.*",

View File

@@ -15,7 +15,6 @@
package google.registry.batch;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.request.Action.Method.POST;
import static google.registry.tools.LockOrUnlockDomainCommand.REGISTRY_LOCK_STATUSES;
@@ -30,8 +29,6 @@ import google.registry.groups.GmailClient;
import google.registry.model.domain.Domain;
import google.registry.model.domain.RegistryLock;
import google.registry.model.eppcommon.StatusValue;
import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.RegistrarPoc;
import google.registry.model.tld.RegistryLockDao;
import google.registry.persistence.VKey;
import google.registry.request.Action;
@@ -70,12 +67,14 @@ public class RelockDomainAction implements Runnable {
"""
The domain %s was successfully re-locked.
Please contact support at %s if you have any questions.""";
Please contact support at %s if you have any questions.\
""";
private static final String RELOCK_NON_RETRYABLE_FAILURE_EMAIL_TEMPLATE =
"""
There was an error when automatically re-locking %s. Error message: %s
Please contact support at %s if you have any questions.""";
Please contact support at %s if you have any questions.\
""";
private static final String RELOCK_TRANSIENT_FAILURE_EMAIL_TEMPLATE =
"There was an unexpected error when automatically re-locking %s. We will continue retrying "
+ "the lock for five hours. Please contact support at %s if you have any questions";
@@ -171,7 +170,7 @@ public class RelockDomainAction implements Runnable {
domainLockUtils.administrativelyApplyLock(
oldLock.getDomainName(),
oldLock.getRegistrarId(),
oldLock.getRegistrarPocId(),
oldLock.getRegistryLockEmail(),
oldLock.isSuperuser());
logger.atInfo().log("Re-locked domain %s.", oldLock.getDomainName());
response.setStatus(SC_OK);
@@ -221,7 +220,7 @@ public class RelockDomainAction implements Runnable {
EmailMessage.newBuilder()
.setBody(body)
.setSubject(String.format("Error re-locking domain %s", oldLock.getDomainName()))
.setRecipients(getEmailRecipients(oldLock.getRegistrarId()))
.setRecipients(ImmutableSet.of(getEmailRecipient(oldLock)))
.build());
}
@@ -250,7 +249,7 @@ public class RelockDomainAction implements Runnable {
EmailMessage.newBuilder()
.setBody(body)
.setSubject(String.format("Successful re-lock of domain %s", oldLock.getDomainName()))
.setRecipients(getEmailRecipients(oldLock.getRegistrarId()))
.setRecipients(ImmutableSet.of(getEmailRecipient(oldLock)))
.build());
}
@@ -261,7 +260,7 @@ public class RelockDomainAction implements Runnable {
// For an unexpected failure, notify both the lock-enabled contacts and our alerting email
ImmutableSet<InternetAddress> allRecipients =
new ImmutableSet.Builder<InternetAddress>()
.addAll(getEmailRecipients(oldLock.getRegistrarId()))
.add(getEmailRecipient(oldLock))
.add(alertRecipientAddress)
.build();
gmailClient.sendEmail(
@@ -281,31 +280,12 @@ public class RelockDomainAction implements Runnable {
.build());
}
private ImmutableSet<InternetAddress> getEmailRecipients(String registrarId) {
Registrar registrar =
Registrar.loadByRegistrarIdCached(registrarId)
.orElseThrow(
() ->
new IllegalStateException(String.format("Unknown registrar %s", registrarId)));
ImmutableSet<String> registryLockEmailAddresses =
registrar.getContacts().stream()
.filter(RegistrarPoc::isRegistryLockAllowed)
.map(RegistrarPoc::getRegistryLockEmailAddress)
.filter(Optional::isPresent)
.map(Optional::get)
.collect(toImmutableSet());
ImmutableSet.Builder<InternetAddress> builder = new ImmutableSet.Builder<>();
// can't use streams due to the 'throws' in the InternetAddress constructor
for (String registryLockEmailAddress : registryLockEmailAddresses) {
try {
builder.add(new InternetAddress(registryLockEmailAddress));
} catch (AddressException e) {
// This shouldn't stop any other emails going out, so swallow it
logger.atWarning().log("Invalid email address '%s'.", registryLockEmailAddress);
}
private InternetAddress getEmailRecipient(RegistryLock lock) {
try {
return new InternetAddress(lock.getRegistryLockEmail());
} catch (AddressException e) {
// this really shouldn't happen
throw new RuntimeException(e);
}
return builder.build();
}
}

View File

@@ -0,0 +1,232 @@
// Copyright 2025 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.batch;
import static com.google.common.base.Preconditions.checkState;
import static com.google.common.net.MediaType.PLAIN_TEXT_UTF_8;
import static google.registry.flows.FlowUtils.marshalWithLenientRetry;
import static google.registry.model.common.FeatureFlag.FeatureName.MINIMUM_DATASET_CONTACTS_PROHIBITED;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import static google.registry.util.ResourceUtils.readResourceUtf8;
import static jakarta.servlet.http.HttpServletResponse.SC_INTERNAL_SERVER_ERROR;
import static jakarta.servlet.http.HttpServletResponse.SC_NO_CONTENT;
import static jakarta.servlet.http.HttpServletResponse.SC_OK;
import static java.nio.charset.StandardCharsets.US_ASCII;
import com.google.common.base.Ascii;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.google.common.flogger.FluentLogger;
import google.registry.config.RegistryConfig.Config;
import google.registry.flows.EppController;
import google.registry.flows.EppRequestSource;
import google.registry.flows.PasswordOnlyTransportCredentials;
import google.registry.flows.StatelessRequestSessionMetadata;
import google.registry.model.common.FeatureFlag;
import google.registry.model.contact.Contact;
import google.registry.model.domain.DesignatedContact;
import google.registry.model.domain.Domain;
import google.registry.model.eppcommon.ProtocolDefinition;
import google.registry.model.eppoutput.EppOutput;
import google.registry.persistence.VKey;
import google.registry.request.Action;
import google.registry.request.Action.GaeService;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.request.lock.LockHandler;
import jakarta.inject.Inject;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.logging.Level;
import javax.annotation.Nullable;
import org.joda.time.Duration;
/**
* An action that removes all contacts from all active (non-deleted) domains.
*
* <p>This implements part 1 of phase 3 of the Minimum Dataset migration, wherein we remove all uses
* of contact objects in preparation for later removing all contact data from the system.
*
* <p>This runs as a singly threaded, resumable action that loads batches of domains still
* containing contacts, and runs a superuser domain update on each one to remove the contacts,
* leaving behind a record recording that update.
*/
@Action(
service = GaeService.BACKEND,
path = RemoveAllDomainContactsAction.PATH,
method = Action.Method.POST,
auth = Auth.AUTH_ADMIN)
public class RemoveAllDomainContactsAction implements Runnable {
public static final String PATH = "/_dr/task/removeAllDomainContacts";
private static final String LOCK_NAME = "Remove all domain contacts";
private static final String CONTACT_FMT = "<domain:contact type=\"%s\">%s</domain:contact>";
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private final EppController eppController;
private final String registryAdminClientId;
private final LockHandler lockHandler;
private final Response response;
private final String updateDomainXml;
private int successes = 0;
private int failures = 0;
private static final int BATCH_SIZE = 10000;
@Inject
RemoveAllDomainContactsAction(
EppController eppController,
@Config("registryAdminClientId") String registryAdminClientId,
LockHandler lockHandler,
Response response) {
this.eppController = eppController;
this.registryAdminClientId = registryAdminClientId;
this.lockHandler = lockHandler;
this.response = response;
this.updateDomainXml =
readResourceUtf8(RemoveAllDomainContactsAction.class, "domain_remove_contacts.xml");
}
@Override
public void run() {
checkState(
tm().transact(() -> FeatureFlag.isActiveNow(MINIMUM_DATASET_CONTACTS_PROHIBITED)),
"Minimum dataset migration must be completed prior to running this action");
response.setContentType(PLAIN_TEXT_UTF_8);
Callable<Void> runner =
() -> {
try {
runLocked();
response.setStatus(SC_OK);
} catch (Exception e) {
logger.atSevere().withCause(e).log("Errored out during execution.");
response.setStatus(SC_INTERNAL_SERVER_ERROR);
response.setPayload(String.format("Errored out with cause: %s", e));
}
return null;
};
if (!lockHandler.executeWithLocks(runner, null, Duration.standardHours(1), LOCK_NAME)) {
// Send a 200-series status code to prevent this conflicting action from retrying.
response.setStatus(SC_NO_CONTENT);
response.setPayload("Could not acquire lock; already running?");
}
}
private void runLocked() {
logger.atInfo().log("Removing contacts on all active domains.");
List<String> domainRepoIdsBatch;
do {
domainRepoIdsBatch =
tm().<List<String>>transact(
() ->
tm().getEntityManager()
.createQuery(
"""
SELECT repoId FROM Domain WHERE deletionTime = :end_of_time AND NOT (
adminContact IS NULL AND billingContact IS NULL
AND registrantContact IS NULL AND techContact IS NULL)
""")
.setParameter("end_of_time", END_OF_TIME)
.setMaxResults(BATCH_SIZE)
.getResultList());
domainRepoIdsBatch.forEach(this::runDomainUpdateFlow);
} while (!domainRepoIdsBatch.isEmpty());
String msg =
String.format(
"Finished; %d domains were successfully updated and %d errored out.",
successes, failures);
logger.at(failures == 0 ? Level.INFO : Level.WARNING).log(msg);
response.setPayload(msg);
}
private void runDomainUpdateFlow(String repoId) {
// Create a new transaction that the flow's execution will be enlisted in that loads the domain
// transactionally. This way we can ensure that nothing else has modified the domain in question
// in the intervening period since the query above found it.
boolean success = tm().transact(() -> runDomainUpdateFlowInner(repoId));
if (success) {
successes++;
} else {
failures++;
}
}
/**
* Runs the actual domain update flow and returns whether the contact removals were successful.
*/
private boolean runDomainUpdateFlowInner(String repoId) {
Domain domain = tm().loadByKey(VKey.create(Domain.class, repoId));
if (!domain.getDeletionTime().equals(END_OF_TIME)) {
// Domain has been deleted since the action began running; nothing further to be
// done here.
logger.atInfo().log("Nothing to process for deleted domain '%s'.", domain.getDomainName());
return false;
}
logger.atInfo().log("Attempting to remove contacts on domain '%s'.", domain.getDomainName());
StringBuilder sb = new StringBuilder();
ImmutableMap<VKey<? extends Contact>, Contact> contacts =
tm().loadByKeys(
domain.getContacts().stream()
.map(DesignatedContact::getContactKey)
.collect(ImmutableSet.toImmutableSet()));
// Collect all the (non-registrant) contacts referenced by the domain and compile an EPP XML
// string that removes each one.
for (DesignatedContact designatedContact : domain.getContacts()) {
@Nullable Contact contact = contacts.get(designatedContact.getContactKey());
if (contact == null) {
logger.atWarning().log(
"Domain '%s' referenced contact with repo ID '%s' that couldn't be" + " loaded.",
domain.getDomainName(), designatedContact.getContactKey().getKey());
continue;
}
sb.append(
String.format(
CONTACT_FMT,
Ascii.toLowerCase(designatedContact.getType().name()),
contact.getContactId()))
.append("\n");
}
String compiledXml =
updateDomainXml
.replace("%DOMAIN%", domain.getDomainName())
.replace("%CONTACTS%", sb.toString());
EppOutput output =
eppController.handleEppCommand(
new StatelessRequestSessionMetadata(
registryAdminClientId, ProtocolDefinition.getVisibleServiceExtensionUris()),
new PasswordOnlyTransportCredentials(),
EppRequestSource.BACKEND,
false,
true,
compiledXml.getBytes(US_ASCII));
if (output.isSuccess()) {
logger.atInfo().log(
"Successfully removed contacts from domain '%s'.", domain.getDomainName());
} else {
logger.atWarning().log(
"Failed removing contacts from domain '%s' with error %s.",
domain.getDomainName(), new String(marshalWithLenientRetry(output), US_ASCII));
}
return output.isSuccess();
}
}

View File

@@ -25,16 +25,16 @@ import static google.registry.request.Action.Method.GET;
import static google.registry.request.Action.Method.POST;
import static jakarta.servlet.http.HttpServletResponse.SC_INTERNAL_SERVER_ERROR;
import static java.nio.charset.StandardCharsets.US_ASCII;
import static java.nio.charset.StandardCharsets.UTF_8;
import com.google.cloud.storage.BlobId;
import com.google.common.base.Joiner;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedSet;
import com.google.common.collect.Ordering;
import com.google.common.flogger.FluentLogger;
import com.google.common.hash.Hasher;
import com.google.common.hash.Hashing;
import com.google.common.io.ByteSource;
import google.registry.bsa.api.BsaCredential;
import google.registry.config.RegistryConfig.Config;
import google.registry.gcs.GcsUtils;
@@ -47,10 +47,13 @@ import google.registry.request.auth.Auth;
import google.registry.util.Clock;
import jakarta.inject.Inject;
import jakarta.persistence.TypedQuery;
import java.io.ByteArrayOutputStream;
import java.io.BufferedInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.OutputStreamWriter;
import java.io.PipedInputStream;
import java.io.PipedOutputStream;
import java.io.Writer;
import java.util.Optional;
import java.util.zip.GZIPOutputStream;
@@ -60,14 +63,17 @@ import okhttp3.OkHttpClient;
import okhttp3.Request;
import okhttp3.RequestBody;
import okhttp3.Response;
import okio.BufferedSink;
import org.jetbrains.annotations.NotNull;
import org.jetbrains.annotations.Nullable;
import org.joda.time.DateTime;
/**
* Daily action that uploads unavailable domain names on applicable TLDs to BSA.
*
* <p>The upload is a single zipped text file containing combined details for all BSA-enrolled TLDs.
* The text is a newline-delimited list of punycoded fully qualified domain names, and contains all
* domains on each TLD that are registered and/or reserved.
* The text is a newline-delimited list of punycoded fully qualified domain names with a trailing
* newline at the end, and contains all domains on each TLD that are registered and/or reserved.
*
* <p>The file is also uploaded to GCS to preserve it as a record for ourselves.
*/
@@ -118,7 +124,7 @@ public class UploadBsaUnavailableDomainsAction implements Runnable {
// TODO(mcilwain): Implement a date Cursor, have the cronjob run frequently, and short-circuit
// the run if the daily upload is already completed.
DateTime runTime = clock.nowUtc();
String unavailableDomains = Joiner.on("\n").join(getUnavailableDomains(runTime));
ImmutableSortedSet<String> unavailableDomains = getUnavailableDomains(runTime);
if (unavailableDomains.isEmpty()) {
logger.atWarning().log("No unavailable domains found; terminating.");
emailSender.sendNotification(
@@ -136,12 +142,16 @@ public class UploadBsaUnavailableDomainsAction implements Runnable {
}
/** Uploads the unavailable domains list to GCS in the unavailable domains bucket. */
boolean uploadToGcs(String unavailableDomains, DateTime runTime) {
boolean uploadToGcs(ImmutableSortedSet<String> unavailableDomains, DateTime runTime) {
logger.atInfo().log("Uploading unavailable names file to GCS in bucket %s", gcsBucket);
BlobId blobId = BlobId.of(gcsBucket, createFilename(runTime));
// `gcsUtils.openOutputStream` returns a buffered stream
try (OutputStream gcsOutput = gcsUtils.openOutputStream(blobId);
Writer osWriter = new OutputStreamWriter(gcsOutput, US_ASCII)) {
osWriter.write(unavailableDomains);
for (var domainName : unavailableDomains) {
osWriter.write(domainName);
osWriter.write("\n");
}
return true;
} catch (Exception e) {
logger.atSevere().withCause(e).log(
@@ -150,10 +160,14 @@ public class UploadBsaUnavailableDomainsAction implements Runnable {
}
}
boolean uploadToBsa(String unavailableDomains, DateTime runTime) {
boolean uploadToBsa(ImmutableSortedSet<String> unavailableDomains, DateTime runTime) {
try {
byte[] gzippedContents = gzipUnavailableDomains(unavailableDomains);
String sha512Hash = ByteSource.wrap(gzippedContents).hash(Hashing.sha512()).toString();
Hasher sha512Hasher = Hashing.sha512().newHasher();
unavailableDomains.stream()
.map(name -> name + "\n")
.forEachOrdered(line -> sha512Hasher.putString(line, UTF_8));
String sha512Hash = sha512Hasher.hash().toString();
String filename = createFilename(runTime);
OkHttpClient client = new OkHttpClient().newBuilder().build();
@@ -169,7 +183,9 @@ public class UploadBsaUnavailableDomainsAction implements Runnable {
.addFormDataPart(
"file",
String.format("%s.gz", filename),
RequestBody.create(gzippedContents, MediaType.parse("application/octet-stream")))
new StreamingRequestBody(
gzippedStream(unavailableDomains),
MediaType.parse("application/octet-stream")))
.build();
Request request =
@@ -196,15 +212,6 @@ public class UploadBsaUnavailableDomainsAction implements Runnable {
}
}
private byte[] gzipUnavailableDomains(String unavailableDomains) throws IOException {
try (ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream()) {
try (GZIPOutputStream gzipOutputStream = new GZIPOutputStream(byteArrayOutputStream)) {
gzipOutputStream.write(unavailableDomains.getBytes(US_ASCII));
}
return byteArrayOutputStream.toByteArray();
}
}
private static String createFilename(DateTime runTime) {
return String.format("unavailable_domains_%s.txt", runTime.toString());
}
@@ -280,4 +287,65 @@ public class UploadBsaUnavailableDomainsAction implements Runnable {
private static String toDomain(String domainLabel, Tld tld) {
return String.format("%s.%s", domainLabel, tld.getTldStr());
}
private InputStream gzippedStream(ImmutableSortedSet<String> unavailableDomains)
throws IOException {
PipedInputStream inputStream = new PipedInputStream();
PipedOutputStream outputStream = new PipedOutputStream(inputStream);
new Thread(
() -> {
try {
gzipUnavailableDomains(outputStream, unavailableDomains);
} catch (Throwable e) {
logger.atSevere().withCause(e).log("Failed to gzip unavailable domains.");
try {
// This will cause the next read to throw an IOException.
inputStream.close();
} catch (IOException ignore) {
// Won't happen for `PipedInputStream.close()`
}
}
})
.start();
return inputStream;
}
private void gzipUnavailableDomains(
PipedOutputStream outputStream, ImmutableSortedSet<String> unavailableDomains)
throws IOException {
// `GZIPOutputStream` is buffered.
try (GZIPOutputStream gzipOutputStream = new GZIPOutputStream(outputStream)) {
for (String name : unavailableDomains) {
var line = name + "\n";
gzipOutputStream.write(line.getBytes(US_ASCII));
}
}
}
private static class StreamingRequestBody extends RequestBody {
private final BufferedInputStream inputStream;
private final MediaType mediaType;
StreamingRequestBody(InputStream inputStream, MediaType mediaType) {
this.inputStream = new BufferedInputStream(inputStream);
this.mediaType = mediaType;
}
@Nullable
@Override
public MediaType contentType() {
return mediaType;
}
@Override
public void writeTo(@NotNull BufferedSink bufferedSink) throws IOException {
byte[] buffer = new byte[2048];
int bytesRead;
while ((bytesRead = inputStream.read(buffer)) != -1) {
bufferedSink.write(buffer, 0, bytesRead);
}
}
}
}

View File

@@ -0,0 +1,661 @@
<?xml version="1.0" encoding="utf-8"?>
<lgr xmlns="urn:ietf:params:xml:ns:lgr-1.0">
<meta>
<version comment="Latin LGR">1</version>
<date>2025-10-01</date>
<language>und-Latn</language>
<unicode-version>2</unicode-version>
<description type="text/html"><![CDATA[
<div class="instructions">
<h2>INSTRUCTIONS</h2>
<ul>
<li>These instructions cover how to adopt an LGR based on this reference LGR for a given
zone and how to prepare the file for deposit in the IANA Repository of IDN Practices.</li>
<li>As described the IANA procedure (https://www.iana.org/help/idn-repository-procedure) an
LGR MUST contain the following elements in its header:
<ul style="list-style-type:square;">
<li>Script or Language Designator (see below for guidance) </li>
<li>Version Number (this must increase with each amendment to the LGR, even if the updates
are limited to the header itself) </li>
<li>Effective Date (the date at which the policy becomes applicable in operational use) </li>
<li>Registry Contact Details (contact name, email address, and/or phone number)</li>
</ul>
</li>
<li>The following information is optional:
<ul style="list-style-type:square;">
<li>Document creation date</li>
<li>Applicable Domain(s)</li>
<li>Changes made to the Reference LGR before adopting</li>
</ul>
</li>
</ul>
<p>Please add or modify the following items in the <b>XML source code for this file</b> before
depositing the document in the IANA Repository. (https://www.iana.org/domains/idn-tables)</p>
<h3>Meta Data</h3>
<p>Note: version numbers start at 1. RFC 7940 recommends using simple integers. The version comment is optional,
please replace or delete the default comment. Version comments may be used by some tools as part of the page header.</p>
<p><code>&lt;version comment=&quot;</code>[Please replace (or delete) the optional comment]<code>&quot;&gt;</code>[Please fill in version number, starting at 1]<code>&lt;/version&gt;</code></p>
<p><code>&lt;date&gt;</code>2025-10-01<code>&lt;/date&gt;</code></p>
<p><code>&lt;validity-start&gt;</code>2025-10-01<code>&lt;/validity-start&gt;</code></p>
<p>Note: the scope element may be repeated, so that the same document can serve for multiple domains.</p>
<p><code>&lt;scope type=&quot;domain&quot;&gt;</code>[Please provide, in &quot;.domain&quot; format]<code>&lt;/scope&gt;</code></p>
<p><strong>Registry Contact Information:</strong></p>
<p>Please fill in the <a href="#registry_contact_details">Registry Contact Details</a>.</p>
<p><strong>Change History</strong></p>
<p>If you made technical modifications to the LGR, please summarize them in the <a href="#change_history">Change History</a> (and also note the details in the appropriate section of the description).</p>
<section id="registry_contact_details">
<h2>Registry Contact Details</h2>
<ul style="list-style:none;">
<li><b>Contact Name:</b> Ben McIlwain</li>
<li><b>Email address:</b> nomulus-discuss@google.com</li>
</ul>
</section>
<h1>Label Generation Rules for the Latin Script</h1>
<h2>Overview</h2>
<p>This document specifies a set of Label Generation Rules (LGR) for the Latin script for the second level domain or domains identified above.
The starting point for the development of this LGR can be found in the related Root Zone LGR [RZ-LGR-Latn].
The format of this file follows [RFC 7940].
This LGR is adapted from the “Reference LGR for the Second Level for the Latin Script” [Ref-LGR-und-Latn], for details, see <a href="#change_history">Change History</a> below.</p>
<p>For details and additional background on the Latin script, see “Proposal for a Latin Script Root Zone Label Generation Rule-Set (LGR)” [Proposal-Latin].</p>
<h2>Repertoire</h2>
<p>The repertoire contains the 164 letters needed to write hundreds of languages in the Latin script.
The repertoire is a subset of [Unicode 11.0.0]. For details, see Section 5, “Repertoire” in [Proposal-Latin].
(The proposal cited has been adopted for the Latin script portion of the Root Zone LGR.)</p>
<p>For the second level, the repertoire has been augmented with the ASCII digits, U+0030 to U+0039, plus U+002D HYPHEN-MINUS, for a total of 175 repertoire elements.</p>
<p>Any code points outside the Latin Script repertoire that are targets for
out-of-repertoire variants would be included here only if the variant is listed
in this file. In this case they are identified as a reflexive (identity) variant
of type “out-of-repertoire-var”. Whether or not they are listed, they do not
form part of the repertoire.</p>
<p><b>Repertoire Listing:</b> Each code point or range is tagged with the script or scripts with which the code point is used and one or more other character categories. For each repertoire element,
one or more references document sufficient justification for inclusion in the repertoire; see the <a href="#ref_desc_sec_References">“References”</a> below.
For code points that are part of the repertoire, comments identify the languages using the code point along with their [EGIDS] level.</p>
<h3>Background on Script and Principal Languages Using It</h3>
<p>The Latin script is a major writing system of the world, and the most widely used in terms of
the number of languages and speakers, with circa 70% of the worlds readers and writers making use of
this script. From a list of 1,189 languages using the Latin script [Omniglot]
the 212 languages that were taken into consideration contain all 182 languages with [EGIDS]
level 1&ndash;4 together with many languages with EGIDS level 5, each spoken by more than
1 million estimated speakers. Altogether over 100 languages are cited here to justify
specific additions to the repertoire, but many other languages may also be written using
some subset of the repertoire of this LGR. In a few cases, code points were excluded in [MSR-5] due to
security concerns; for the affected languages, only a subrepertoire could be supported.
More details in Section 3, “Background on Script and Principal Languages Using It”
of [Proposal-Latin].</p>
<h2>Variants</h2>
<p>The variants defined in this LGR are limited to those required for use in zones not shared with any other script.
As such, this LGR does not define cross-script variants. However, using this LGR concurrently with any LGR for Armenian, Cyrillic, and Greek in the same zone will introduce some in-script variants due to cross-script variant transitivity. This will also create potential cross-script issues when used with the same LGRs.
For details, see Section 6, “Variants” in [Proposal-Latin].
Mitigation of these in-script and cross-script variants can be addressed by using the Common LGR.
For details, see Section 3, “Use of Multiple Reference LGRs in the Same Zone” in [Level-2-Overview].
In addition to variants defined by this LGR, the full variant information related to this script and added by concurrent use with the Armenian, Cyrillic, and Greek LGR(s) can be found
in the following LGR: [Ref-LGR-Latin-Full-Variant-Script]
</p>
<p>In particular, the Latin LGR contains a number of in-script variants resulting from transitivity with
variants needed if the LGR is used concurrently with other LGRs from the related scripts, primarily
due to variants with the Greek script.
These variant definitions are required when using this LGR together with the Common LGR in label processing,
but they also reflect a certain consistency in the approach to variants across typographically related scripts.
They can be removed if the LGR is used strictly as standalone.<p>
<p>All other in-script variants defined in this LGR largely follow the methodology defined in
Section 6, “Variants”, in [Proposal-Latin]. In a separate appendix that proposal identifies additional
candidates for visually confusable code points (see [Proposal-Latin-Appendices]). They provide data
on the basis of which additional variants, or fallback variants (see following) might be defined. However,
doing so would require additional review and analysis that has not been carried out for this version of the
LGR.</p>
<p>The LGR defines certain allocatable fallback variants as
described in Section 4.5.5 “Allocatable Fallback Variants” in [Level-2-Overview]. A fallback variant is a variant label that uses
substitute code points for code points or sequences not available (or not allowed) in some contexts, that would
otherwise be required for a linguistically accurate rendering of some label.</p>
<p>When “fallback” variants are defined, two labels may be allocated: a single label with the spelling preferred by the
applicant, plus a single fallback variant for that label.
The fallback exclusively uses the fallback characters for any characters for which fallbacks are defined, while the
“preferred” label may use any otherwise valid mix of code points.
If the fallback variant is the one applied for, no other variant label is allocatable.</p>
<p>An allocatable fallback variant exists for the following pairs where the second element of each pair is the fallback:</p>
<ul>
<h3>In-script Variant Mapping Types</h3>
<p>In each of the fallback variant pairs defined above, the mapping type from the first element to the second is of type
“fallback”, while the variant type for the other direction is “blocked”. In addition, the first element of each pair uses the
reflexive mapping “r-original”.
(By convention, the prefix “r-” marks a type used in a reflexive variant mapping, that is, it represents an instance
of the original code point at that location in a variant label, see Section 5.3.4 in [RFC 7940].)</p>
<p><b>Variant Disposition:</b> Except for limited exceptions for the fallback variants defined above, variants defined here result
in a variant label disposition of “blocked”.</p>
<p>The specification of variants in this LGR follows the guidelines in [RFC 8228].</p>
<h2>Character Classes</h2>
<p>This LGR does not define named character classes.</p>
<h2>Whole Label Evaluation (WLE) and Context rules</h2>
<h3>Default Whole Label Evaluation Rules and Actions</h3>
<p>By default, the LGR includes the rules and actions to implement the following restrictions mandated by the IDNA protocol. They are marked with &#x235F;.</p>
<ul>
<li><b>Hyphen Restrictions</b> &mdash; restrictions on the allowable placement of hyphens (no leading/ending hyphen
and no hyphen in positions 3 and 4). These restrictions are described in Section 4.2.3.1 of RFC 5891 [320].
They are implemented here as context rule on U+002D (-) HYPHEN-MINUS.</li>
<li><b>Leading Combining Marks</b> &mdash; restrictions on the allowable placement of combining marks
(no leading combining mark). This rule is described in Section 4.2.3.2 of RFC 5891 [320].</li>
</ul>
<h3>Latin-specific Rules</h3>
<h2>Actions</h2>
<h3>Default Actions</h3>
<p>This LGR includes the default actions for LGRs as well as the action needed to
invalidate labels with misplaced combining marks. They are marked with &#x235F;.
For a description see [RFC 7940].</p>
<p>Default actions that are
triggered by the LGR-specific variant types described above limit the “allocatable” variant
labels to those containing only “ss”, dotted “i” or hyphen variants, while
disallowing mixed use of “ss” and “ß”, Dotless i and “i”, or middle-dot and hyphen respectively, except
as in the original applied-for label.</p>
<p>Note that the mapping types for variants are not symmetric: they depend on which code point is considered
the source or the target in a given mapping. As specified in [RFC 7940], when mapping types are evaluated
code points in a label that are unchanged use the type of their “reflexive” mapping.
Per [RFC 7940] the actions are always applied one after the other, and the evaluation stops at the first
action that assigns a disposition to a given label.</p>
<h3>Script-specific Actions</h3>
<p>An action has been defined to invalidate labels containing the sequence U+00B7 U+006C U+00B7 which could otherwise result in two Ela Geminada sequences overlapping.</p>
<h2>Methodology and Contributors</h2>
<p>The LGR in this document has been adapted from the corresponding Reference LGR for the Second Level. The Second Level Reference LGR for the Latin Script was developed by Michel Suignard and Asmus Freytag, based on the Root Zone LGR for the Latin
script and information contained or referenced therein; see [RZ-LGR-Latn]. Suitable extensions for the second level have been applied according to the [Guidelines] and with community input.
The original proposal for a Root Zone LGR for the Latin script, that this LGR is based on, was developed by the Latin Generation Panel.
For more information on methodology and contributors to the underlying Root Zone LGR, see Sections 4 and 8 in [Proposal-Latin], as well as [RZ-LGR-Overview].</p>
<section id="change_history">
<h3>Changes from Version Dated 25 October 2024</h3>
<p>Adopted from the Second Level Reference LGR for the Latin Script [Ref-LGR-und-Latn] with security improvements implemented by removing confusable variants.</p>
</section>
<h2>References</h2>
<p>The following general references are cited in this document:</p>
<dl class="references">
<dt>[EGIDS]</dt>
<dd>Lewis and Simons, “EGIDS: Expanded Graded Intergenerational Disruption Scale,”
documented in [SIL-Ethnologue] and summarized here:
https://en.wikipedia.org/wiki/Expanded_Graded_Intergenerational_Disruption_Scale_(EGIDS)</dd>
<dt>[Guidelines]</dt>
<dd>ICANN, “Guidelines for Developing Reference LGRs for the Second Level”, (Los Angeles, California: ICANN, 27 May 2020), https://www.icann.org/en/system/files/files/lgr-guidelines-second-level-27may20-en.pdf</dd>
<dt>[Level-2-Overview]</dt>
<dd>Internet Corporation for Assigned Names and Numbers, (ICANN),“Reference Label Generation Rules (LGR) for the Second Level: Overview and Summary” (PDF),
(Los Angeles, California: ICANN, 25 October 2024), https://www.icann.org/en/system/files/files/level2-lgr-overview-summary-25oct24-en.pdf
</dd>
<dt>[MSR-5]</dt>
<dd>Integration Panel, “Maximal Starting Repertoire — MSR-5 Overview and Rationale”, 24 June 2021,
https://www.icann.org/en/system/files/files/msr-5-overview-24jun21-en.pdf</dd>
<dt>[Omniglot]</dt>
<dd>Omniglot, “Writing Systems by Language”, https://www.omniglot.com/writing/langalph.htm (accessed on 13 January 2022)</dd>
<dt>[Proposal-Latin]</dt>
<dd>Latin Generation Panel, “Proposal for a Latin Script Root Zone Label Generation Rule-Set (LGR)”, 27 January 2022 (PDF), https://www.icann.org/en/system/files/files/proposal-latin-lgr-27jan22-en.pdf</dd>
<dt>[Proposal-Latin-Appendices]</dt>
<dd>Appendices to “Proposal for a Latin Script Root Zone Label Generation Rule-Set (LGR)”,
27 January 2022 (ZIP), https://www.icann.org/en/system/files/files/proposal-latin-lgr-appendices-27jan22-en.zip</dd>
<dt>[Ref-LGR-und-Latn]</dt>
<dd>ICANN, Second Level Reference Label Generation Rules for the Latin Script (und-Latn), 25 October 2024 (XML)
https://www.icann.org/sites/default/files/packages/lgr/lgr-second-level-latin-script-25oct24-en.xml
non-normative HTML presentation: https://www.icann.org/sites/default/files/packages/lgr/lgr-second-level-latin-script-25oct24-en.html</dd>
<dt>[Ref-LGR-Latin-Full-Variant-Script]</dt>
<dd>ICANN, Second Level Reference Label Generation Rules for the Latin Script (und-Latn), 25 October 2024 (XML)
https://www.icann.org/sites/default/files/packages/lgr/lgr-second-level-latin-full-variant-script-25oct24-en.xml
non-normative HTML presentation: https://www.icann.org/sites/default/files/packages/lgr/lgr-second-level-latin-full-variant-script-25oct24-en.html</dd>
<dt>[RFC 7940]</dt>
<dd>Davies, K. and A. Freytag, “Representing Label Generation Rulesets Using XML”,
RFC 7940, August 2016, https://www.rfc-editor.org/info/rfc7940</dd>
<dt>[RFC 8228]</dt>
<dd>A. Freytag, “Guidance on Designing Label Generation Rulesets (LGRs) Supporting Variant Labels”, RFC 8228, August 2017,
https://www.rfc-editor.org/info/rfc8228</dd>
<dt>[RZ-LGR-Overview]</dt>
<dd>Integration Panel, “Root Zone Label Generation Rules (RZ LGR-5): Overview and Summary”, 26 May 2022 (PDF), https://www.icann.org/sites/default/files/lgr/rz-lgr-5-overview-26may22-en.pdf</dd>
<dt>[RZ-LGR-5]</dt>
<dd>Integration Panel, “Root Zone Label Generation Rules (RZ-LGR-5)”, 26 May 2022 (XML), https://www.icann.org/sites/default/files/lgr/rz-lgr-5-common-26may22-en.xml <br/>
<i>non-normative HTML presentation: https://www.icann.org/sites/default/files/lgr/rz-lgr-5-common-26may22-en.html</i></dd>
<dt>[RZ-LGR-Latn]</dt>
<dd>ICANN, Root Zone Label Generation Rules for the Latin Script (und-Latn), 26 May 2022 (XML)
https://www.icann.org/sites/default/files/lgr/rz-lgr-5-latin-script-26may22-en.xml</dd>
<dt>[SIL-Ethnologue]</dt>
<dd>David M. Eberhard, Gary F. Simons &amp; Charles D. Fennig (eds.). 2021.
Ethnologue: Languages of the World, Twenty fourth edition. Dallas, Texas: SIL
International. Online version available as https://www.ethnologue.com</dd>
<dt>[Unicode 11.0.0]</dt>
<dd>The Unicode Consortium. The Unicode Standard, Version 11.0.0, (Mountain View, CA: The Unicode Consortium, 2018. ISBN 978-1-936213-19-1)
https://www.unicode.org/versions/Unicode11.0.0/</dd>
</dl>
<p>For references consulted particularly in designing the repertoire for the Latin Script for the second level
please see details in the <a href="#table_of_references">Table of References</a> below.</p>
<p>References [0] and up refer to the Unicode Standard versions in which corresponding code points
were initially encoded. References [101] and up correspond to a source given in [Proposal-Latin] for justifying
the inclusion of the corresponding code points. In the listing of <a href="#whole_label_evaluation_and_context_rules">whole label evaluation and context rules</a>,
reference [320] indicates the source for common rules.</p>
]]></description>
<references>
<reference id="0" comment="Any code point originally encoded in Unicode Version 1.1">The Unicode Standard, Version 1.1</reference>
<reference id="3" comment="Any code point originally encoded in Unicode Version 3.0">The Unicode Standard, Version 3.0</reference>
<reference id="8" comment="Any code point originally encoded in Unicode Version 5.0">The Unicode Standard, Version 5.0</reference>
<reference id="99" comment="Any code point cited is part of the Basic Latin (ASCII) set">C0 Controls and Basic Latin, The Unicode Standard https://unicode.org/charts/PDF/U0000.pdf</reference>
<reference id="100">ICANN, Second Level Reference Label Generation Rules for Spanish https://www.icann.org/sites/default/files/packages/lgr/lgr-second-level-spanish-30aug16-en.html (Accessed on 31 August 2018)</reference>
<reference id="101">Omniglot, Czech (čeština) https://www.omniglot.com/writing/czech.htm (Accessed on 31 August 2018)</reference>
<reference id="102">Omniglot, Icelandic (Íslenska) https://www.omniglot.com/writing/icelandic.htm (Accessed on 31 August 2018)</reference>
<reference id="103">Omniglot, Faroese (føroyskt mál) https://www.omniglot.com/writing/faroese.htm (Accessed on 31 August 2018)</reference>
<reference id="105">Omniglot, Chuukese (Chuuk) https://www.omniglot.com/writing/chuukese.htm (Accessed on 31 August 2018)</reference>
<reference id="106">ScriptSource, Galician written with Latin script https://scriptsource.org/cms/scripts/page.php?item_id=wrSys_detail&amp;key=gl-Latn (Accessed on 1 May 2023)</reference>
<reference id="107">Omniglot, Lule Sámi (julevsámegiella) https://www.omniglot.com/writing/lulesami.htm (Accessed on 31 August 2018)</reference>
<reference id="108">Wikipedia, Northern Sami https://en.wikipedia.org/wiki/Northern_Sami (Accessed on 4 September 2018)</reference>
<reference id="109">Omniglot, Vietnamese (tiếng việt / 㗂越) https://www.omniglot.com/writing/vietnamese.htm (Accessed on 4 September 2018)</reference>
<reference id="110">Omniglot, Romanian (limba română) https://www.omniglot.com/writing/romanian.htm (Accessed on 4 September 2018)</reference>
<reference id="113">Omniglot, Skolt Sámi (Sääˊmǩiõll / Nuõrttsääm) https://www.omniglot.com/writing/skoltsami.htm (Accessed on 4 September 2018)</reference>
<reference id="114">Omniglot, French (français) https://www.omniglot.com/writing/french.htm (Accessed on 4 September 2018)</reference>
<reference id="115">Omniglot, West Frisian (Frysk) https://www.omniglot.com/writing/westfrisian.htm (Accessed on 4 September 2018)</reference>
<reference id="116">Omniglot, Friulian (furlan/marilenghe) https://www.omniglot.com/writing/friulian.htm (Accessed on 4 September 2018)</reference>
<reference id="117">Summer Institute of Linguistics, Pequeno dicionário: Xavante-Português, Português-Xavante https://www.sil.org/resources/archives/17019 (Accessed on 1 October 2020)</reference>
<reference id="119">Omniglot, German (Deutsch) https://www.omniglot.com/writing/german.htm (Accessed on 4 September 2018)</reference>
<reference id="120">Omniglot, Finnish (suomi) https://www.omniglot.com/writing/finnish.htm (Accessed on 4 September 2018)</reference>
<reference id="121">Omniglot, Turkmen (Türkmen dili / Түркмен дили) https://www.omniglot.com/writing/turkmen.htm (Accessed on 4 September 2018)</reference>
<reference id="122">Omniglot, Estonian (eesti keel) https://www.omniglot.com/writing/estonian.htm (Accessed on 4 September 2018)</reference>
<reference id="123">Omniglot, Swedish (svenska) https://www.omniglot.com/writing/swedish.htm (Accessed on 4 September 2018)</reference>
<reference id="124">Omniglot, Yapese (Waab) https://www.omniglot.com/writing/yapese.htm (Accessed on 4 September 2018)</reference>
<reference id="125">Omniglot, Dinka (Thuɔŋjäŋ) https://www.omniglot.com/writing/dinka.php (Accessed on 4 September 2018)</reference>
<reference id="126">Omniglot, Kaqchikel (Kaqchikel Chabäl) https://www.omniglot.com/writing/kaqchikel.htm (Accessed on 4 September 2018)</reference>
<reference id="127">Omniglot, Bashkir/Bashkort (Башҡорт теле / Başqort tele) https://www.omniglot.com/writing/bashkir.htm (Accessed on 4 September 2018)</reference>
<reference id="128">Omniglot, Alsatian (Ëlsässisch) https://www.omniglot.com/writing/alsatian.htm (Accessed on 4 September 2018)</reference>
<reference id="129">Wikipedia, Nuer language https://en.wikipedia.org/wiki/Nuer_language (Accessed on 4 September 2018)</reference>
<reference id="130">Omniglot, Italian (italiano) https://www.omniglot.com/writing/italian.htm (Accessed on 4 September 2018)</reference>
<reference id="131">Wikipedia, Italian orthography https://en.wikipedia.org/wiki/Italian_orthography (Accessed on 4 September 2018)</reference>
<reference id="132">Omniglot, Wolof (Wollof) https://www.omniglot.com/writing/wolof.htm (Accessed on 4 September 2018)</reference>
<reference id="133">Omniglot, Latvian (latviešu valoda) https://www.omniglot.com/writing/latvian.htm (Accessed on 4 September 2018)</reference>
<reference id="134">Omniglot, Tongan (Faka-Tonga) https://www.omniglot.com/writing/tongan.htm (Accessed on 4 September 2018)</reference>
<reference id="135">Omniglot, Hawaiian (ʻŌlelo Hawaiʻi) https://www.omniglot.com/writing/hawaiian.htm (Accessed on 4 September 2018)</reference>
<reference id="136">Omniglot, Marshallese (kajin m̧ajeļ) https://www.omniglot.com/writing/marshallese.php (Accessed on 4 September 2018)</reference>
<reference id="137">Omniglot, Polish (polski) https://www.omniglot.com/writing/polish.htm (Accessed on 4 September 2018)</reference>
<reference id="138">Omniglot, Lithuanian (lietuvių kalba) https://www.omniglot.com/writing/lithuanian.htm (Accessed on 4 September 2018)</reference>
<reference id="139">Omniglot, Danish (dansk) https://www.omniglot.com/writing/danish.htm (Accessed on 4 September 2018)</reference>
<reference id="140">Omniglot, Chamorro (chamoru) https://www.omniglot.com/writing/chamorro.htm (Accessed on 4 September 2018)</reference>
<reference id="141">Omniglot, Umbundu (Úmbúndú) https://www.omniglot.com/writing/umbundu.htm (Accessed on 4 September 2018)</reference>
<reference id="142">Omniglot, Guaraní (Avañeẽ) https://www.omniglot.com/writing/guarani.htm (Accessed on 4 September 2018)</reference>
<reference id="143">Wikipedia, Guarani alphabet https://en.wikipedia.org/wiki/Guarani_alphabet (Accessed on 4 September 2018)</reference>
<reference id="144">Omniglot, Nauruan (Ekaiairũ Naoero) https://www.omniglot.com/writing/nauruan.htm (Accessed on 4 September 2018)</reference>
<reference id="145">Omniglot, Khoekhoe (Khoekhoegowab) https://www.omniglot.com/writing/khoekhoe.htm (Accessed on 4 September 2018)</reference>
<reference id="146">Omniglot, Nuer (Naath) https://www.omniglot.com/writing/nuer.htm (Accessed on 4 September 2018)</reference>
<reference id="147">Omniglot, Hausa (Harshen Hausa / هَرْشَن هَوْسَ) https://www.omniglot.com/writing/hausa.htm (Accessed on 4 September 2018)</reference>
<reference id="148">Omniglot, Dagaare https://www.omniglot.com/writing/dagaare.htm (Accessed on 4 September 2018)</reference>
<reference id="149">Omniglot, Fula (Fulfulde, Pulaar, PularFulaare) https://www.omniglot.com/writing/fula.htm (Accessed on 4 September 2018)</reference>
<reference id="150">Omniglot, Croatian (Hrvatski) https://www.omniglot.com/writing/croatian.htm (Accessed on 4 September 2018)</reference>
<reference id="151">Omniglot, Serbian (српски / srpski) https://www.omniglot.com/writing/serbian.htm (Accessed on 4 September 2018)</reference>
<reference id="152">Wikipedia, Polish language https://en.wikipedia.org/wiki/Polish_language (Accessed on 4 September 2018)</reference>
<reference id="153">Omniglot, Slovak (slovenčina) https://www.omniglot.com/writing/slovak.htm (Accessed on 4 September 2018)</reference>
<reference id="154">Evertype Publishing, Lithuanian lietuvių kalba Version 1.1 https://www.evertype.com/alphabets/lithuanian.pdf (Accessed on 4 September 2018)</reference>
<reference id="157">Omniglot, Turkish (Türkçe) https://www.omniglot.com/writing/turkish.htm (Accessed on 4 September 2018)</reference>
<reference id="158">Omniglot, Kurdish (Kurdî / کوردی) https://www.omniglot.com/writing/kurdish.htm (Accessed on 4 September 2018)</reference>
<reference id="159">Omniglot, Azerbaijani (آذربايجانجا ديلي / Азәрбајҹан дили / Azərbaycan dili) https://www.omniglot.com/writing/azeri.htm (Accessed on 4 September 2018)</reference>
<reference id="160">Omniglot, Basque (euskara) https://www.omniglot.com/writing/basque.htm (Accessed on 4 September 2018)</reference>
<reference id="161">Wikipedia, Basque language https://en.wikipedia.org/wiki/Basque_language#Writing_system (Accessed on 4 September 2018)</reference>
<reference id="163">Omniglot, Maltese (Malti) https://www.omniglot.com/writing/maltese.htm (Accessed on 4 September 2018)</reference>
<reference id="164">Omniglot, Venda (Tshivenḓa / Luvenḓa) https://www.omniglot.com/writing/venda.htm (Accessed on 4 September 2018)</reference>
<reference id="168">Omniglot, Brahui (Bráhuí / براوی) https://www.omniglot.com/writing/brahui.htm (Accessed on 4 September 2018)</reference>
<reference id="169">Wikipedia, Fon language https://en.wikipedia.org/wiki/Fon_language (Accessed on 4 September 2018)</reference>
<reference id="170">Omniglot, Ewe (Eʋegbe) https://www.omniglot.com/writing/ewe.htm (Accessed on 4 September 2018)</reference>
<reference id="172">Omniglot, Sorbian (hornjoserbsce/dolnoserbski) https://www.omniglot.com/writing/sorbian.htm (Accessed on 4 September 2018)</reference>
<reference id="173">Peace corps, Botswana, An Introduction to Setswana Language https://files.peacecorps.gov/multimedia/audio/languagelessons/botswana/Bw_Setswana_Language_Lessons.pdf (Accessed on 4 September 2018)</reference>
<reference id="174">Omniglot, Tswana (Setswana) https://www.omniglot.com/writing/tswana.php (Accessed on 4 September 2018)</reference>
<reference id="175">Wikipedia, Afrikaans https://en.wikipedia.org/wiki/Afrikaans (Accessed on 4 September 2018)</reference>
<reference id="176">Omniglot, Albanian (shqip / gjuha shqipe) https://www.omniglot.com/writing/albanian.htm (Accessed on 4 September 2018)</reference>
<reference id="177">Wikipedia, Albanian alphabet https://en.wikipedia.org/wiki/Albanian_alphabet (Accessed on 4 September 2018)</reference>
<reference id="179">Wikipedia, Uyghur Latin alphabet https://en.wikipedia.org/wiki/Uyghur_Latin_alphabet (Accessed on 4 September 2018)</reference>
<reference id="180">Omniglot, Drehu (Deʼu) https://www.omniglot.com/writing/drehu.php (Accessed on 4 September 2018)</reference>
<reference id="182">Omniglot, Haitian Creole (Kreyòl ayisyen) https://www.omniglot.com/writing/haitiancreole.htm (Accessed on 4 September 2018)</reference>
<reference id="183">Wikipedia, Haitian Creole https://en.wikipedia.org/wiki/Haitian_Creole#Orthography (Accessed on 4 September 2018)</reference>
<reference id="184">Omniglot, Minangkabau (Baso Minangkabau / باسو مينڠكاباو) https://www.omniglot.com/writing/minangkabau.htm (Accessed on 4 September 2018)</reference>
<reference id="185">Omniglot, Palauan (a tekoi er a Belau) https://www.omniglot.com/writing/palauan.htm (Accessed on 4 September 2018)</reference>
<reference id="186">Omniglot, Cubeo (pãmié) https://www.omniglot.com/writing/cubeo.htm (Accessed on 4 September 2018)</reference>
<reference id="187">Editorial Alberto Lleras Camargo, Diccionario Ilustrado Bilingüe cubeo-español español-cubeo https://www.sil.org/system/files/reapdata/10/58/27/10582785843693992331766506069073895620/40337_01.pdf (Accessed on 4 September 2018)</reference>
<reference id="188">Omniglot, Inari Saami (Anarâškielâ) https://www.omniglot.com/writing/inarisami.htm (Accessed on 4 September 2018)</reference>
<reference id="189">Omniglot, Compiled by Wolfram Siegel, DAGBANI https://www.omniglot.com/charts/dagbani.pdf (Accessed on 4 September 2018)</reference>
<reference id="190">Omniglot, Ewondo https://www.omniglot.com/writing/ewondo.php (Accessed on 4 September 2018)</reference>
<reference id="191">Omniglot, Luganda (Oluganda) https://www.omniglot.com/writing/ganda.php (Accessed on 4 September 2018)</reference>
<reference id="192">Omniglot, Adzera https://www.omniglot.com/writing/adzera.htm (Accessed on 4 September 2018)</reference>
<reference id="193">Omniglot, Ga (Gã) https://www.omniglot.com/writing/ga.htm (Accessed on 4 September 2018)</reference>
<reference id="194">Omniglot, Duala (Duálá) https://www.omniglot.com/writing/duala.php (Accessed on 4 September 2018)</reference>
<reference id="195">Omniglot, Soga (Lusoga) https://www.omniglot.com/writing/soga.htm (Accessed on 4 September 2018)</reference>
<reference id="196">Omniglot, Alur (Lur) https://www.omniglot.com/writing/alur.htm (Accessed on 4 September 2018)</reference>
<reference id="197">Omniglot, Mandinka (Mandinka kango / لغة مندنكا) https://www.omniglot.com/writing/mandinka.htm (Accessed on 4 September 2018)</reference>
<reference id="198">Omniglot, Acholi (Lwo) https://www.omniglot.com/writing/acholi.htm (Accessed on 4 September 2018)</reference>
<reference id="199">Omniglot, Bambara (Bamanankan) https://www.omniglot.com/writing/bambara.htm (Accessed on 4 September 2018)</reference>
<reference id="200">Omniglot, Raga (Hano) https://www.omniglot.com/writing/raga.htm (Accessed on 4 September 2018)</reference>
<reference id="201">Omniglot, Tatar (tatarça / татарча / تاتارچا) https://www.omniglot.com/writing/tatar.htm (Accessed on 4 September 2018)</reference>
<reference id="202">Omniglot, Zaza (Zazaki / زازاکی) https://www.omniglot.com/writing/zazaki.htm (Accessed on 4 September 2018)</reference>
<reference id="203">Wikipedia, Turkish alphabet https://en.wikipedia.org/wiki/Turkish_alphabet (Accessed on 4 September 2018)</reference>
<reference id="204">School of English, Adam Michiewicz University, Poznań, Poland, Poznań Studies in Contemporary Linguistics 43(1),2007, pp. 169-180, A Demographic Igbo Orthography https://www.degruyter.com/downloadpdf/j/psicl.2007.43.issue-1/v10010-007-0009-0/v10010-007-0009-0.pdf (Accessed on 4 September 2018)</reference>
<reference id="205">Omniglot, Igbo (Asụsụ Igbo) https://www.omniglot.com/writing/igbo.htm (Accessed on 4 September 2018)</reference>
<reference id="206">ItalianPod101, Italian Accents and Proper Italian Pronunciation https://www.italianpod101.com/italian-accents (Accessed on 4 September 2018)</reference>
<reference id="208">Reverso Dictionary, venerdì translation | Italian-English dictionary https://dictionary.reverso.net/italian-english/venerd%C3%AC (Accessed on 4 September 2018)</reference>
<reference id="209">Omniglot, Kikuyu (Gĩkũyũ) https://www.omniglot.com/writing/kikuyu.htm (Accessed on 4 September 2018)</reference>
<reference id="210">Omniglot, Hixkaryána https://www.omniglot.com/writing/hixkaryana.htm (Accessed on 4 September 2018)</reference>
<reference id="211">Omniglot, Maasai (ɔl Maa) https://www.omniglot.com/writing/maasai.htm (Accessed on 4 September 2018)</reference>
<reference id="212">Omniglot, Mossi (Mòoré) https://www.omniglot.com/writing/mossi.htm (Accessed on 4 September 2018)</reference>
<reference id="213">Omniglot, Jenesis. The Bible in Marshallese, 2009., Contributed by Wolfgang Kuhl https://www.omniglot.com/babel/marshallese.htm (Accessed on 4 September 2018)</reference>
<reference id="214">Wikipedia, Cedilla https://en.wikipedia.org/wiki/Cedilla#Marshallese (Accessed on 4 September 2018)</reference>
<reference id="215">Wikipedia, Marshallese language https://en.wikipedia.org/wiki/Marshallese_language#Display_issues (Accessed on 4 September 2018)</reference>
<reference id="216">Trussel, Marshallese-English Online Dictionary https://www.trussel2.com/MOD/ (Accessed on 4 September 2018)</reference>
<reference id="218">Omniglot, Susu (Sosoxi) https://www.omniglot.com/writing/susu.htm (Accessed on 4 September 2018)</reference>
<reference id="219">Omniglot, Zarma (Zarmaciine) https://www.omniglot.com/writing/zarma.htm (Accessed on 4 September 2018)</reference>
<reference id="220">Omniglot, Pitjantjatjara https://www.omniglot.com/writing/pitjantjatjara.htm (Accessed on 4 September 2018)</reference>
<reference id="221">Omniglot, Spanish (español/castellano) https://www.omniglot.com/writing/spanish.htm (Accessed on 4 September 2018)</reference>
<reference id="222">Omniglot, Filipino (wikang Filipino) https://www.omniglot.com/writing/filipino.htm (Accessed on 4 September 2018)</reference>
<reference id="223">Omniglot, Chavacano https://www.omniglot.com/writing/chavacano.php (Accessed on 4 September 2018)</reference>
<reference id="224">Wikipedia, Ilocano language https://en.wikipedia.org/wiki/Ilocano_language#Modern_alphabet (Accessed on 4 September 2018)</reference>
<reference id="225">Omniglot, Quechua (Runasimi) https://www.omniglot.com/writing/quechua.htm (Accessed on 4 September 2018)</reference>
<reference id="226">Wikipedia, Quechua alphabet https://en.wikipedia.org/wiki/Quechua_alphabet (Accessed on 4 September 2018)</reference>
<reference id="227">Omniglot, Cape Verdean Creole (Kriolu) https://www.omniglot.com/writing/kriol.php (Accessed on 4 September 2018)</reference>
<reference id="228">Omniglot, Waray-Waray https://www.omniglot.com/writing/waray.php (Accessed on 4 September 2018)</reference>
<reference id="229">Omniglot, Lozi (siLozi) https://www.omniglot.com/writing/lozi.htm (Accessed on 4 September 2018)</reference>
<reference id="230">africanlanguages.com, Sesotho sa Leboa (Northern Sotho) https://africanlanguages.com/northern_sotho/ (Accessed on 4 September 2018)</reference>
<reference id="232">Wikipedia, Chechen language https://en.wikipedia.org/wiki/Chechen_language (Accessed on 4 September 2018)</reference>
<reference id="233">Omniglot, Hungarian (magyar) https://www.omniglot.com/writing/hungarian.htm (Accessed on 4 September 2018)</reference>
<reference id="234">Wikipedia, Hungarian alphabet https://en.wikipedia.org/wiki/Hungarian_alphabet (Accessed on 4 September 2018)</reference>
<reference id="236">Omniglot, Lingala https://www.omniglot.com/writing/lingala.htm (Accessed on 4 September 2018)</reference>
<reference id="237">Omniglot, Akan https://www.omniglot.com/writing/akan.htm (Accessed on 4 September 2018)</reference>
<reference id="238">Wikipedia, Mossi language https://en.wikipedia.org/wiki/Mossi_language (Accessed on 4 September 2018)</reference>
<reference id="239">SIL-Sudan, OCCASIONAL PAPERS in the study of SUDANESE LANGUAGES No. 9 (p. 75) https://www.sil.org/system/files/reapdata/10/06/46/100646256099282892829790816212446104791/OPSL_9.pdf (Accessed on 4 September 2018)</reference>
<reference id="240">Omniglot, Kanuri https://www.omniglot.com/writing/kanuri.htm (Accessed on 4 September 2018)</reference>
<reference id="241">Omniglot, Bugis (Basa Ugi ) https://www.omniglot.com/writing/bugis.htm (Accessed on 4 September 2018)</reference>
<reference id="242">Omniglot, Mizo (Mizo ṭawng) https://www.omniglot.com/writing/mizo.htm (Accessed on 4 September 2018)</reference>
<reference id="243">Omniglot, Miskito (Mískitu) https://www.omniglot.com/writing/miskito.htm (Accessed on 4 September 2018)</reference>
<reference id="245">Wikipedia, Papiamento https://en.wikipedia.org/wiki/Papiamento (Accessed on 4 September 2018)</reference>
<reference id="246">Omniglot, Papiamento (Papiamentu) https://www.omniglot.com/writing/papiamento.php (Accessed on 4 September 2018)</reference>
<reference id="247">Omniglot, Chichewa (Chicheŵa) https://www.omniglot.com/writing/chichewa.php (Accessed on 4 September 2018)</reference>
<reference id="248">Native Languages of the Americas website, Vocabulary in Native American Languages: Mam Words https://www.native-languages.org/mam_words.htm (Accessed on 4 September 2018)</reference>
<reference id="249">Omniglot, Mam (Qyol Mam) https://www.omniglot.com/writing/mam.htm (Accessed on 4 September 2018)</reference>
<reference id="250">Wikipedia, Pulaar language https://en.wikipedia.org/wiki/Pulaar_language (Accessed on 4 September 2018)</reference>
<reference id="251">Wikipedia, Fula language https://en.wikipedia.org/wiki/Fula_language#Writing_systems (Accessed on 4 September 2018)</reference>
<reference id="252">Wikipedia, Polish alphabet https://en.wikipedia.org/wiki/Polish_alphabet (Accessed on 4 September 2018)</reference>
<reference id="253">Wikipedia, French orthography https://en.wikipedia.org/wiki/French_orthography (Accessed on 4 September 2018)</reference>
<reference id="254">Omniglot, Yoruba (Èdè Yorùbá) https://www.omniglot.com/writing/yoruba.htm (Accessed on 4 September 2018)</reference>
<reference id="255">Omniglot, Esperanto https://www.omniglot.com/writing/esperanto.htm (Accessed on 4 September 2018)</reference>
<reference id="256">Omniglot, Welsh (Cymraeg) https://www.omniglot.com/writing/welsh.htm (Accessed on 4 September 2018)</reference>
<reference id="257">Wikipedia, List of Latin-script letters https://en.wikipedia.org/wiki/List_of_Latin-script_letters (Accessed on 4 September 2018)</reference>
<reference id="258">Omniglot, Montenegrin https://www.omniglot.com/writing/montenegrin.htm (Accessed on 20 March 2019)</reference>
<reference id="275">Omniglot, Shavante https://www.omniglot.com/writing/shavante.php (Accessed on 24 September 2020)</reference>
<reference id="276">Wikipedia, Malagasy Language https://en.wikipedia.org/wiki/Malagasy_language (Accessed on 24 September 2020)</reference>
<reference id="277">Wikipedia, Serer language, https://en.wikipedia.org/wiki/Serer_language, accessed on 6 April 2021</reference>
<reference id="278">Wikipedia, Kpelle language, https://en.wikipedia.org/wiki/Kpelle_language, accessed on 6 April 2021</reference>
<reference id="279">Wikipedia, Catalan Orthography, https://en.wikipedia.org/wiki/Catalan_orthography, accessed on 28 November 2022</reference>
<reference id="280">Fundacio PuntCAT registry (.CAT), IDN table for CAT_ca Version 1.0, 12 February 2006, https://www.iana.org/domains/idn-tables/tables/cat_ca_1.0.html</reference>
<reference id="281">Fundacio PuntCAT registry (.CAT), Rules of the .cat domain, https://domini.cat/en/rules-of-the-cat-domain/</reference>
<reference id="320">RFC 5891, Internationalized Domain Names in Applications (IDNA): Protocol https://tools.ietf.org/html/rfc5891</reference>
</references>
</meta>
<data>
<char cp="002D" not-when="hyphen-minus-disallowed" tag="sc:Zyyy" ref="0" comment="HYPHEN-MINUS; &#x235F;" />
<char cp="0030" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT ZERO; &#x235F;" />
<char cp="0031" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT ONE; &#x235F;" />
<char cp="0032" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT TWO; &#x235F;" />
<char cp="0033" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT THREE; &#x235F;" />
<char cp="0034" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT FOUR; &#x235F;" />
<char cp="0035" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT FIVE; &#x235F;" />
<char cp="0036" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT SIX; &#x235F;" />
<char cp="0037" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT SEVEN; &#x235F;" />
<char cp="0038" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT EIGHT; &#x235F;" />
<char cp="0039" tag="Common-digit sc:Zyyy" ref="0" comment="DIGIT NINE; &#x235F;" />
<char cp="0061" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0062" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0063" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0064" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0065" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0066" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0067" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0068" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0069" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="006A" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="006B" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="006C" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="006D" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="006E" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="006F" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0070" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0071" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0072" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0073" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0074" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0075" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0076" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0077" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0078" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="0079" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="007A" tag="sc:Latn" ref="0 99" comment="Basic Latin" />
<char cp="00E0" tag="sc:Latn" ref="0 106 114 130 131 132" comment="Italian (1), French (1), Galician (2), Wolof (4)" />
<char cp="00E2" tag="sc:Latn" ref="0 106 109 110 113 114 115 116 117 275" comment="Vietnamese (1), Romanian (1), Skolt Sami (2), French (1), Galician (2), West Frisian (1), Friulian (4), Xavante (4)" />
<char cp="00E3" tag="sc:Latn" ref="0 141 142 143 144 145" comment="Umbundu (3), Guarani (1), Nauruan (3), Khoekhoe (4)" />
<char cp="00E4" tag="sc:Latn" ref="0 107 119 120 121 122 123 124 125 126 127 128 129" comment="German (1), Finnish (1), Turkmen (1), Estonian (1), Swedish (1), Lule Sami (2), Yapese (2), Dinka (4), Kaqchikel (4), Bashkir (4), Alsatian (5), Nuer (4)" />
<char cp="00E5" tag="sc:Latn" ref="0 107 120 123 139 140" comment="Danish (1), Finnish (1), Chamorro (1), Swedish (1), Lule Sami (2)" />
<char cp="00E6" tag="sc:Latn" ref="0 102 103 139" comment="Danish (1), Icelandic (1), Faroese (2)" />
<char cp="00E7" tag="sc:Latn" ref="0 106 114 116 121 127 157 158 159 160 161" comment="Turkish (1), Turkmen (1), Kurdish (2), French (1), Azerbaijani (1), Basque (1), Galician (2), Friulian (4), Bashkir (4)" />
<char cp="00E8" tag="sc:Latn" ref="0 114 130 175 182 183" comment="French (1), Italian (1), Afrikaans (1), Haitian Creole (1), French (1)" />
<char cp="00E9" tag="sc:Latn" ref="0 100 101 102 105 106 114 115 117 130 132 275" comment="French (1), Italian (1), Spanish (1), Czech (1), Icelandic (1), Chuukese (2), Galician (2), Wolof (4), Xavante (4), West Frisian (2)" />
<char cp="00EA" tag="sc:Latn" ref="0 109 114 115 116 158 173 174 175" comment="French (1), Tswana (1), Afrikaans (1), Vietnamese (1), Kurdish (2), West Frisian (2), Friulian (4)" />
<char cp="00EB" tag="sc:Latn" ref="0 114 115 124 126 129 132 175 176 177 179 180" comment="Afrikaans (1), Albanian (1), French (1), Uyghur (2), Yapese (2), Wolof (4), Drehu (4), Kaqchikel (4), West Frisian (2), Nuer (4)" />
<char cp="00EC" tag="sc:Latn" ref="0 130 206 208" comment="Italian (1)" />
<char cp="00EE" tag="sc:Latn" ref="0 110 114 116 158 175" comment="Afrikaans (1), Romanian (1), Kurdish (2), French (1), Friulian (4)" />
<char cp="00F0" tag="sc:Latn" ref="0 102 103" comment="Faroese (2), Icelandic (1)" />
<char cp="00F2" tag="sc:Latn" ref="0 130 182 183" comment="Italian (1), Haitian Creole (1)" />
<char cp="00F4" tag="sc:Latn" ref="0 106 109 114 115 116 117 173 174 175 230 275" comment="Tswana (1), Afrikaans (1), Vietnamese (1), French (1), Northern Sotho (1), West Frisian (2), Galician (2), Friulian (4), Xavante (4)" />
<char cp="00F5" tag="sc:Latn" ref="0 113 117 122 141 142 143 144 145 275" comment="Estonian (1), Skolt Sami (2), Umbundu (3), Guarani (1), Nauruan (3), Xavante (4), Khoekhoe (4)" />
<char cp="00F6" tag="sc:Latn" ref="0 115 119 120 123 124 125 126 127 129 157 175 179 180 232" comment="German (1), Finnish (1), Afrikaans (1), Turkish (1), Swedish (1), Uygur (2), Yapese (2), Drehu (4), Kaqchikel (4), Dinka (4), Bashkir (4), Chechen (2), 1992 Version, West Frisian (2), Nuer (4)" />
<char cp="00F8" tag="sc:Latn" ref="0 103 139" comment="Danish (1), Faroese (2)" />
<char cp="00F9" tag="sc:Latn" ref="0 114 130 206 245 246 253" comment="Italian (1), French (1), Papiamento (1)" />
<char cp="00FB" tag="sc:Latn" ref="0 114 115 116 158 175 202 243" comment="Afrikaans (1), Kurdish (2), French (1), Miskito (2), West Frisian (2), Friulian (4), Zazaki (4)" />
<char cp="00FD" tag="sc:Latn" ref="0 101 102 103 121 142 143" comment="Turkmen (1), Czech (1), Icelandic (1), Faroese (2), Guarani (1)" />
<char cp="00FE" tag="sc:Latn" ref="0 102" comment="Icelandic (1)" />
<char cp="00FF" tag="sc:Latn" ref="0 114 253 257" comment="French (1)" />
<char cp="0103" tag="sc:Latn" ref="0 109 110" comment="Vietnamese (1), Romanian (1)" />
<char cp="0105" tag="sc:Latn" ref="0 137 138" comment="Polish (1), Lithuanian (1)" />
<char cp="0107" tag="sc:Latn" ref="0 150 151 152" comment="Croatian (1), Serbian (1), Polish (1)" />
<char cp="0109" tag="sc:Latn" ref="0 255" comment="Esperanto (3)" />
<char cp="010D" tag="sc:Latn" ref="0 108 133 150 151 153 154" comment="Croatian (1), Serbian (1), Latvian (1), Slovak (1), Northern Sami (2), Lithuanian (1)" />
<char cp="010F" tag="sc:Latn" ref="0 101 153" comment="Czech (1), Slovak (1)" />
<char cp="0111" tag="sc:Latn" ref="0 108 109 150 151 168" comment="Croatian (1), Serbian (1), Vietnamese (1), Northern Sami (2), Brahui (5)" />
<char cp="0113" tag="sc:Latn" ref="0 133 134 135 184" comment="Latvian (1), Hawaiian (2), Tongan (1), Minangkabau (5)" />
<char cp="0117" tag="sc:Latn" ref="0 138 154" comment="Lithuanian (1)" />
<char cp="0119" tag="sc:Latn" ref="0 138 152 154 185" comment="Polish (1), Palauan (2), Lithuanian (1)" />
<char cp="011B" tag="sc:Latn" ref="0 101 172" comment="Czech (1), Sorbian (4)" />
<char cp="011D" tag="sc:Latn" ref="0 255" comment="Esperanto (3)" />
<char cp="011F" tag="sc:Latn" ref="0 127 157 159 201 202" comment="Turkish (1), Tatar (2), Azeri (1), Bashkir (4), Zaza (5)" />
<char cp="0123" tag="sc:Latn" ref="0 133 168" comment="Latvian (1), Brahui (5)" />
<char cp="0125" tag="sc:Latn" ref="0 255" comment="Esperanto (3)" />
<char cp="0127" tag="sc:Latn" ref="0 163" comment="Maltese (1)" />
<char cp="012B" tag="sc:Latn" ref="0 133 134 135 138" comment="Latvian (1), Lithuanian (1), Hawaiian (2), Tongan (1)" />
<char cp="012F" tag="sc:Latn" ref="0 154" comment="Lithuanian (1)" />
<char cp="0135" tag="sc:Latn" ref="0 255" comment="Esperanto (3)" />
<char cp="0137" tag="sc:Latn" ref="0 133" comment="Latvian (1)" />
<char cp="013A" tag="sc:Latn" ref="0 153" comment="Slovak (1)" />
<char cp="013C" tag="sc:Latn" ref="0 133 168 213 214" comment="Latvian (1), Marshallese (1), Brahui (5)" />
<char cp="013E" tag="sc:Latn" ref="0 153" comment="Slovak (1)" />
<char cp="0142" tag="sc:Latn" ref="0 152" comment="Polish (1)" />
<char cp="0146" tag="sc:Latn" ref="0 133 136" comment="Latvian (1), Marshallese (1)" />
<char cp="0148" tag="sc:Latn" ref="0 101 121 153" comment="Turkmen (1), Czech (1), Slovak (1)" />
<char cp="0151" tag="sc:Latn" ref="0 233 234" comment="Hungarian (1)" />
<char cp="0153" tag="sc:Latn" ref="0 114 253" comment="French (1)" />
<char cp="0155" tag="sc:Latn" ref="0 153 168" comment="Slovak (1), Brahui (5)" />
<char cp="0159" tag="sc:Latn" ref="0 101 172" comment="Czech (1), Sorbian (4)" />
<char cp="015B" tag="sc:Latn" ref="0 152 258" comment="Polish (1), Montenegrin (1)" />
<char cp="015D" tag="sc:Latn" ref="0 255" comment="Esperanto (3)" />
<char cp="015F" tag="sc:Latn" ref="0 121 127 157 158 159 168 201 202" comment="Turkish (1), Turkmen (1), Kurdish (2), Tatar (2), Azeri (1), Bashkir (4), Brahui (5), Zaza (5)" />
<char cp="0161" tag="sc:Latn" ref="0 108 133 150 151 154 174 230" comment="Tswana (1), Croatian (1), Serbian (1), Latvian (1), Northern Sotho (1), Northern Sami (2), Lithuanian (1)" />
<char cp="0165" tag="sc:Latn" ref="0 101 153" comment="Czech (1), Slovak (1)" />
<char cp="0167" tag="sc:Latn" ref="0 108 168" comment="Northern Sami (2), Brahui (5)" />
<char cp="016B" tag="sc:Latn" ref="0 133 134 135 136 138 154" comment="Latvian (1), Hawaiian (2), Lithuanian (1), Marshallese (1), Tongan (1)" />
<char cp="016D" tag="sc:Latn" ref="0 255" comment="Esperanto (3)" />
<char cp="016F" tag="sc:Latn" ref="0 101" comment="Czech (1)" />
<char cp="0171" tag="sc:Latn" ref="0 233 234" comment="Hungarian (1)" />
<char cp="0173" tag="sc:Latn" ref="0 138 154" comment="Lithuanian (1)" />
<char cp="0175" tag="sc:Latn" ref="0 247 256" comment="Chichewa (3), Welsh (2)" />
<char cp="0177" tag="sc:Latn" ref="0 256" comment="Welsh (2)" />
<char cp="017A" tag="sc:Latn" ref="0 152 168 172 252 258" comment="Polish (1), Brahui (5), Sorbian (4), Montenegrin (1)" />
<char cp="017E" tag="sc:Latn" ref="0 108 121 133 150 151 153 154 232" comment="Lithuanian (1), Croatian (1), Serbian (1), Turkmen (1), Latvian (1), Slovak (1), Northern Sami (2), Chechen (2) 1925 Version" />
<char cp="0188" tag="sc:Latn" ref="0 277" comment="Serer (5)" />
<char cp="0199" tag="sc:Latn" ref="0 147" comment="Hausa (2)" />
<char cp="01A1" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="01A5" tag="sc:Latn" ref="0 277" comment="Serer (5)" />
<char cp="01AD" tag="sc:Latn" ref="0 277" comment="Serer (5)" />
<char cp="01B0" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="01B4" tag="sc:Latn" ref="0 148 149 251" comment="Dagaare - Burkina Faso (4), Fula (3)" />
<char cp="01E9" tag="sc:Latn" ref="0 113" comment="Skolt Sami (2)" />
<char cp="01EF" tag="sc:Latn" ref="0 113" comment="Skolt Sami (2)" />
<char cp="0219" tag="sc:Latn" ref="3 110" comment="Romanian (1)" />
<char cp="021B" tag="sc:Latn" ref="3 110" comment="Romanian (1)" />
<char cp="024D" tag="sc:Latn" ref="8 240" comment="Kanuri (3)" />
<char cp="0253" tag="sc:Latn" ref="0 147 148 250" comment="Hausa (2), Dagaare - Burkina Faso (4), Pulaar (3)" />
<char cp="0254" tag="sc:Latn" ref="0 129 146 148 169 170 189 190 193 194 236 237" comment="Dagaare - Burkina Faso (4), Dagbani (Dagomba) (4), Lingala (2), Akan (3), Ewondo (3), Fon (3), Nuer (4), Ga (4), Duala (3), EWE (3), Nuer (4)" />
<char cp="0256" tag="sc:Latn" ref="0 169 170" comment="Fon (3), Ewe (3)" />
<char cp="0257" tag="sc:Latn" ref="0 147 149 250" comment="Hausa (2), Fula (3)" />
<char cp="0259" tag="sc:Latn" ref="0 159 170 190 241" comment="Azeri, Azerbaijani (1), Ewondo (3), Ewe (3), Bugis (3)" />
<char cp="025B" tag="sc:Latn" ref="0 129 148 169 170 189 190 193 194 199 212 236 237 238" comment="Dagaare - Burkina Faso (4), Lingala (2), Akan (3), Ewondo (3), Dagbani (Dagomba), (4), Fon (3), Mossi (3), Ga (4), Ewe (3), Duala (3), Bambara (4), Nuer (4)" />
<char cp="0260" tag="sc:Latn" ref="0 278" comment="Kpelle (5)" />
<char cp="0268" tag="sc:Latn" ref="0 186 189 210 211" comment="Cubeo (3), Dagbani (Dagomba) (4), HIxkaryána (4), Maasai (5)" />
<char cp="0272" tag="sc:Latn" ref="0 199 218 219" comment="Susu (4), Zarma (4), Bambara (4)" />
<char cp="0289" tag="sc:Latn" ref="0 186 187 211" comment="Cubeo (3), Maasai (5)" />
<char cp="0292" tag="sc:Latn" ref="0 113 189" comment="Skolt Sami (2), Dagbani (Dagomba) (4)" />
<char cp="1E13" tag="sc:Latn" ref="0 164 257" comment="Venda (1)" />
<char cp="1E3D" tag="sc:Latn" ref="0 164 257" comment="Venda (1)" />
<char cp="1E49" tag="sc:Latn" ref="0 220" comment="Pitjantjatjara (4)" />
<char cp="1E4B" tag="sc:Latn" ref="0 164 257" comment="Venda (1)" />
<char cp="1E63" tag="sc:Latn" ref="0 254" comment="Yoruba (2)" />
<char cp="1E6D" tag="sc:Latn" ref="0 242" comment="Mizo (4)" />
<char cp="1E71" tag="sc:Latn" ref="0 164 257" comment="Venda (1)" />
<char cp="1E8D" tag="sc:Latn" ref="0 248 249" comment="Mam (4)" />
<char cp="1EA1" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EA5" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EA7" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EA9" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EAB" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EAD" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EAF" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EB1" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EB3" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EB5" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EB7" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EB9" tag="sc:Latn" ref="0 254" comment="Yoruba (2)" />
<char cp="1EBB" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EBF" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EC1" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EC3" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EC5" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EC7" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1ECB" tag="sc:Latn" ref="0 205" comment="Igbo (2)" />
<char cp="1ECD" tag="sc:Latn" ref="0 136 204 205 215 216 254" comment="Igbo (2), Yoruba (2), Marshallese (1)" />
<char cp="1ED1" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1ED3" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1ED5" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1ED7" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1ED9" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EDB" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EDD" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EDF" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EE1" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EE3" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EE5" tag="sc:Latn" ref="0 109 204 205" comment="Vietnamese (1), Igbo (2)" />
<char cp="1EE9" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EEB" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EED" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EEF" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EF1" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EF5" tag="sc:Latn" ref="0 109" comment="Vietnamese (1)" />
<char cp="1EF9" tag="sc:Latn" ref="0 109 142" comment="Vietnamese (1), Guarani (1)" />
</data>
<!--Rules section goes here-->
<rules>
<!--Character class definitions go here-->
<!--Whole label evaluation and context rules go here-->
<rule name="leading-combining-mark" ref="320" comment="RFC 5891 restrictions on placement of combining marks &#x235F;">
<start />
<union>
<class property="gc:Mn" />
<class property="gc:Mc" />
</union>
</rule>
<rule name="hyphen-minus-disallowed" ref="320" comment="RFC 5891 restrictions on placement of U+002D HYPHEN-MINUS &#x235F;">
<choice>
<rule comment="no leading hyphen">
<look-behind>
<start />
</look-behind>
<anchor />
</rule>
<rule comment="no trailing hyphen">
<anchor />
<look-ahead>
<end />
</look-ahead>
</rule>
<rule comment="no consecutive hyphens in third and fourth">
<look-behind>
<start />
<any />
<any />
<char cp="002D" comment="hyphen-minus" />
</look-behind>
<anchor />
</rule>
</choice>
</rule>
<!--Action elements go here - order defines precedence-->
<action disp="invalid" match="leading-combining-mark" comment="labels with leading combining marks are invalid &#x235F;" />
<action disp="invalid" any-variant="out-of-repertoire-var" comment="any variant label with a code point out of repertoire is invalid &#x235F;" />
<action disp="invalid" match="dot-L-dot" comment="labels with one L sharing two middle dots are invalid" />
<action disp="blocked" any-variant="blocked" comment="any variant label containing blocked variants is blocked &#x235F;" />
<action disp="allocatable" all-variants="allocatable" comment="variant labels with all variants allocatable are allocatable &#x235F;" />
<action disp="allocatable" all-variants="fallback" comment="any label with all variants of type fallback is allocatable &#x235F;" />
<action disp="blocked" any-variant="fallback" comment="any variant label with a mix of variant forms is blocked &#x235F;" />
<action disp="valid" all-variants="r-original" comment="any remaining label containing only original code points is valid &#x235F;" />
<action disp="valid" comment="catch all (default action) &#x235F;" />
</rules>
</lgr>

View File

@@ -101,8 +101,15 @@ public final class RegistryLock extends UpdateAutoTimestampEntity implements Bui
@Column(nullable = false)
private String registrarId;
/** The POC that performed the action, or null if it was a superuser. */
@Expose private String registrarPocId;
/**
* The email address of the user that performed the action, or null if it was a superuser.
*
* <p>Note: this is misnamed in the database due to historical reasons, where we used the
* registrar POC ID as the email address rather than a separate specialized field.
*/
@Column(name = "registrarPocId")
@Expose
private String registryLockEmail;
/** When the lock is first requested. */
@AttributeOverrides({
@@ -161,8 +168,8 @@ public final class RegistryLock extends UpdateAutoTimestampEntity implements Bui
return registrarId;
}
public String getRegistrarPocId() {
return registrarPocId;
public String getRegistryLockEmail() {
return registryLockEmail;
}
public DateTime getLockRequestTime() {
@@ -255,7 +262,7 @@ public final class RegistryLock extends UpdateAutoTimestampEntity implements Bui
checkArgumentNotNull(getInstance().registrarId, "Registrar ID cannot be null");
checkArgumentNotNull(getInstance().verificationCode, "Verification code cannot be null");
checkArgument(
getInstance().registrarPocId != null || getInstance().isSuperuser,
getInstance().registryLockEmail != null || getInstance().isSuperuser,
"Registrar POC ID must be provided if superuser is false");
return super.build();
}
@@ -275,8 +282,8 @@ public final class RegistryLock extends UpdateAutoTimestampEntity implements Bui
return this;
}
public Builder setRegistrarPocId(String registrarPocId) {
getInstance().registrarPocId = registrarPocId;
public Builder setRegistryLockEmail(String registryLockEmail) {
getInstance().registryLockEmail = registryLockEmail;
return this;
}

View File

@@ -14,16 +14,11 @@
package google.registry.model.registrar;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.base.Strings.isNullOrEmpty;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static com.google.common.io.BaseEncoding.base64;
import static google.registry.model.registrar.Registrar.checkValidEmail;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.CollectionUtils.nullToEmptyImmutableSortedCopy;
import static google.registry.util.PasswordUtils.SALT_SUPPLIER;
import static google.registry.util.PasswordUtils.hashPassword;
import static java.util.stream.Collectors.joining;
import com.google.common.annotations.VisibleForTesting;
@@ -38,7 +33,6 @@ import google.registry.model.Jsonifiable;
import google.registry.model.UnsafeSerializable;
import google.registry.persistence.VKey;
import google.registry.persistence.transaction.QueryComposer;
import google.registry.util.PasswordUtils;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.EnumType;
@@ -47,9 +41,7 @@ import jakarta.persistence.Id;
import jakarta.persistence.IdClass;
import java.io.Serializable;
import java.util.Map;
import java.util.Optional;
import java.util.Set;
import javax.annotation.Nullable;
/**
* A contact for a Registrar. Note, equality, hashCode and comparable have been overridden to only
@@ -107,9 +99,6 @@ public class RegistrarPoc extends ImmutableObject implements Jsonifiable, Unsafe
@Id @Expose public String registrarId;
/** External email address of this contact used for registry lock confirmations. */
String registryLockEmailAddress;
/** The voice number of the contact. */
@Expose String phoneNumber;
@@ -147,22 +136,10 @@ public class RegistrarPoc extends ImmutableObject implements Jsonifiable, Unsafe
@Expose
boolean visibleInDomainWhoisAsAbuse = false;
/**
* Whether the contact is allowed to set their registry lock password through the registrar
* console. This will be set to false on contact creation and when the user sets a password.
*/
/** Legacy field, around until we can remove the non-null constraint and the column from SQL. */
@Column(nullable = false)
boolean allowedToSetRegistryLockPassword = false;
/**
* A hashed password that exists iff this contact is registry-lock-enabled. The hash is a base64
* encoded SHA256 string.
*/
String registryLockPasswordHash;
/** Randomly generated hash salt. */
String registryLockPasswordSalt;
/**
* Helper to update the contacts associated with a Registrar. This requires querying for the
* existing contacts, deleting existing contacts that are not part of the given {@code contacts}
@@ -197,10 +174,6 @@ public class RegistrarPoc extends ImmutableObject implements Jsonifiable, Unsafe
return emailAddress;
}
public Optional<String> getRegistryLockEmailAddress() {
return Optional.ofNullable(registryLockEmailAddress);
}
public String getPhoneNumber() {
return phoneNumber;
}
@@ -229,24 +202,6 @@ public class RegistrarPoc extends ImmutableObject implements Jsonifiable, Unsafe
return new Builder(clone(this));
}
public boolean isAllowedToSetRegistryLockPassword() {
return allowedToSetRegistryLockPassword;
}
public boolean isRegistryLockAllowed() {
return !isNullOrEmpty(registryLockPasswordHash) && !isNullOrEmpty(registryLockPasswordSalt);
}
public boolean verifyRegistryLockPassword(String registryLockPassword) {
if (isNullOrEmpty(registryLockPassword)
|| isNullOrEmpty(registryLockPasswordSalt)
|| isNullOrEmpty(registryLockPasswordHash)) {
return false;
}
return PasswordUtils.verifyPassword(
registryLockPassword, registryLockPasswordHash, registryLockPasswordSalt);
}
/**
* Returns a string representation that's human friendly.
*
@@ -296,15 +251,12 @@ public class RegistrarPoc extends ImmutableObject implements Jsonifiable, Unsafe
return new JsonMapBuilder()
.put("name", name)
.put("emailAddress", emailAddress)
.put("registryLockEmailAddress", registryLockEmailAddress)
.put("phoneNumber", phoneNumber)
.put("faxNumber", faxNumber)
.put("types", getTypes().stream().map(Object::toString).collect(joining(",")))
.put("visibleInWhoisAsAdmin", visibleInWhoisAsAdmin)
.put("visibleInWhoisAsTech", visibleInWhoisAsTech)
.put("visibleInDomainWhoisAsAbuse", visibleInDomainWhoisAsAbuse)
.put("allowedToSetRegistryLockPassword", allowedToSetRegistryLockPassword)
.put("registryLockAllowed", isRegistryLockAllowed())
.put("id", getId())
.build();
}
@@ -344,14 +296,6 @@ public class RegistrarPoc extends ImmutableObject implements Jsonifiable, Unsafe
public RegistrarPoc build() {
checkNotNull(getInstance().registrarId, "Registrar ID cannot be null");
checkValidEmail(getInstance().emailAddress);
// Check allowedToSetRegistryLockPassword here because if we want to allow the user to set
// a registry lock password, we must also set up the correct registry lock email concurrently
// or beforehand.
if (getInstance().allowedToSetRegistryLockPassword) {
checkArgument(
!isNullOrEmpty(getInstance().registryLockEmailAddress),
"Registry lock email must not be null if allowing registry lock access");
}
return cloneEmptyToNull(super.build());
}
@@ -365,11 +309,6 @@ public class RegistrarPoc extends ImmutableObject implements Jsonifiable, Unsafe
return this;
}
public Builder setRegistryLockEmailAddress(@Nullable String registryLockEmailAddress) {
getInstance().registryLockEmailAddress = registryLockEmailAddress;
return this;
}
public Builder setPhoneNumber(String phoneNumber) {
getInstance().phoneNumber = phoneNumber;
return this;
@@ -409,28 +348,6 @@ public class RegistrarPoc extends ImmutableObject implements Jsonifiable, Unsafe
getInstance().visibleInDomainWhoisAsAbuse = visible;
return this;
}
public Builder setAllowedToSetRegistryLockPassword(boolean allowedToSetRegistryLockPassword) {
if (allowedToSetRegistryLockPassword) {
getInstance().registryLockPasswordSalt = null;
getInstance().registryLockPasswordHash = null;
}
getInstance().allowedToSetRegistryLockPassword = allowedToSetRegistryLockPassword;
return this;
}
public Builder setRegistryLockPassword(String registryLockPassword) {
checkArgument(
getInstance().allowedToSetRegistryLockPassword,
"Not allowed to set registry lock password for this contact");
checkArgument(
!isNullOrEmpty(registryLockPassword), "Registry lock password was null or empty");
byte[] salt = SALT_SUPPLIER.get();
getInstance().registryLockPasswordSalt = base64().encode(salt);
getInstance().registryLockPasswordHash = hashPassword(registryLockPassword, salt);
getInstance().allowedToSetRegistryLockPassword = false;
return this;
}
}
public static ImmutableList<RegistrarPoc> loadForRegistrar(String registrarId) {

View File

@@ -23,6 +23,7 @@ import google.registry.batch.DeleteLoadTestDataAction;
import google.registry.batch.DeleteProberDataAction;
import google.registry.batch.ExpandBillingRecurrencesAction;
import google.registry.batch.RelockDomainAction;
import google.registry.batch.RemoveAllDomainContactsAction;
import google.registry.batch.ResaveAllEppResourcesPipelineAction;
import google.registry.batch.ResaveEntityAction;
import google.registry.batch.SendExpiringCertificateNotificationEmailAction;
@@ -270,6 +271,8 @@ interface RequestComponent {
ReadinessProbeActionFrontend readinessProbeActionFrontend();
RemoveAllDomainContactsAction removeAllDomainContactsAction();
RdapAutnumAction rdapAutnumAction();
RdapDomainAction rdapDomainAction();

View File

@@ -23,6 +23,7 @@ import google.registry.batch.DeleteLoadTestDataAction;
import google.registry.batch.DeleteProberDataAction;
import google.registry.batch.ExpandBillingRecurrencesAction;
import google.registry.batch.RelockDomainAction;
import google.registry.batch.RemoveAllDomainContactsAction;
import google.registry.batch.ResaveAllEppResourcesPipelineAction;
import google.registry.batch.ResaveEntityAction;
import google.registry.batch.SendExpiringCertificateNotificationEmailAction;
@@ -153,6 +154,8 @@ public interface BackendRequestComponent {
RelockDomainAction relockDomainAction();
RemoveAllDomainContactsAction removeAllDomainContactsAction();
ResaveAllEppResourcesPipelineAction resaveAllEppResourcesPipelineAction();
ResaveEntityAction resaveEntityAction();

View File

@@ -112,6 +112,15 @@ public abstract class OidcTokenAuthenticationMechanism implements Authentication
logger.atInfo().log("No email address from the OIDC token:\n%s", token.getPayload());
return AuthResult.NOT_AUTHENTICATED;
}
// Short-circuit the user lookup for known service accounts.
// This check bypasses the database lookup for high-volume
// traffic from trusted system accounts to reduce database load.
if (serviceAccountEmails.contains(email)) {
return AuthResult.createApp(email);
}
logger.atInfo().log("No service account found for email address %s, loading the User", email);
Optional<User> maybeUser =
tm().transact(() -> tm().loadByKeyIfPresent(VKey.create(User.class, email)));
stopwatch.tick("OidcTokenAuthenticationMechanism maybeUser loaded");
@@ -119,10 +128,7 @@ public abstract class OidcTokenAuthenticationMechanism implements Authentication
return AuthResult.createUser(maybeUser.get());
}
logger.atInfo().log("No end user found for email address %s", email);
if (serviceAccountEmails.stream().anyMatch(e -> e.equals(email))) {
return AuthResult.createApp(email);
}
logger.atInfo().log("No service account found for email address %s", email);
logger.atWarning().log(
"The email address %s is not tied to a principal with access to Nomulus", email);
return AuthResult.NOT_AUTHENTICATED;

View File

@@ -74,11 +74,12 @@ public final class DomainLockUtils {
* <p>The lock will not be applied until {@link #verifyVerificationCode} is called.
*/
public RegistryLock saveNewRegistryLockRequest(
String domainName, String registrarId, @Nullable String registrarPocId, boolean isAdmin) {
String domainName, String registrarId, @Nullable String registryLockEmail, boolean isAdmin) {
return tm().transact(
() ->
RegistryLockDao.save(
createLockBuilder(domainName, registrarId, registrarPocId, isAdmin).build()));
createLockBuilder(domainName, registrarId, registryLockEmail, isAdmin)
.build()));
}
/**
@@ -129,13 +130,13 @@ public final class DomainLockUtils {
* the case of relocks, isAdmin is determined by the previous lock.
*/
public RegistryLock administrativelyApplyLock(
String domainName, String registrarId, @Nullable String registrarPocId, boolean isAdmin) {
String domainName, String registrarId, @Nullable String registryLockEmail, boolean isAdmin) {
return tm().transact(
() -> {
DateTime now = tm().getTransactionTime();
RegistryLock newLock =
RegistryLockDao.save(
createLockBuilder(domainName, registrarId, registrarPocId, isAdmin)
createLockBuilder(domainName, registrarId, registryLockEmail, isAdmin)
.setLockCompletionTime(now)
.build());
applyLockStatuses(newLock, now, isAdmin);
@@ -235,7 +236,7 @@ public final class DomainLockUtils {
}
private RegistryLock.Builder createLockBuilder(
String domainName, String registrarId, @Nullable String registrarPocId, boolean isAdmin) {
String domainName, String registrarId, @Nullable String registryLockEmail, boolean isAdmin) {
DateTime now = tm().getTransactionTime();
Domain domain = getDomain(domainName, registrarId, now);
verifyDomainNotLocked(domain, isAdmin);
@@ -255,7 +256,7 @@ public final class DomainLockUtils {
.setDomainName(domainName)
.setRepoId(domain.getRepoId())
.setRegistrarId(registrarId)
.setRegistrarPocId(registrarPocId)
.setRegistryLockEmail(registryLockEmail)
.isSuperuser(isAdmin);
}

View File

@@ -85,12 +85,6 @@ final class RegistrarPocCommand extends MutatingCommand {
+ " and will be used as the console login email, if --login_email is not specified.")
String email;
@Nullable
@Parameter(
names = "--registry_lock_email",
description = "Email address used for registry lock confirmation emails")
String registryLockEmail;
@Nullable
@Parameter(
names = "--phone",
@@ -132,15 +126,6 @@ final class RegistrarPocCommand extends MutatingCommand {
arity = 1)
private Boolean visibleInDomainWhoisAsAbuse;
@Nullable
@Parameter(
names = "--allowed_to_set_registry_lock_password",
description =
"Allow this contact to set their registry lock password in the console,"
+ " enabling registry lock",
arity = 1)
private Boolean allowedToSetRegistryLockPassword;
@Parameter(
names = {"-o", "--output"},
description = "Output file when --mode=LIST",
@@ -235,9 +220,6 @@ final class RegistrarPocCommand extends MutatingCommand {
builder.setRegistrar(registrar);
builder.setName(name);
builder.setEmailAddress(email);
if (!isNullOrEmpty(registryLockEmail)) {
builder.setRegistryLockEmailAddress(registryLockEmail);
}
if (phone != null) {
builder.setPhoneNumber(phone.orElse(null));
}
@@ -255,9 +237,6 @@ final class RegistrarPocCommand extends MutatingCommand {
if (visibleInDomainWhoisAsAbuse != null) {
builder.setVisibleInDomainWhoisAsAbuse(visibleInDomainWhoisAsAbuse);
}
if (allowedToSetRegistryLockPassword != null) {
builder.setAllowedToSetRegistryLockPassword(allowedToSetRegistryLockPassword);
}
return builder.build();
}
@@ -269,9 +248,6 @@ final class RegistrarPocCommand extends MutatingCommand {
if (!isNullOrEmpty(name)) {
builder.setName(name);
}
if (!isNullOrEmpty(registryLockEmail)) {
builder.setRegistryLockEmailAddress(registryLockEmail);
}
if (phone != null) {
builder.setPhoneNumber(phone.orElse(null));
}
@@ -290,9 +266,6 @@ final class RegistrarPocCommand extends MutatingCommand {
if (visibleInDomainWhoisAsAbuse != null) {
builder.setVisibleInDomainWhoisAsAbuse(visibleInDomainWhoisAsAbuse);
}
if (allowedToSetRegistryLockPassword != null) {
builder.setAllowedToSetRegistryLockPassword(allowedToSetRegistryLockPassword);
}
return builder.build();
}

View File

@@ -21,7 +21,6 @@ import static google.registry.util.DomainNameUtils.canonicalizeHostname;
import com.google.common.base.Ascii;
import com.google.common.base.Splitter;
import com.google.common.base.Strings;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.net.InternetDomainName;
@@ -29,7 +28,6 @@ import com.google.re2j.Pattern;
import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.RegistrarAddress;
import google.registry.model.registrar.RegistrarPoc;
import google.registry.ui.forms.FormException;
import google.registry.ui.forms.FormField;
import google.registry.ui.forms.FormFieldException;
import google.registry.ui.forms.FormFields;
@@ -38,7 +36,6 @@ import google.registry.util.X509Utils;
import java.security.cert.CertificateParsingException;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import java.util.Set;
import javax.annotation.Nullable;
@@ -192,14 +189,6 @@ public final class RegistrarFormFields {
public static final FormField<String, String> CONTACT_FAX_NUMBER_FIELD =
FormFields.PHONE_NUMBER.asBuilderNamed("faxNumber").build();
public static final FormField<Object, Boolean> CONTACT_ALLOWED_TO_SET_REGISTRY_LOCK_PASSWORD =
FormField.named("allowedToSetRegistryLockPassword", Object.class)
.transform(Boolean.class, b -> Boolean.valueOf(Objects.toString(b)))
.build();
public static final FormField<String, String> CONTACT_REGISTRY_LOCK_PASSWORD_FIELD =
FormFields.NAME.asBuilderNamed("registryLockPassword").build();
public static final FormField<String, Set<RegistrarPoc.Type>> CONTACT_TYPES =
FormField.named("types")
.uppercased()
@@ -369,8 +358,6 @@ public final class RegistrarFormFields {
private static void applyRegistrarContactArgs(RegistrarPoc.Builder builder, Map<String, ?> args) {
builder.setName(CONTACT_NAME_FIELD.extractUntyped(args).orElse(null));
builder.setEmailAddress(CONTACT_EMAIL_ADDRESS_FIELD.extractUntyped(args).orElse(null));
builder.setRegistryLockEmailAddress(
REGISTRY_LOCK_EMAIL_ADDRESS_FIELD.extractUntyped(args).orElse(null));
builder.setVisibleInWhoisAsAdmin(
CONTACT_VISIBLE_IN_WHOIS_AS_ADMIN_FIELD.extractUntyped(args).orElse(false));
builder.setVisibleInWhoisAsTech(
@@ -380,23 +367,5 @@ public final class RegistrarFormFields {
builder.setPhoneNumber(CONTACT_PHONE_NUMBER_FIELD.extractUntyped(args).orElse(null));
builder.setFaxNumber(CONTACT_FAX_NUMBER_FIELD.extractUntyped(args).orElse(null));
builder.setTypes(CONTACT_TYPES.extractUntyped(args).orElse(ImmutableSet.of()));
// The parser is inconsistent with whether it retrieves boolean values as strings or booleans.
// As a result, use a potentially-redundant converter that can deal with both.
builder.setAllowedToSetRegistryLockPassword(
CONTACT_ALLOWED_TO_SET_REGISTRY_LOCK_PASSWORD.extractUntyped(args).orElse(false));
// Registry lock password does not need to be set every time
CONTACT_REGISTRY_LOCK_PASSWORD_FIELD
.extractUntyped(args)
.ifPresent(
password -> {
if (!Strings.isNullOrEmpty(password)) {
if (password.length() < 8) {
throw new FormException(
"Registry lock password must be at least 8 characters long");
}
builder.setRegistryLockPassword(password);
}
});
}
}

View File

@@ -53,8 +53,10 @@ import java.util.Optional;
public class RegistrarsAction extends ConsoleApiAction {
private static final int PASSWORD_LENGTH = 16;
private static final int PASSCODE_LENGTH = 5;
private static final ImmutableList<Registrar.Type> allowedRegistrarTypes =
ImmutableList.of(Registrar.Type.REAL, Registrar.Type.OTE);
private static final ImmutableSet<Registrar.Type> TYPES_ALLOWED_FOR_USERS =
ImmutableSet.of(Registrar.Type.REAL, Registrar.Type.OTE);
private static final ImmutableSet<Registrar.Type> TYPES_ALLOWED_FOR_ADMINS =
ImmutableSet.of(Registrar.Type.INTERNAL, Registrar.Type.REAL, Registrar.Type.OTE);
private static final String SQL_TEMPLATE =
"""
SELECT * FROM "Registrar"
@@ -80,6 +82,8 @@ public class RegistrarsAction extends ConsoleApiAction {
@Override
protected void getHandler(User user) {
if (user.getUserRoles().hasGlobalPermission(ConsolePermission.VIEW_REGISTRARS)) {
ImmutableSet<Registrar.Type> allowedRegistrarTypes =
user.getUserRoles().isAdmin() ? TYPES_ALLOWED_FOR_ADMINS : TYPES_ALLOWED_FOR_USERS;
ImmutableList<Registrar> registrars =
Streams.stream(Registrar.loadAll())
.filter(r -> allowedRegistrarTypes.contains(r.getType()))
@@ -94,6 +98,7 @@ public class RegistrarsAction extends ConsoleApiAction {
.map(Map.Entry::getKey)
.collect(toImmutableSet());
@SuppressWarnings("unchecked")
List<Registrar> registrars =
tm().transact(
() ->

View File

@@ -45,7 +45,6 @@ import google.registry.ui.server.console.ConsoleApiAction;
import google.registry.ui.server.console.ConsoleApiParams;
import jakarta.inject.Inject;
import java.util.HashSet;
import java.util.Objects;
import java.util.Optional;
import java.util.Set;
import java.util.function.BiFunction;
@@ -246,7 +245,6 @@ public class ContactAction extends ConsoleApiAction {
throw new ContactRequirementException(
"An abuse contact visible in domain WHOIS query must be designated");
}
checkContactRegistryLockRequirements(existingContacts, updatedContacts);
}
private static void enforcePrimaryContactRestrictions(
@@ -266,69 +264,6 @@ public class ContactAction extends ConsoleApiAction {
}
}
private static void checkContactRegistryLockRequirements(
ImmutableSet<RegistrarPoc> existingContacts, ImmutableSet<RegistrarPoc> updatedContacts) {
// Any contact(s) with new passwords must be allowed to set them
for (RegistrarPoc updatedContact : updatedContacts) {
if (updatedContact.isRegistryLockAllowed()
|| updatedContact.isAllowedToSetRegistryLockPassword()) {
RegistrarPoc existingContact =
existingContacts.stream()
.filter(
contact -> contact.getEmailAddress().equals(updatedContact.getEmailAddress()))
.findFirst()
.orElseThrow(
() ->
new FormException(
"Cannot set registry lock password directly on new contact"));
// Can't modify registry lock email address
if (!Objects.equals(
updatedContact.getRegistryLockEmailAddress(),
existingContact.getRegistryLockEmailAddress())) {
throw new FormException("Cannot modify registryLockEmailAddress through the UI");
}
if (updatedContact.isRegistryLockAllowed()) {
// the password must have been set before or the user was allowed to set it now
if (!existingContact.isAllowedToSetRegistryLockPassword()
&& !existingContact.isRegistryLockAllowed()) {
throw new FormException("Registrar contact not allowed to set registry lock password");
}
}
if (updatedContact.isAllowedToSetRegistryLockPassword()) {
if (!existingContact.isAllowedToSetRegistryLockPassword()) {
throw new FormException(
"Cannot modify isAllowedToSetRegistryLockPassword through the UI");
}
}
}
}
// Any previously-existing contacts with registry lock enabled cannot be deleted
existingContacts.stream()
.filter(RegistrarPoc::isRegistryLockAllowed)
.forEach(
contact -> {
Optional<RegistrarPoc> updatedContactOptional =
updatedContacts.stream()
.filter(
updatedContact ->
updatedContact.getEmailAddress().equals(contact.getEmailAddress()))
.findFirst();
if (updatedContactOptional.isEmpty()) {
throw new FormException(
String.format(
"Cannot delete the contact %s that has registry lock enabled",
contact.getEmailAddress()));
}
if (!updatedContactOptional.get().isRegistryLockAllowed()) {
throw new FormException(
String.format(
"Cannot remove the ability to use registry lock on the contact %s",
contact.getEmailAddress()));
}
});
}
/**
* Retrieves the registrar contact whose phone number and email address is visible in domain WHOIS
* query as abuse contact (if any).

View File

@@ -0,0 +1,22 @@
<epp xmlns="urn:ietf:params:xml:ns:epp-1.0">
<command>
<update>
<domain:update xmlns:domain="urn:ietf:params:xml:ns:domain-1.0">
<domain:name>%DOMAIN%</domain:name>
<domain:rem>
%CONTACTS%
</domain:rem>
<domain:chg>
<domain:registrant/>
</domain:chg>
</domain:update>
</update>
<extension>
<metadata:metadata xmlns:metadata="urn:google:params:xml:ns:metadata-1.0">
<metadata:reason>Registry minimum data set phase 3: Removed all contacts from domain.</metadata:reason>
<metadata:requestedByRegistrar>false</metadata:requestedByRegistrar>
</metadata:metadata>
</extension>
<clTRID>ABC-12345</clTRID>
</command>
</epp>

View File

@@ -68,7 +68,7 @@ public class RelockDomainActionTest {
private static final String DOMAIN_NAME = "example.tld";
private static final String CLIENT_ID = "TheRegistrar";
private static final String POC_ID = "marla.singer@example.com";
private static final String LOCK_EMAIL_ADDRESS = "Marla.Singer.RegistryLock@crr.com";
private final FakeResponse response = new FakeResponse();
private final FakeClock clock = new FakeClock(DateTime.parse("2015-05-18T12:34:56Z"));
@@ -94,7 +94,9 @@ public class RelockDomainActionTest {
Host host = persistActiveHost("ns1.example.net");
domain = persistResource(DatabaseHelper.newDomain(DOMAIN_NAME, host));
oldLock = domainLockUtils.administrativelyApplyLock(DOMAIN_NAME, CLIENT_ID, POC_ID, false);
oldLock =
domainLockUtils.administrativelyApplyLock(
DOMAIN_NAME, CLIENT_ID, LOCK_EMAIL_ADDRESS, false);
assertThat(loadByEntity(domain).getStatusValues())
.containsAtLeastElementsIn(REGISTRY_LOCK_STATUSES);
oldLock =
@@ -255,9 +257,10 @@ public class RelockDomainActionTest {
.setSubject("Successful re-lock of domain example.tld")
.setBody(
"""
The domain example.tld was successfully re-locked.
The domain example.tld was successfully re-locked.
Please contact support at support@example.com if you have any questions.""")
Please contact support at support@example.com if you have any questions.\
""")
.setRecipients(
ImmutableSet.of(new InternetAddress("Marla.Singer.RegistryLock@crr.com")))
.build();
@@ -268,9 +271,10 @@ public class RelockDomainActionTest {
String expectedBody =
String.format(
"""
There was an error when automatically re-locking example.tld. Error message: %s
There was an error when automatically re-locking example.tld. Error message: %s
Please contact support at support@example.com if you have any questions.""",
Please contact support at support@example.com if you have any questions.\
""",
exceptionMessage);
assertFailureEmailWithBody(
expectedBody, ImmutableSet.of(new InternetAddress("Marla.Singer.RegistryLock@crr.com")));

View File

@@ -0,0 +1,116 @@
// Copyright 2025 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.batch;
import static com.google.common.truth.Truth.assertThat;
import static google.registry.model.common.FeatureFlag.FeatureName.MINIMUM_DATASET_CONTACTS_PROHIBITED;
import static google.registry.model.common.FeatureFlag.FeatureStatus.ACTIVE;
import static google.registry.testing.DatabaseHelper.createTld;
import static google.registry.testing.DatabaseHelper.loadByEntity;
import static google.registry.testing.DatabaseHelper.newDomain;
import static google.registry.testing.DatabaseHelper.persistActiveContact;
import static google.registry.testing.DatabaseHelper.persistResource;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedMap;
import google.registry.flows.DaggerEppTestComponent;
import google.registry.flows.EppController;
import google.registry.flows.EppTestComponent.FakesAndMocksModule;
import google.registry.model.common.FeatureFlag;
import google.registry.model.contact.Contact;
import google.registry.model.domain.Domain;
import google.registry.persistence.transaction.JpaTestExtensions;
import google.registry.persistence.transaction.JpaTestExtensions.JpaIntegrationTestExtension;
import google.registry.testing.FakeClock;
import google.registry.testing.FakeLockHandler;
import google.registry.testing.FakeResponse;
import java.util.Optional;
import org.joda.time.DateTime;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;
/** Unit tests for {@link RemoveAllDomainContactsAction}. */
class RemoveAllDomainContactsActionTest {
@RegisterExtension
final JpaIntegrationTestExtension jpa =
new JpaTestExtensions.Builder().buildIntegrationTestExtension();
private final FakeResponse response = new FakeResponse();
private RemoveAllDomainContactsAction action;
@BeforeEach
void beforeEach() {
createTld("tld");
persistResource(
new FeatureFlag.Builder()
.setFeatureName(MINIMUM_DATASET_CONTACTS_PROHIBITED)
.setStatusMap(ImmutableSortedMap.of(START_OF_TIME, ACTIVE))
.build());
EppController eppController =
DaggerEppTestComponent.builder()
.fakesAndMocksModule(FakesAndMocksModule.create(new FakeClock()))
.build()
.startRequest()
.eppController();
action =
new RemoveAllDomainContactsAction(
eppController, "NewRegistrar", new FakeLockHandler(true), response);
}
@Test
void test_removesAllContactsFromMultipleDomains_andDoesntModifyDomainThatHasNoContacts() {
Contact c1 = persistActiveContact("contact12345");
Domain d1 = persistResource(newDomain("foo.tld", c1));
assertThat(d1.getAllContacts()).hasSize(3);
Contact c2 = persistActiveContact("contact23456");
Domain d2 = persistResource(newDomain("bar.tld", c2));
assertThat(d2.getAllContacts()).hasSize(3);
Domain d3 =
persistResource(
newDomain("baz.tld")
.asBuilder()
.setRegistrant(Optional.empty())
.setContacts(ImmutableSet.of())
.build());
assertThat(d3.getAllContacts()).isEmpty();
DateTime lastUpdate = d3.getUpdateTimestamp().getTimestamp();
action.run();
assertThat(loadByEntity(d1).getAllContacts()).isEmpty();
assertThat(loadByEntity(d2).getAllContacts()).isEmpty();
assertThat(loadByEntity(d3).getUpdateTimestamp().getTimestamp()).isEqualTo(lastUpdate);
}
@Test
void test_removesContacts_onDomainsThatOnlyPartiallyHaveContacts() {
Contact c1 = persistActiveContact("contact12345");
Domain d1 =
persistResource(
newDomain("foo.tld", c1).asBuilder().setContacts(ImmutableSet.of()).build());
assertThat(d1.getAllContacts()).hasSize(1);
Contact c2 = persistActiveContact("contact23456");
Domain d2 =
persistResource(
newDomain("bar.tld", c2).asBuilder().setRegistrant(Optional.empty()).build());
assertThat(d2.getAllContacts()).hasSize(2);
action.run();
assertThat(loadByEntity(d1).getAllContacts()).isEmpty();
assertThat(loadByEntity(d2).getAllContacts()).isEmpty();
}
}

View File

@@ -20,13 +20,24 @@ import static google.registry.testing.DatabaseHelper.persistActiveDomain;
import static google.registry.testing.DatabaseHelper.persistDeletedDomain;
import static google.registry.testing.DatabaseHelper.persistReservedList;
import static google.registry.testing.DatabaseHelper.persistResource;
import static google.registry.testing.LogsSubject.assertAboutLogs;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import static google.registry.util.NetworkUtils.pickUnusedPort;
import static java.nio.charset.StandardCharsets.UTF_8;
import static java.util.concurrent.Executors.newSingleThreadExecutor;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;
import com.google.cloud.storage.BlobId;
import com.google.cloud.storage.contrib.nio.testing.LocalStorageHelper;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.flogger.FluentLogger;
import com.google.common.hash.Hashing;
import com.google.common.io.ByteStreams;
import com.google.common.net.HostAndPort;
import com.google.common.testing.TestLogHandler;
import com.google.gson.Gson;
import google.registry.bsa.api.BsaCredential;
import google.registry.gcs.GcsUtils;
import google.registry.model.tld.Tld;
@@ -35,9 +46,25 @@ import google.registry.model.tld.label.ReservedList;
import google.registry.persistence.transaction.JpaTestExtensions;
import google.registry.persistence.transaction.JpaTestExtensions.JpaIntegrationTestExtension;
import google.registry.request.UrlConnectionService;
import google.registry.server.Route;
import google.registry.server.TestServer;
import google.registry.testing.FakeClock;
import google.registry.testing.FakeResponse;
import jakarta.servlet.ServletException;
import jakarta.servlet.annotation.MultipartConfig;
import jakarta.servlet.http.HttpServlet;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import jakarta.servlet.http.Part;
import java.io.IOException;
import java.io.InputStream;
import java.io.PrintWriter;
import java.net.InetAddress;
import java.util.Map;
import java.util.Optional;
import java.util.logging.Level;
import java.util.logging.Logger;
import java.util.zip.GZIPInputStream;
import org.joda.time.DateTime;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
@@ -102,13 +129,112 @@ public class UploadBsaUnavailableDomainsActionTest {
BlobId existingFile =
BlobId.of(BUCKET, String.format("unavailable_domains_%s.txt", clock.nowUtc()));
String blockList = new String(gcsUtils.readBytesFrom(existingFile), UTF_8);
assertThat(blockList).isEqualTo("ace.tld\nflagrant.tld\nfoobar.tld\njimmy.tld\ntine.tld");
assertThat(blockList).isEqualTo("ace.tld\nflagrant.tld\nfoobar.tld\njimmy.tld\ntine.tld\n");
assertThat(blockList).doesNotContain("not-blocked.tld");
// This test currently fails in the upload-to-bsa step.
verify(emailSender, times(1))
.sendNotification("BSA daily upload completed with errors", "Please see logs for details.");
}
// TODO(mcilwain): Add test of BSA API upload as well.
@Test
void uploadToBsaTest() throws Exception {
TestLogHandler logHandler = new TestLogHandler();
Logger loggerToIntercept =
Logger.getLogger(UploadBsaUnavailableDomainsAction.class.getCanonicalName());
loggerToIntercept.addHandler(logHandler);
persistActiveDomain("foobar.tld");
persistActiveDomain("ace.tld");
persistDeletedDomain("not-blocked.tld", clock.nowUtc().minusDays(1));
var testServer = startTestServer();
action.apiUrl = testServer.getUrl("/upload").toURI().toString();
try {
action.run();
} finally {
testServer.stop();
}
String dataSent = "ace.tld\nflagrant.tld\nfoobar.tld\njimmy.tld\ntine.tld\n";
String checkSum = Hashing.sha512().hashString(dataSent, UTF_8).toString();
String expectedResponse =
"Received response with code 200 from server: "
+ String.format("Checksum: [%s]\n%s\n", checkSum, dataSent);
assertAboutLogs().that(logHandler).hasLogAtLevelWithMessage(Level.INFO, expectedResponse);
verify(emailSender, times(1)).sendNotification("BSA daily upload completed successfully", "");
}
private TestServer startTestServer() throws Exception {
TestServer testServer =
new TestServer(
HostAndPort.fromParts(InetAddress.getLocalHost().getHostAddress(), pickUnusedPort()),
ImmutableMap.of(),
ImmutableList.of(Route.route("/upload", Servelet.class)));
testServer.start();
newSingleThreadExecutor()
.execute(
() -> {
try {
while (true) {
testServer.process();
}
} catch (InterruptedException e) {
// Expected
}
});
return testServer;
}
@MultipartConfig(
location = "", // Directory for storing uploaded files. Use default when blank
maxFileSize = 10485760L, // 10MB
maxRequestSize = 20971520L, // 20MB
fileSizeThreshold = 1048576 // Save in memory if file size < 1MB
)
public static class Servelet extends HttpServlet {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
@Override
protected void doPost(HttpServletRequest req, HttpServletResponse resp)
throws ServletException, IOException {
String checkSum = null;
String content = null;
try {
for (Part part : req.getParts()) {
switch (part.getName()) {
case "zone" -> checkSum = readChecksum(part);
case "file" -> content = readGzipped(part);
}
}
} catch (Exception e) {
logger.atInfo().withCause(e).log("");
}
int status = checkSum == null || content == null ? 400 : 200;
resp.setStatus(status);
resp.setContentType("text/plain");
try (PrintWriter writer = resp.getWriter()) {
writer.printf("Checksum: [%s]\n%s\n", checkSum, content);
}
}
private String readChecksum(Part part) {
try (InputStream is = part.getInputStream()) {
return new Gson()
.fromJson(new String(ByteStreams.toByteArray(is), UTF_8), Map.class)
.getOrDefault("checkSum", "Not found")
.toString();
} catch (IOException e) {
throw new RuntimeException(e);
}
}
private String readGzipped(Part part) {
try (InputStream is = part.getInputStream();
GZIPInputStream gis = new GZIPInputStream(is)) {
return new String(ByteStreams.toByteArray(gis), UTF_8);
} catch (IOException e) {
throw new RuntimeException(e);
}
}
}
}

View File

@@ -434,11 +434,8 @@ public abstract class JpaTransactionManagerExtension
.setRegistrar(makeRegistrar2())
.setName("Marla Singer")
.setEmailAddress("Marla.Singer@crr.com")
.setRegistryLockEmailAddress("Marla.Singer.RegistryLock@crr.com")
.setPhoneNumber("+1.2128675309")
.setTypes(ImmutableSet.of(RegistrarPoc.Type.TECH))
.setAllowedToSetRegistryLockPassword(true)
.setRegistryLockPassword("hi")
.build();
}

View File

@@ -54,6 +54,9 @@ public class OidcTokenAuthenticationMechanismTest {
private static final String rawToken = "this-token";
private static final String email = "user@email.test";
private static final String unknownEmail = "bad-guy@evil.real";
private static final String gaiaId = "gaia-id";
private static final ImmutableSet<String> serviceAccounts =
ImmutableSet.of("service@email.test", "email@service.goog");
@@ -148,13 +151,12 @@ public class OidcTokenAuthenticationMechanismTest {
payload.setEmail("service@email.test");
authResult = authenticationMechanism.authenticate(request);
assertThat(authResult.isAuthenticated()).isTrue();
assertThat(authResult.authLevel()).isEqualTo(AuthLevel.USER);
assertThat(authResult.user().get()).isEqualTo(serviceUser);
assertThat(authResult.authLevel()).isEqualTo(AuthLevel.APP);
}
@Test
void testAuthenticate_unknownEmailAddress() throws Exception {
payload.setEmail("bad-guy@evil.real");
payload.setEmail(unknownEmail);
authResult = authenticationMechanism.authenticate(request);
assertThat(authResult).isEqualTo(AuthResult.NOT_AUTHENTICATED);
}

View File

@@ -50,7 +50,6 @@ class RegistrarPocTest {
.setRegistrar(testRegistrar)
.setName("Judith Registrar")
.setEmailAddress("judith.doe@example.com")
.setRegistryLockEmailAddress("judith.doe@external.com")
.setPhoneNumber("+1.2125650000")
.setFaxNumber("+1.2125650001")
.setTypes(ImmutableSet.of(WHOIS))

View File

@@ -26,10 +26,16 @@ import com.google.common.net.HostAndPort;
import com.google.common.util.concurrent.SimpleTimeLimiter;
import google.registry.util.RegistryEnvironment;
import google.registry.util.UrlChecker;
import jakarta.servlet.MultipartConfigElement;
import jakarta.servlet.annotation.MultipartConfig;
import jakarta.servlet.http.HttpServlet;
import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URL;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.FutureTask;
@@ -49,6 +55,8 @@ import org.eclipse.jetty.server.ServerConnector;
* {@link #stop()} methods. However, a {@link #process()} method was added, which is used to process
* requests made to servlets (not static files) in the calling thread.
*
* <p>A servlet that expects multi-part requests should be annotated with {@link MultipartConfig}.
*
* <p><b>Note:</b> This server is intended for development purposes. For the love all that is good,
* do not make this public-facing.
*
@@ -70,6 +78,7 @@ public final class TestServer {
private final HostAndPort urlAddress;
private final Server server = new Server();
private final BlockingQueue<FutureTask<Void>> requestQueue = new LinkedBlockingDeque<>();
private List<Path> multiPartTmpDirs = new ArrayList<>();
/**
* Creates a new instance, but does not begin serving.
@@ -134,6 +143,13 @@ public final class TestServer {
},
SHUTDOWN_TIMEOUT_MS,
TimeUnit.MILLISECONDS);
for (var dir : multiPartTmpDirs) {
try {
Files.delete(dir);
} catch (Exception e) {
// Ignore
}
}
} catch (Exception e) {
throwIfUnchecked(e);
throw new RuntimeException(e);
@@ -161,7 +177,27 @@ public final class TestServer {
StaticResourceServlet.configureServletHolder(holder, runfile.getKey(), runfile.getValue());
}
for (Route route : routes) {
context.addServlet(wrapServlet(route.servletClass()), route.path());
holder = context.addServlet(wrapServlet(route.servletClass()), route.path());
MultipartConfig multipartConfig = route.servletClass().getAnnotation(MultipartConfig.class);
if (multipartConfig != null) {
try {
var location = multipartConfig.location();
if (location == null || location.isBlank()) {
Path tmpDir = Files.createTempDirectory("TestServer_");
multiPartTmpDirs.add(tmpDir);
location = tmpDir.toString();
}
MultipartConfigElement multipartConfigElement =
new MultipartConfigElement(
location,
multipartConfig.maxFileSize(),
multipartConfig.maxRequestSize(),
multipartConfig.fileSizeThreshold());
holder.getRegistration().setMultipartConfig(multipartConfigElement);
} catch (IOException e) {
throw new RuntimeException(e);
}
}
}
holder = context.addServlet(DefaultServlet.class, "/*");
holder.setInitParameter("aliases", "1");

View File

@@ -291,7 +291,7 @@ public final class DomainLockUtilsTest {
.setRegistrarId("TheRegistrar")
.setRepoId(domain.getRepoId())
.isSuperuser(false)
.setRegistrarPocId(POC_ID)
.setRegistryLockEmail(POC_ID)
.build());
clock.advanceOneMilli();
RegistryLock resultLock =
@@ -477,7 +477,7 @@ public final class DomainLockUtilsTest {
.setRepoId("repoId")
.setRelockDuration(standardHours(6))
.setRegistrarId("TheRegistrar")
.setRegistrarPocId("someone@example.com")
.setRegistryLockEmail("someone@example.com")
.setVerificationCode("hi")
.build());
domainLockUtils.enqueueDomainRelock(lock.getRelockDuration().get(), lock.getRevisionId(), 0);
@@ -504,7 +504,7 @@ public final class DomainLockUtilsTest {
.setDomainName("example.tld")
.setRepoId("repoId")
.setRegistrarId("TheRegistrar")
.setRegistrarPocId("someone@example.com")
.setRegistryLockEmail("someone@example.com")
.setVerificationCode("hi")
.build());
IllegalArgumentException thrown =

View File

@@ -92,7 +92,6 @@ class RegistrarPocCommandTest extends CommandTestCase<RegistrarPocCommand> {
"--mode=UPDATE",
"--name=Judith Registrar",
"--email=judith.doe@example.com",
"--registry_lock_email=judith.doe@external.com",
"--phone=+1.2125650000",
"--fax=+1.2125650001",
"--contact_type=WHOIS",
@@ -108,7 +107,6 @@ class RegistrarPocCommandTest extends CommandTestCase<RegistrarPocCommand> {
.setRegistrar(registrar)
.setName("Judith Registrar")
.setEmailAddress("judith.doe@example.com")
.setRegistryLockEmailAddress("judith.doe@external.com")
.setPhoneNumber("+1.2125650000")
.setFaxNumber("+1.2125650001")
.setTypes(ImmutableSet.of(WHOIS))
@@ -255,7 +253,6 @@ class RegistrarPocCommandTest extends CommandTestCase<RegistrarPocCommand> {
"--mode=CREATE",
"--name=Jim Doe",
"--email=jim.doe@example.com",
"--registry_lock_email=jim.doe@external.com",
"--contact_type=ADMIN,ABUSE",
"--visible_in_whois_as_admin=true",
"--visible_in_whois_as_tech=false",
@@ -269,7 +266,6 @@ class RegistrarPocCommandTest extends CommandTestCase<RegistrarPocCommand> {
.setRegistrar(registrar)
.setName("Jim Doe")
.setEmailAddress("jim.doe@example.com")
.setRegistryLockEmailAddress("jim.doe@external.com")
.setTypes(ImmutableSet.of(ADMIN, ABUSE))
.setVisibleInWhoisAsAdmin(true)
.setVisibleInWhoisAsTech(false)
@@ -318,87 +314,6 @@ class RegistrarPocCommandTest extends CommandTestCase<RegistrarPocCommand> {
assertThat(loadRegistrar("NewRegistrar").getContactsRequireSyncing()).isTrue();
}
@Test
void testCreate_setAllowedToSetRegistryLockPassword() throws Exception {
runCommandForced(
"--mode=CREATE",
"--name=Jim Doe",
"--email=jim.doe@example.com",
"--registry_lock_email=jim.doe.registry.lock@example.com",
"--allowed_to_set_registry_lock_password=true",
"NewRegistrar");
RegistrarPoc registrarPoc = loadRegistrar("NewRegistrar").getContacts().asList().get(1);
assertThat(registrarPoc.isAllowedToSetRegistryLockPassword()).isTrue();
registrarPoc.asBuilder().setRegistryLockPassword("foo");
}
@Test
void testUpdate_setAllowedToSetRegistryLockPassword() throws Exception {
Registrar registrar = loadRegistrar("NewRegistrar");
RegistrarPoc registrarPoc =
persistResource(
new RegistrarPoc.Builder()
.setRegistrar(registrar)
.setName("Jim Doe")
.setEmailAddress("jim.doe@example.com")
.build());
assertThat(registrarPoc.isAllowedToSetRegistryLockPassword()).isFalse();
// First, try (and fail) to set the password directly
assertThrows(
IllegalArgumentException.class,
() -> registrarPoc.asBuilder().setRegistryLockPassword("foo"));
// Next, try (and fail) to allow registry lock without a registry lock email
assertThat(
assertThrows(
IllegalArgumentException.class,
() ->
runCommandForced(
"--mode=UPDATE",
"--email=jim.doe@example.com",
"--allowed_to_set_registry_lock_password=true",
"NewRegistrar")))
.hasMessageThat()
.isEqualTo("Registry lock email must not be null if allowing registry lock access");
// Next, include the email and it should succeed
runCommandForced(
"--mode=UPDATE",
"--email=jim.doe@example.com",
"--registry_lock_email=jim.doe.registry.lock@example.com",
"--allowed_to_set_registry_lock_password=true",
"NewRegistrar");
RegistrarPoc newContact = reloadResource(registrarPoc);
assertThat(newContact.isAllowedToSetRegistryLockPassword()).isTrue();
// should be allowed to set the password now
newContact.asBuilder().setRegistryLockPassword("foo");
}
@Test
void testUpdate_setAllowedToSetRegistryLockPassword_removesOldPassword() throws Exception {
Registrar registrar = loadRegistrar("NewRegistrar");
RegistrarPoc registrarPoc =
persistResource(
new RegistrarPoc.Builder()
.setRegistrar(registrar)
.setName("Jim Doe")
.setEmailAddress("jim.doe@example.com")
.setRegistryLockEmailAddress("jim.doe.registry.lock@example.com")
.setAllowedToSetRegistryLockPassword(true)
.setRegistryLockPassword("hi")
.build());
assertThat(registrarPoc.verifyRegistryLockPassword("hi")).isTrue();
assertThat(registrarPoc.verifyRegistryLockPassword("hello")).isFalse();
runCommandForced(
"--mode=UPDATE",
"--email=jim.doe@example.com",
"--allowed_to_set_registry_lock_password=true",
"NewRegistrar");
registrarPoc = reloadResource(registrarPoc);
assertThat(registrarPoc.verifyRegistryLockPassword("hi")).isFalse();
}
@Test
void testCreate_failure_badEmail() {
IllegalArgumentException thrown =

View File

@@ -75,7 +75,8 @@ public class ConsoleRegistryLockActionTest extends ConsoleActionBaseTestCase {
Note: this code will expire in one hour.
https://registrarconsole.tld/console/#/registry-lock-verify?lockVerificationCode=\
123456789ABCDEFGHJKLMNPQRSTUVWXY""";
123456789ABCDEFGHJKLMNPQRSTUVWXY\
""";
@Mock GmailClient gmailClient;
private ConsoleRegistryLockAction action;
@@ -112,8 +113,8 @@ public class ConsoleRegistryLockActionTest extends ConsoleActionBaseTestCase {
assertThat(response.getStatus()).isEqualTo(SC_OK);
assertThat(response.getPayload())
.isEqualTo(
"""
[{"domainName":"example.test","registrarPocId":"johndoe@theregistrar.com","lockRequestTime":\
"""
[{"domainName":"example.test","registryLockEmail":"johndoe@theregistrar.com","lockRequestTime":\
{"creationTime":"2024-04-15T00:00:00.000Z"},"unlockRequestTime":"null","lockCompletionTime":\
"2024-04-15T00:00:00.000Z","unlockCompletionTime":"null","isSuperuser":false}]\
""");
@@ -127,7 +128,7 @@ public class ConsoleRegistryLockActionTest extends ConsoleActionBaseTestCase {
.setDomainName("expired.test")
.setRegistrarId("TheRegistrar")
.setVerificationCode("123456789ABCDEFGHJKLMNPQRSTUVWXY")
.setRegistrarPocId("johndoe@theregistrar.com")
.setRegistryLockEmail("johndoe@theregistrar.com")
.build();
saveRegistryLock(expiredLock);
RegistryLock expiredUnlock =
@@ -136,7 +137,7 @@ public class ConsoleRegistryLockActionTest extends ConsoleActionBaseTestCase {
.setDomainName("expiredunlock.test")
.setRegistrarId("TheRegistrar")
.setVerificationCode("123456789ABCDEFGHJKLMNPQRSTUVWXY")
.setRegistrarPocId("johndoe@theregistrar.com")
.setRegistryLockEmail("johndoe@theregistrar.com")
.setLockCompletionTime(clock.nowUtc())
.setUnlockRequestTime(clock.nowUtc())
.build();
@@ -149,7 +150,7 @@ public class ConsoleRegistryLockActionTest extends ConsoleActionBaseTestCase {
.setDomainName("example.test")
.setRegistrarId("TheRegistrar")
.setVerificationCode("123456789ABCDEFGHJKLMNPQRSTUVWXY")
.setRegistrarPocId("johndoe@theregistrar.com")
.setRegistryLockEmail("johndoe@theregistrar.com")
.setLockCompletionTime(clock.nowUtc())
.build();
clock.advanceOneMilli();
@@ -168,7 +169,7 @@ public class ConsoleRegistryLockActionTest extends ConsoleActionBaseTestCase {
.setDomainName("pending.test")
.setRegistrarId("TheRegistrar")
.setVerificationCode("111111111ABCDEFGHJKLMNPQRSTUVWXY")
.setRegistrarPocId("johndoe@theregistrar.com")
.setRegistryLockEmail("johndoe@theregistrar.com")
.build();
RegistryLock incompleteUnlock =
@@ -177,7 +178,7 @@ public class ConsoleRegistryLockActionTest extends ConsoleActionBaseTestCase {
.setDomainName("incompleteunlock.test")
.setRegistrarId("TheRegistrar")
.setVerificationCode("123456789ABCDEFGHJKLMNPQRSTUVWXY")
.setRegistrarPocId("johndoe@theregistrar.com")
.setRegistryLockEmail("johndoe@theregistrar.com")
.setLockCompletionTime(clock.nowUtc())
.setUnlockRequestTime(clock.nowUtc())
.build();
@@ -187,7 +188,7 @@ public class ConsoleRegistryLockActionTest extends ConsoleActionBaseTestCase {
.setRepoId("repoId")
.setDomainName("unlocked.test")
.setRegistrarId("TheRegistrar")
.setRegistrarPocId("johndoe@theregistrar.com")
.setRegistryLockEmail("johndoe@theregistrar.com")
.setVerificationCode("123456789ABCDEFGHJKLMNPQRSTUUUUU")
.setLockCompletionTime(clock.nowUtc())
.setUnlockRequestTime(clock.nowUtc())
@@ -206,26 +207,27 @@ public class ConsoleRegistryLockActionTest extends ConsoleActionBaseTestCase {
// locks or completed unlocks
assertThat(response.getPayload())
.isEqualTo(
"""
"""
[{"domainName":"adminexample.test","lockRequestTime":{"creationTime":"2024-04-16T00:00:00.001Z"},\
"unlockRequestTime":"null","lockCompletionTime":"2024-04-16T00:00:00.001Z","unlockCompletionTime":\
"null","isSuperuser":true},\
\
{"domainName":"example.test","registrarPocId":"johndoe@theregistrar.com","lockRequestTime":\
{"domainName":"example.test","registryLockEmail":"johndoe@theregistrar.com","lockRequestTime":\
{"creationTime":"2024-04-16T00:00:00.001Z"},"unlockRequestTime":"null","lockCompletionTime":\
"2024-04-16T00:00:00.000Z","unlockCompletionTime":"null","isSuperuser":false},\
\
{"domainName":"expiredunlock.test","registrarPocId":"johndoe@theregistrar.com","lockRequestTime":\
{"domainName":"expiredunlock.test","registryLockEmail":"johndoe@theregistrar.com","lockRequestTime":\
{"creationTime":"2024-04-15T00:00:00.000Z"},"unlockRequestTime":"2024-04-15T00:00:00.000Z",\
"lockCompletionTime":"2024-04-15T00:00:00.000Z","unlockCompletionTime":"null","isSuperuser":false},\
\
{"domainName":"incompleteunlock.test","registrarPocId":"johndoe@theregistrar.com","lockRequestTime":\
{"domainName":"incompleteunlock.test","registryLockEmail":"johndoe@theregistrar.com","lockRequestTime":\
{"creationTime":"2024-04-16T00:00:00.001Z"},"unlockRequestTime":"2024-04-16T00:00:00.001Z",\
"lockCompletionTime":"2024-04-16T00:00:00.001Z","unlockCompletionTime":"null","isSuperuser":false},\
\
{"domainName":"pending.test","registrarPocId":"johndoe@theregistrar.com","lockRequestTime":\
{"domainName":"pending.test","registryLockEmail":"johndoe@theregistrar.com","lockRequestTime":\
{"creationTime":"2024-04-16T00:00:00.001Z"},"unlockRequestTime":"null","lockCompletionTime":"null",\
"unlockCompletionTime":"null","isSuperuser":false}]""");
"unlockCompletionTime":"null","isSuperuser":false}]\
""");
}
@Test
@@ -500,7 +502,7 @@ public class ConsoleRegistryLockActionTest extends ConsoleActionBaseTestCase {
.setRepoId(defaultDomain.getRepoId())
.setDomainName(defaultDomain.getDomainName())
.setRegistrarId(defaultDomain.getCurrentSponsorRegistrarId())
.setRegistrarPocId("johndoe@theregistrar.com")
.setRegistryLockEmail("johndoe@theregistrar.com")
.setVerificationCode("123456789ABCDEFGHJKLMNPQRSTUUUUU");
}

View File

@@ -187,7 +187,7 @@ public class ConsoleRegistryLockVerifyActionTest extends ConsoleActionBaseTestCa
.setRepoId(defaultDomain.getRepoId())
.setDomainName(defaultDomain.getDomainName())
.setRegistrarId(defaultDomain.getCurrentSponsorRegistrarId())
.setRegistrarPocId("johndoe@theregistrar.com")
.setRegistryLockEmail("johndoe@theregistrar.com")
.setVerificationCode(DEFAULT_CODE);
}

View File

@@ -284,39 +284,27 @@ class ContactActionTest extends ConsoleActionBaseTestCase {
"Registrar New Registrar (registrarId) updated in registry unittest"
+ " environment")
.setBody(
"The following changes were made in registry unittest environment to the"
+ " registrar registrarId by admin fte@email.tld:\n"
+ "\n"
+ "contacts:\n"
+ " ADDED:\n"
+ " {id="
+ id
+ ", name=Test Registrar 2,"
+ " emailAddress=incorrect@example.com, registrarId=registrarId,"
+ " registryLockEmailAddress=null, phoneNumber=+1.1234567890,"
+ " faxNumber=+1.1234567891, types=[TECH],"
+ " visibleInWhoisAsAdmin=false, visibleInWhoisAsTech=true,"
+ " visibleInDomainWhoisAsAbuse=false,"
+ " allowedToSetRegistryLockPassword=false}\n"
+ " REMOVED:\n"
+ " {id="
+ id
+ ", name=Test Registrar 2, emailAddress=test.registrar2@example.com,"
+ " registrarId=registrarId, registryLockEmailAddress=null,"
+ " phoneNumber=+1.1234567890, faxNumber=+1.1234567891, types=[TECH],"
+ " visibleInWhoisAsAdmin=false,"
+ " visibleInWhoisAsTech=true, visibleInDomainWhoisAsAbuse=false,"
+ " allowedToSetRegistryLockPassword=false}\n"
+ " FINAL CONTENTS:\n"
+ " {id="
+ id
+ ", name=Test Registrar 2,"
+ " emailAddress=incorrect@example.com, registrarId=registrarId,"
+ " registryLockEmailAddress=null, phoneNumber=+1.1234567890,"
+ " faxNumber=+1.1234567891, types=[TECH],"
+ " visibleInWhoisAsAdmin=false, visibleInWhoisAsTech=true,"
+ " visibleInDomainWhoisAsAbuse=false,"
+ " allowedToSetRegistryLockPassword=false}\n")
"""
The following changes were made in registry unittest environment to the registrar registrarId \
by admin fte@email.tld:
contacts:
ADDED:
{id=5, name=Test Registrar 2, emailAddress=incorrect@example.com, registrarId=registrarId, \
phoneNumber=+1.1234567890, faxNumber=+1.1234567891, types=[TECH], visibleInWhoisAsAdmin=false, \
visibleInWhoisAsTech=true, visibleInDomainWhoisAsAbuse=false, \
allowedToSetRegistryLockPassword=false}
REMOVED:
{id=5, name=Test Registrar 2, emailAddress=test.registrar2@example.com, \
registrarId=registrarId, phoneNumber=+1.1234567890, faxNumber=+1.1234567891, types=[TECH], \
visibleInWhoisAsAdmin=false, visibleInWhoisAsTech=true, visibleInDomainWhoisAsAbuse=false, \
allowedToSetRegistryLockPassword=false}
FINAL CONTENTS:
{id=5, name=Test Registrar 2, emailAddress=incorrect@example.com, registrarId=registrarId, \
phoneNumber=+1.1234567890, faxNumber=+1.1234567891, types=[TECH], visibleInWhoisAsAdmin=false, \
visibleInWhoisAsTech=true, visibleInDomainWhoisAsAbuse=false, \
allowedToSetRegistryLockPassword=false}
""")
.setRecipients(ImmutableList.of(new InternetAddress("notification@test.example")))
.build());
}

View File

@@ -26,6 +26,7 @@ BACKEND /_dr/task/rdeUpload RdeUploadAction
BACKEND /_dr/task/readDnsRefreshRequests ReadDnsRefreshRequestsAction POST y APP ADMIN
BACKEND /_dr/task/refreshDnsOnHostRename RefreshDnsOnHostRenameAction POST n APP ADMIN
BACKEND /_dr/task/relockDomain RelockDomainAction POST y APP ADMIN
BACKEND /_dr/task/removeAllDomainContacts RemoveAllDomainContactsAction POST n APP ADMIN
BACKEND /_dr/task/resaveAllEppResourcesPipeline ResaveAllEppResourcesPipelineAction GET n APP ADMIN
BACKEND /_dr/task/resaveEntity ResaveEntityAction POST n APP ADMIN
BACKEND /_dr/task/sendExpiringCertificateNotificationEmail SendExpiringCertificateNotificationEmailAction GET n APP ADMIN

View File

@@ -44,6 +44,7 @@ BACKEND /_dr/task/readDnsRefreshRequests ReadDnsRefreshReques
BACKEND /_dr/task/refreshDnsForAllDomains RefreshDnsForAllDomainsAction GET n APP ADMIN
BACKEND /_dr/task/refreshDnsOnHostRename RefreshDnsOnHostRenameAction POST n APP ADMIN
BACKEND /_dr/task/relockDomain RelockDomainAction POST y APP ADMIN
BACKEND /_dr/task/removeAllDomainContacts RemoveAllDomainContactsAction POST n APP ADMIN
BACKEND /_dr/task/resaveAllEppResourcesPipeline ResaveAllEppResourcesPipelineAction GET n APP ADMIN
BACKEND /_dr/task/resaveEntity ResaveEntityAction POST n APP ADMIN
BACKEND /_dr/task/sendExpiringCertificateNotificationEmail SendExpiringCertificateNotificationEmailAction GET n APP ADMIN

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -194,3 +194,16 @@ V193__password_reset_request.sql
V194__password_reset_request_registrar.sql
V195__registrar_poc_id.sql
V196__tld_expiry_access_period_enabled.sql
V197__poc_rlock_drop_not_null.sql
V198__billing_cancellation_hash.sql
V199__billing_event_hash.sql
V200__billing_recurrence_hash.sql
V201__domain_hash.sql
V202__delegation_signer_data_hash.sql
V203__domain_history_hash.sql
V204__domain_host_hash.sql
V205__domain_transaction_record_hash.sql
V206__grace_period_hash.sql
V207__grace_period_history_hash.sql
V208__host_hash.sql
V209__poll_message_hash.sql

View File

@@ -0,0 +1,16 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- In order to remove the field from the Java class it needs to be nullable
ALTER table "RegistrarPoc" ALTER column allowed_to_set_registry_lock_password DROP NOT NULL;

View File

@@ -0,0 +1,16 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- Add hash indexes on columns that are commonly queried with a direct equals
CREATE INDEX CONCURRENTLY IF NOT EXISTS billingcancellation_billing_cancellation_id_hash ON "BillingCancellation" USING hash (billing_cancellation_id);

View File

@@ -0,0 +1,16 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- Add hash indexes on columns that are commonly queried with a direct equals
CREATE INDEX CONCURRENTLY IF NOT EXISTS billingevent_billing_event_id_hash ON "BillingEvent" USING hash (billing_event_id);

View File

@@ -0,0 +1,16 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- Add hash indexes on columns that are commonly queried with a direct equals
CREATE INDEX CONCURRENTLY IF NOT EXISTS billingrecurrence_billing_recurrence_id_hash ON "BillingRecurrence" USING hash (billing_recurrence_id);

View File

@@ -0,0 +1,17 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- Add hash indexes on columns that are commonly queried with a direct equals
CREATE INDEX CONCURRENTLY IF NOT EXISTS domain_domain_name_hash ON "Domain" USING hash (domain_name);
CREATE INDEX CONCURRENTLY IF NOT EXISTS domain_domain_repo_id_hash ON "Domain" USING hash (repo_id);

View File

@@ -0,0 +1,16 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- Add hash indexes on columns that are commonly queried with a direct equals
CREATE INDEX CONCURRENTLY IF NOT EXISTS delegationsignerdata_domain_repo_id_hash ON "DelegationSignerData" USING hash (domain_repo_id);

View File

@@ -0,0 +1,17 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- Add hash indexes on columns that are commonly queried with a direct equals
CREATE INDEX CONCURRENTLY IF NOT EXISTS domainhistory_domain_repo_id_hash ON "DomainHistory" USING hash (domain_repo_id);
CREATE INDEX CONCURRENTLY IF NOT EXISTS domainhistory_history_revision_id_hash ON "DomainHistory" USING hash (history_revision_id);

View File

@@ -0,0 +1,16 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- Add hash indexes on columns that are commonly queried with a direct equals
CREATE INDEX CONCURRENTLY IF NOT EXISTS domainhost_domain_repo_id_hash ON "DomainHost" USING hash (domain_repo_id);

View File

@@ -0,0 +1,17 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- Add hash indexes on columns that are commonly queried with a direct equals
CREATE INDEX CONCURRENTLY IF NOT EXISTS domaintransactionrecord_id_hash ON "DomainTransactionRecord" USING hash (id);
CREATE INDEX CONCURRENTLY IF NOT EXISTS domaintransactionrecord_domain_history_revision_id_hash ON "DomainTransactionRecord" USING hash (history_revision_id);

View File

@@ -0,0 +1,17 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- Add hash indexes on columns that are commonly queried with a direct equals
CREATE INDEX CONCURRENTLY IF NOT EXISTS graceperiod_grace_period_id_hash ON "GracePeriod" USING hash (grace_period_id);
CREATE INDEX CONCURRENTLY IF NOT EXISTS graceperiod_domain_repo_id_hash ON "GracePeriod" USING hash (domain_repo_id);

View File

@@ -0,0 +1,16 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- Add hash indexes on columns that are commonly queried with a direct equals
CREATE INDEX CONCURRENTLY IF NOT EXISTS graceperiodhistory_grace_period_history_revision_id_hash ON "GracePeriodHistory" USING hash (grace_period_history_revision_id);

View File

@@ -0,0 +1,17 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- Add hash indexes on columns that are commonly queried with a direct equals
CREATE INDEX CONCURRENTLY IF NOT EXISTS host_host_name_hash ON "Host" USING hash (host_name);
CREATE INDEX CONCURRENTLY IF NOT EXISTS host_repo_id_hash ON "Host" USING hash (repo_id);

View File

@@ -0,0 +1,16 @@
-- Copyright 2025 The Nomulus Authors. All Rights Reserved.
--
-- Licensed under the Apache License, Version 2.0 (the "License");
-- you may not use this file except in compliance with the License.
-- You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
-- Add hash indexes on columns that are commonly queried with a direct equals
CREATE INDEX CONCURRENTLY IF NOT EXISTS pollmessage_poll_message_id_hash ON "PollMessage" USING hash (poll_message_id);

View File

@@ -688,9 +688,6 @@
id bigint,
name text,
phone_number text,
registry_lock_email_address text,
registry_lock_password_hash text,
registry_lock_password_salt text,
types text[],
visible_in_domain_whois_as_abuse boolean not null,
visible_in_whois_as_admin boolean not null,

View File

@@ -1009,7 +1009,7 @@ CREATE TABLE public."Registrar" (
CREATE TABLE public."RegistrarPoc" (
email_address text NOT NULL,
allowed_to_set_registry_lock_password boolean NOT NULL,
allowed_to_set_registry_lock_password boolean,
fax_number text,
name text,
phone_number text,
@@ -1924,6 +1924,48 @@ ALTER TABLE ONLY public."User"
CREATE INDEX allocation_token_domain_name_idx ON public."AllocationToken" USING btree (domain_name);
--
-- Name: billingcancellation_billing_cancellation_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX billingcancellation_billing_cancellation_id_hash ON public."BillingCancellation" USING hash (billing_cancellation_id);
--
-- Name: billingevent_billing_event_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX billingevent_billing_event_id_hash ON public."BillingEvent" USING hash (billing_event_id);
--
-- Name: billingrecurrence_billing_recurrence_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX billingrecurrence_billing_recurrence_id_hash ON public."BillingRecurrence" USING hash (billing_recurrence_id);
--
-- Name: delegationsignerdata_domain_repo_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX delegationsignerdata_domain_repo_id_hash ON public."DelegationSignerData" USING hash (domain_repo_id);
--
-- Name: domain_domain_name_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX domain_domain_name_hash ON public."Domain" USING hash (domain_name);
--
-- Name: domain_domain_repo_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX domain_domain_repo_id_hash ON public."Domain" USING hash (repo_id);
--
-- Name: domain_history_to_ds_data_history_idx; Type: INDEX; Schema: public; Owner: -
--
@@ -1938,6 +1980,76 @@ CREATE INDEX domain_history_to_ds_data_history_idx ON public."DomainDsDataHistor
CREATE INDEX domain_history_to_transaction_record_idx ON public."DomainTransactionRecord" USING btree (domain_repo_id, history_revision_id);
--
-- Name: domainhistory_domain_repo_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX domainhistory_domain_repo_id_hash ON public."DomainHistory" USING hash (domain_repo_id);
--
-- Name: domainhistory_history_revision_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX domainhistory_history_revision_id_hash ON public."DomainHistory" USING hash (history_revision_id);
--
-- Name: domainhost_domain_repo_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX domainhost_domain_repo_id_hash ON public."DomainHost" USING hash (domain_repo_id);
--
-- Name: domaintransactionrecord_domain_history_revision_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX domaintransactionrecord_domain_history_revision_id_hash ON public."DomainTransactionRecord" USING hash (history_revision_id);
--
-- Name: domaintransactionrecord_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX domaintransactionrecord_id_hash ON public."DomainTransactionRecord" USING hash (id);
--
-- Name: graceperiod_domain_repo_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX graceperiod_domain_repo_id_hash ON public."GracePeriod" USING hash (domain_repo_id);
--
-- Name: graceperiod_grace_period_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX graceperiod_grace_period_id_hash ON public."GracePeriod" USING hash (grace_period_id);
--
-- Name: graceperiodhistory_grace_period_history_revision_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX graceperiodhistory_grace_period_history_revision_id_hash ON public."GracePeriodHistory" USING hash (grace_period_history_revision_id);
--
-- Name: host_host_name_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX host_host_name_hash ON public."Host" USING hash (host_name);
--
-- Name: host_repo_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX host_repo_id_hash ON public."Host" USING hash (repo_id);
--
-- Name: idx1dyqmqb61xbnj7mt7bk27ds25; Type: INDEX; Schema: public; Owner: -
--
@@ -2617,6 +2729,13 @@ CREATE INDEX idxtmlqd31dpvvd2g1h9i7erw6aj ON public."AllocationToken" USING btre
CREATE INDEX idxy98mebut8ix1v07fjxxdkqcx ON public."Host" USING btree (creation_time);
--
-- Name: pollmessage_poll_message_id_hash; Type: INDEX; Schema: public; Owner: -
--
CREATE INDEX pollmessage_poll_message_id_hash ON public."PollMessage" USING hash (poll_message_id);
--
-- Name: premiumlist_name_idx; Type: INDEX; Schema: public; Owner: -
--

View File

@@ -379,9 +379,9 @@
"type": "string",
"description": "Name of the domain."
},
"registrarPocId": {
"registryLockEmail": {
"type": "string",
"description": "Registrar point of contact ID."
"description": "Email address of the requester."
},
"lockRequestTime": {
"type": "object",

View File

@@ -99,7 +99,7 @@ spec:
apiVersion: apps/v1
kind: Deployment
name: frontend
minReplicas: 8
minReplicas: 12
maxReplicas: 16
metrics:
- type: Resource

View File

@@ -119,6 +119,7 @@ public class SslClientInitializer<C extends Channel> extends ChannelInitializer<
sslContextBuilder
.build()
.newHandler(channel.alloc(), hostProvider.apply(channel), portProvider.apply(channel));
sslHandler.setHandshakeTimeoutMillis(20000);
// Enable hostname verification.
SSLEngine sslEngine = sslHandler.engine();

View File

@@ -139,6 +139,8 @@ public class SslServerInitializer<C extends Channel> extends ChannelInitializer<
logger.atInfo().log("Available Cipher Suites: %s", sslContext.cipherSuites());
SslHandler sslHandler = sslContext.newHandler(channel.alloc());
sslHandler.setHandshakeTimeoutMillis(20000);
if (requireClientCert) {
Promise<X509Certificate> clientCertificatePromise = channel.eventLoop().newPromise();
Future<Channel> unusedFuture =
@@ -159,15 +161,15 @@ public class SslServerInitializer<C extends Channel> extends ChannelInitializer<
}
logger.atInfo().log(
"""
--SSL Information--
Client Certificate Hash: %s
SSL Protocol: %s
Cipher Suite: %s
Not Before: %s
Not After: %s
Client Certificate Type: %s
Client Certificate Length: %s
""",
--SSL Information--
Client Certificate Hash: %s
SSL Protocol: %s
Cipher Suite: %s
Not Before: %s
Not After: %s
Client Certificate Type: %s
Client Certificate Length: %s
""",
getCertificateHash(clientCertificate),
sslSession.getProtocol(),
sslSession.getCipherSuite(),

View File

@@ -31,7 +31,6 @@ do
echo "Updating cluster ${parts[0]} in zone ${parts[1]}..."
gcloud container clusters get-credentials "${parts[0]}" \
--project "${project}" --zone "${parts[1]}"
kubectl apply -f "./kubernetes/proxy-limit-range.yaml" --force
sed s/GCP_PROJECT/${project}/g "./kubernetes/proxy-deployment-${environment}.yaml" | \
kubectl apply -f -
kubectl apply -f "./kubernetes/proxy-service.yaml" --force

View File

@@ -33,6 +33,13 @@ spec:
port: health-check
initialDelaySeconds: 15
periodSeconds: 20
resources:
requests:
cpu: "400m"
memory: "350Mi"
limits:
cpu: "600m"
memory: "512Mi"
imagePullPolicy: Always
args: ["--env", "production_canary"]
env:

View File

@@ -33,6 +33,13 @@ spec:
port: health-check
initialDelaySeconds: 15
periodSeconds: 20
resources:
requests:
cpu: "400m"
memory: "350Mi"
limits:
cpu: "600m"
memory: "512Mi"
imagePullPolicy: Always
args: ["--env", "production"]
env:

View File

@@ -33,6 +33,13 @@ spec:
port: health-check
initialDelaySeconds: 15
periodSeconds: 20
resources:
requests:
cpu: "400m"
memory: "350Mi"
limits:
cpu: "600m"
memory: "512Mi"
imagePullPolicy: Always
args: ["--env", "sandbox_canary", "--log"]
env:

View File

@@ -33,6 +33,13 @@ spec:
port: health-check
initialDelaySeconds: 15
periodSeconds: 20
resources:
requests:
cpu: "400m"
memory: "350Mi"
limits:
cpu: "600m"
memory: "512Mi"
imagePullPolicy: Always
args: ["--env", "sandbox", "--log"]
env:

View File

@@ -1,14 +0,0 @@
apiVersion: v1
kind: LimitRange
metadata:
name: resource-limits
namespace: default
spec:
limits:
- type: Container
default:
cpu: "600m"
memory: "512Mi"
defaultRequest:
cpu: "400m"
memory: "350Mi"