1
0
mirror of https://github.com/google/nomulus synced 2026-05-20 23:01:53 +00:00

Compare commits

...

13 Commits

Author SHA1 Message Date
Weimin Yu
8415c8bbe4 Fix typo in BsaRefreshAction (#2297) 2024-01-22 16:03:35 -05:00
Lai Jiang
dc48c257b5 Use Java 17 runtime on sandbox and production (#2296)
The blocking issue is fixed in
https://github.com/google/nomulus/pull/2224.

Java 8 support is being deprecated on 2024-01-31 and no further deployment is
possible afterwards without exception:

https://cloud.google.com/appengine/docs/legacy/standard/java/deprecations

We have been using Java 17 on alpha/crash/qa for several months and have
not oberved any other blocking issue other than possible missing email
attachements, which is being mitigated by including a link to the
attachments saved in GCS.
2024-01-22 15:21:17 -05:00
sarahcaseybot
2bf3867532 Add an example tld YAML config file (#2295) 2024-01-22 13:32:36 -05:00
Weimin Yu
44f44be643 Add bsa-refresh cron job to sandbox and prod (#2290)
This is the job that updates the unblockable domains according to recent
changes in domain registration and reservation.
2024-01-22 12:24:09 -05:00
Weimin Yu
f61579b350 Fix BsaRefreshAction bugs (#2294)
* Fix BsaRefreshAction bugs

Added functional tests for BsaRefreshAction, which checks for changes in
domain registration and reservation, and apply them to the Unblockable
domain list.

Fixed a few bugs exposed by the tests.

Also refactored a few other tests.
2024-01-22 12:23:29 -05:00
Ben McIlwain
c414e38a98 Add batching to BSA unavailable domains list generation (#2282)
This also moves it back to the replica transaction manager now that it shouldn't be timing
out its queries.

And this adds a test as well (more to come!).
2024-01-19 14:58:09 -05:00
sarahcaseybot
2cf2d7e7b1 Define the --build_environment flag and change --break_glass flag to a Boolean type (#2277)
* Define the --end_breakglass and --build_environment flags

It is necessary to define these flags in a deployment before merging go/r3pr/2273 in order to prevent breaking the exisitng TLD syncing and entity presubmit testing that has already been enabled

* make break glass 2 words

* Change break_glass flag to take a Boolean and use false value to end break glass mode

* small fixes

* Fix spacing

* Add missing G

* Add clarifying comment
2024-01-19 14:23:13 -05:00
Weimin Yu
432871add9 Fix a BSA bug and refactor some unit tests (#2291)
* Refactor a few BSA unit tests

Added a few helpers for managing reserved list in tests and updated the
tests to use them.

Also fixed a bug: when quering for newly created domains, the query
should be restricted to bsa-enrolled tlds.
2024-01-18 16:12:59 -05:00
dependabot[bot]
2621b2d679 Bump follow-redirects from 1.15.2 to 1.15.4 in /docs/console-endpoints (#2278)
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.15.2 to 1.15.4.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.15.2...v1.15.4)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Lai Jiang <jianglai@google.com>
2024-01-17 09:20:29 -05:00
Lai Jiang
7a5db3b8fe Upgrade builder image to use Java 17 (#2289)
TESTED=ran nomulus-release on alpha with the new image
2024-01-16 17:05:11 -05:00
dependabot[bot]
055f9c012c Bump follow-redirects from 1.15.3 to 1.15.4 in /console-webapp (#2283)
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.15.3 to 1.15.4.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.15.3...v1.15.4)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-16 15:34:45 -05:00
Pavlo Tkach
14ab9423f8 Update Angular to v17 (#2260) 2024-01-16 13:45:56 -05:00
sarahcaseybot
9223b81ab3 Remove create_tld and update_tld commands (#2261)
* Remove create_tld and update_tld commands

These commands are no longer necessary now that configure_tld command is available. However, the configure_tld command should only be used for crash, QA, and alpha environments. TLDs in production and sandbox must be modified using modifications to their config files in Gerrit unless using the configure_tld command in breakglass mode. Check the "How to configure TLDs" procedure doc for more info.

* re-delete file
2024-01-16 11:32:59 -05:00
73 changed files with 10769 additions and 9399 deletions

View File

@@ -61,7 +61,7 @@ dependencyLocking {
node {
download = false
version = "16.19.0"
version = "20.10.0"
}
wrapper {

View File

@@ -31,7 +31,10 @@
"style": "kebab-case"
}
],
"eol-last": ["error", "always"]
"eol-last": [
"error",
"always"
]
}
},
{

File diff suppressed because it is too large Load Diff

View File

@@ -16,35 +16,35 @@
},
"private": true,
"dependencies": {
"@angular/animations": "^15.2.2",
"@angular/cdk": "^15.2.2",
"@angular/common": "^15.2.2",
"@angular/compiler": "^15.2.2",
"@angular/core": "^15.2.2",
"@angular/forms": "^15.2.2",
"@angular/material": "^15.2.2",
"@angular/platform-browser": "^15.2.2",
"@angular/platform-browser-dynamic": "^15.2.2",
"@angular/router": "^15.2.2",
"@angular/animations": "^17.0.7",
"@angular/cdk": "^17.0.4",
"@angular/common": "^17.0.7",
"@angular/compiler": "^17.0.7",
"@angular/core": "^17.0.7",
"@angular/forms": "^17.0.7",
"@angular/material": "^17.0.4",
"@angular/platform-browser": "^17.0.7",
"@angular/platform-browser-dynamic": "^17.0.7",
"@angular/router": "^17.0.7",
"rxjs": "~7.5.0",
"tslib": "^2.3.0",
"zone.js": "~0.11.4"
"zone.js": "~0.14.2"
},
"devDependencies": {
"@angular-devkit/build-angular": "^15.2.4",
"@angular-eslint/builder": "15.2.1",
"@angular-eslint/eslint-plugin": "15.2.1",
"@angular-eslint/eslint-plugin-template": "15.2.1",
"@angular-eslint/schematics": "15.2.1",
"@angular-eslint/template-parser": "15.2.1",
"@angular/cli": "~15.2.4",
"@angular/compiler-cli": "^15.2.2",
"@angular-devkit/build-angular": "^17.0.7",
"@angular-eslint/builder": "17.1.1",
"@angular-eslint/eslint-plugin": "17.1.1",
"@angular-eslint/eslint-plugin-template": "17.1.1",
"@angular-eslint/schematics": "17.1.1",
"@angular-eslint/template-parser": "17.1.1",
"@angular/cli": "~17.0.7",
"@angular/compiler-cli": "^17.0.7",
"@types/jasmine": "~4.0.0",
"@types/node": "^18.11.18",
"@typescript-eslint/eslint-plugin": "5.48.2",
"@typescript-eslint/parser": "5.48.2",
"@typescript-eslint/eslint-plugin": "^5.59.2",
"@typescript-eslint/parser": "^5.59.2",
"concurrently": "^7.6.0",
"eslint": "^8.33.0",
"eslint": "^8.39.0",
"jasmine-core": "~4.3.0",
"karma": "~6.4.0",
"karma-chrome-launcher": "~3.1.0",
@@ -52,6 +52,6 @@
"karma-jasmine": "~5.1.0",
"karma-jasmine-html-reporter": "~2.0.0",
"prettier": "2.8.7",
"typescript": "~4.9.4"
"typescript": "~5.2.2"
}
}
}

View File

@@ -17,11 +17,19 @@ import { RouterTestingModule } from '@angular/router/testing';
import { AppComponent } from './app.component';
import { BrowserAnimationsModule } from '@angular/platform-browser/animations';
import { MaterialModule } from './material.module';
import { HttpClientTestingModule } from '@angular/common/http/testing';
import { BackendService } from './shared/services/backend.service';
describe('AppComponent', () => {
beforeEach(async () => {
await TestBed.configureTestingModule({
imports: [RouterTestingModule, MaterialModule, BrowserAnimationsModule],
imports: [
HttpClientTestingModule,
RouterTestingModule,
MaterialModule,
BrowserAnimationsModule,
],
providers: [BackendService],
declarations: [AppComponent],
}).compileComponents();
});

View File

@@ -15,6 +15,10 @@
import { ComponentFixture, TestBed } from '@angular/core/testing';
import { DomainListComponent } from './domainList.component';
import { HttpClientTestingModule } from '@angular/common/http/testing';
import { MaterialModule } from '../material.module';
import { BackendService } from '../shared/services/backend.service';
import { BrowserAnimationsModule } from '@angular/platform-browser/animations';
describe('DomainListComponent', () => {
let component: DomainListComponent;
@@ -23,6 +27,12 @@ describe('DomainListComponent', () => {
beforeEach(async () => {
await TestBed.configureTestingModule({
declarations: [DomainListComponent],
imports: [
HttpClientTestingModule,
MaterialModule,
BrowserAnimationsModule,
],
providers: [BackendService],
}).compileComponents();
fixture = TestBed.createComponent(DomainListComponent);

View File

@@ -18,6 +18,8 @@
text-decoration: none;
}
&__header {
margin-top: 0;
margin-bottom: 10px;
@media (max-width: 599px) {
.mat-toolbar {
padding: 0;

View File

@@ -15,18 +15,23 @@
import { TestBed } from '@angular/core/testing';
import { RegistrarGuard } from './registrar.guard';
import { Router, RouterStateSnapshot } from '@angular/router';
import {
ActivatedRouteSnapshot,
Router,
RouterStateSnapshot,
} from '@angular/router';
import { RegistrarService } from './registrar.service';
describe('RegistrarGuard', () => {
let guard: RegistrarGuard;
let dummyRegistrarService: RegistrarService;
let routeSpy: Router;
let dummyRoute: RouterStateSnapshot = {} as RouterStateSnapshot;
let dummyRoute: RouterStateSnapshot;
beforeEach(() => {
routeSpy = jasmine.createSpyObj<Router>('Router', ['navigate']);
dummyRegistrarService = { activeRegistrarId: '' } as RegistrarService;
dummyRoute = { url: '/value' } as RouterStateSnapshot;
TestBed.configureTestingModule({
providers: [
@@ -39,7 +44,7 @@ describe('RegistrarGuard', () => {
it('should not be able to activate when activeRegistrarId is empty', () => {
guard = TestBed.inject(RegistrarGuard);
const res = guard.canActivate();
const res = guard.canActivate(new ActivatedRouteSnapshot(), dummyRoute);
expect(res).toBeFalsy();
});
@@ -48,17 +53,16 @@ describe('RegistrarGuard', () => {
useValue: { activeRegistrarId: 'value' },
});
guard = TestBed.inject(RegistrarGuard);
const res = guard.canActivate();
const res = guard.canActivate(new ActivatedRouteSnapshot(), dummyRoute);
expect(res).toBeTrue();
});
it('should navigate to registrars when activeRegistrarId is empty', () => {
const dummyRoute = { url: '/value' } as RouterStateSnapshot;
it('should navigate to empty-registrar screen when activeRegistrarId is empty', () => {
guard = TestBed.inject(RegistrarGuard);
guard.canActivate();
guard.canActivate(new ActivatedRouteSnapshot(), dummyRoute);
expect(routeSpy.navigate).toHaveBeenCalledOnceWith([
'/registrars',
{ nextUrl: '/value' },
'/empty-registrar',
{ nextUrl: dummyRoute.url },
]);
});
});

View File

@@ -17,6 +17,7 @@ import { TestBed } from '@angular/core/testing';
import { RegistrarService } from './registrar.service';
import { BackendService } from '../shared/services/backend.service';
import { HttpClientTestingModule } from '@angular/common/http/testing';
import { MatSnackBar } from '@angular/material/snack-bar';
describe('RegistrarService', () => {
let service: RegistrarService;
@@ -24,7 +25,7 @@ describe('RegistrarService', () => {
beforeEach(() => {
TestBed.configureTestingModule({
imports: [HttpClientTestingModule],
providers: [BackendService],
providers: [BackendService, MatSnackBar],
});
service = TestBed.inject(RegistrarService);
});

View File

@@ -15,6 +15,10 @@
import { ComponentFixture, TestBed } from '@angular/core/testing';
import { RegistrarSelectorComponent } from './registrarSelector.component';
import { HttpClientTestingModule } from '@angular/common/http/testing';
import { MaterialModule } from '../material.module';
import { BackendService } from '../shared/services/backend.service';
import { BrowserAnimationsModule } from '@angular/platform-browser/animations';
describe('RegistrarSelectorComponent', () => {
let component: RegistrarSelectorComponent;
@@ -22,6 +26,12 @@ describe('RegistrarSelectorComponent', () => {
beforeEach(async () => {
await TestBed.configureTestingModule({
imports: [
HttpClientTestingModule,
MaterialModule,
BrowserAnimationsModule,
],
providers: [BackendService],
declarations: [RegistrarSelectorComponent],
}).compileComponents();

View File

@@ -15,19 +15,24 @@
import { ComponentFixture, TestBed, waitForAsync } from '@angular/core/testing';
import SecurityComponent from './security.component';
import { SecurityService } from './security.service';
import { SecurityService, apiToUiConverter } from './security.service';
import { BackendService } from 'src/app/shared/services/backend.service';
import { HttpClientTestingModule } from '@angular/common/http/testing';
import { MaterialModule } from 'src/app/material.module';
import { BrowserAnimationsModule } from '@angular/platform-browser/animations';
import { of } from 'rxjs';
import { FormsModule } from '@angular/forms';
import {
Registrar,
RegistrarService,
} from 'src/app/registrar/registrar.service';
describe('SecurityComponent', () => {
let component: SecurityComponent;
let fixture: ComponentFixture<SecurityComponent>;
let fetchSecurityDetailsSpy: Function;
let saveSpy: Function;
let dummyRegistrarService: RegistrarService;
beforeEach(async () => {
const securityServiceSpy = jasmine.createSpyObj(SecurityService, [
@@ -40,9 +45,9 @@ describe('SecurityComponent', () => {
saveSpy = securityServiceSpy.saveChanges;
securityServiceSpy.securitySettings = {
ipAddressAllowList: [{ value: '123.123.123.123' }],
};
dummyRegistrarService = {
registrar: { ipAddressAllowList: ['123.123.123.123'] },
} as RegistrarService;
await TestBed.configureTestingModule({
imports: [
@@ -52,7 +57,10 @@ describe('SecurityComponent', () => {
FormsModule,
],
declarations: [SecurityComponent],
providers: [BackendService],
providers: [
BackendService,
{ provide: RegistrarService, useValue: dummyRegistrarService },
],
})
.overrideComponent(SecurityComponent, {
set: {
@@ -72,10 +80,6 @@ describe('SecurityComponent', () => {
expect(component).toBeTruthy();
});
it('should call fetch spy', () => {
expect(fetchSecurityDetailsSpy).toHaveBeenCalledTimes(1);
});
it('should render ip allow list', waitForAsync(() => {
component.enableEdit();
fixture.whenStable().then(() => {
@@ -121,16 +125,17 @@ describe('SecurityComponent', () => {
});
it('should create temporary data structure', () => {
expect(component.dataSource).toBe(
component.securityService.securitySettings
);
component.enableEdit();
expect(component.dataSource).not.toBe(
component.securityService.securitySettings
expect(component.dataSource).toEqual(
apiToUiConverter(dummyRegistrarService.registrar)
);
component.removeIpEntry(0);
expect(component.dataSource).toEqual({ ipAddressAllowList: [] });
expect(dummyRegistrarService.registrar).toEqual({
ipAddressAllowList: ['123.123.123.123'],
} as Registrar);
component.cancel();
expect(component.dataSource).toBe(
component.securityService.securitySettings
expect(component.dataSource).toEqual(
apiToUiConverter(dummyRegistrarService.registrar)
);
});

View File

@@ -24,6 +24,7 @@ import {
import { HttpClientTestingModule } from '@angular/common/http/testing';
import SecurityComponent from './security.component';
import { BackendService } from 'src/app/shared/services/backend.service';
import { MatSnackBar } from '@angular/material/snack-bar';
describe('SecurityService', () => {
const uiMockData: SecuritySettings = {
@@ -43,7 +44,7 @@ describe('SecurityService', () => {
TestBed.configureTestingModule({
imports: [HttpClientTestingModule],
declarations: [SecurityComponent],
providers: [SecurityService, BackendService],
providers: [MatSnackBar, SecurityService, BackendService],
});
service = TestBed.inject(SecurityService);
});

View File

@@ -133,9 +133,8 @@
<h3>Address Line 1:</h3>
</div>
<div class="settings-whois__section-form">
<mat-form-field>
<mat-form-field *ngIf="registrar.localizedAddress?.street">
<input
*ngIf="registrar.localizedAddress?.street"
matInput
type="text"
[(ngModel)]="(registrar.localizedAddress?.street)![0]"
@@ -149,9 +148,8 @@
<h3>City:</h3>
</div>
<div class="settings-whois__section-form">
<mat-form-field>
<mat-form-field *ngIf="registrar.localizedAddress">
<input
*ngIf="registrar.localizedAddress"
matInput
type="text"
[(ngModel)]="registrar.localizedAddress.city"
@@ -167,9 +165,8 @@
<h3>Address Line 2:</h3>
</div>
<div class="settings-whois__section-form">
<mat-form-field>
<mat-form-field *ngIf="registrar.localizedAddress?.street">
<input
*ngIf="registrar.localizedAddress?.street"
matInput
type="text"
[(ngModel)]="(registrar.localizedAddress?.street)![1]"
@@ -183,9 +180,8 @@
<h3>State/Region:</h3>
</div>
<div class="settings-whois__section-form">
<mat-form-field>
<mat-form-field *ngIf="registrar.localizedAddress">
<input
*ngIf="registrar.localizedAddress"
matInput
type="text"
[(ngModel)]="registrar.localizedAddress.state"
@@ -201,9 +197,8 @@
<h3>Address Line 3:</h3>
</div>
<div class="settings-whois__section-form">
<mat-form-field>
<mat-form-field *ngIf="registrar.localizedAddress?.street">
<input
*ngIf="registrar.localizedAddress?.street"
matInput
type="text"
[(ngModel)]="(registrar.localizedAddress?.street)![2]"
@@ -217,9 +212,8 @@
<h3>Country Code:</h3>
</div>
<div class="settings-whois__section-form">
<mat-form-field>
<mat-form-field *ngIf="registrar.localizedAddress">
<input
*ngIf="registrar.localizedAddress"
matInput
type="text"
[(ngModel)]="registrar.localizedAddress.countryCode"

View File

@@ -15,6 +15,11 @@
import { ComponentFixture, TestBed } from '@angular/core/testing';
import WhoisComponent from './whois.component';
import { MaterialModule } from 'src/app/material.module';
import { BackendService } from 'src/app/shared/services/backend.service';
import { HttpClientTestingModule } from '@angular/common/http/testing';
import { RegistrarService } from 'src/app/registrar/registrar.service';
import { BrowserAnimationsModule } from '@angular/platform-browser/animations';
describe('WhoisComponent', () => {
let component: WhoisComponent;
@@ -23,6 +28,15 @@ describe('WhoisComponent', () => {
beforeEach(async () => {
await TestBed.configureTestingModule({
declarations: [WhoisComponent],
imports: [
HttpClientTestingModule,
MaterialModule,
BrowserAnimationsModule,
],
providers: [
BackendService,
{ provide: RegistrarService, useValue: { registrar: {} } },
],
}).compileComponents();
fixture = TestBed.createComponent(WhoisComponent);

View File

@@ -35,15 +35,6 @@ $theme-warn: mat.define-palette(mat.$red-palette);
@include form-field-density(-5);
}
@import "@angular/material/theming";
// Define application specific typography settings, font-family, etc
$typography-configuration: mat-typography-config(
$font-family: 'Roboto, "Helvetica Neue", sans-serif',
);
@include angular-material-typography($typography-configuration);
/**
** Light theme
**/

View File

@@ -54,7 +54,7 @@ public class BsaRefreshAction implements Runnable {
private final GcsClient gcsClient;
private final BsaReportSender bsaReportSender;
private final int transactionBatchSize;
private final Duration domainTxnMaxDuration;
private final Duration domainCreateTxnCommitTimeLag;
private final BsaLock bsaLock;
private final Clock clock;
private final Response response;
@@ -65,7 +65,7 @@ public class BsaRefreshAction implements Runnable {
GcsClient gcsClient,
BsaReportSender bsaReportSender,
@Config("bsaTxnBatchSize") int transactionBatchSize,
@Config("domainTxnMaxDuration") Duration domainTxnMaxDuration,
@Config("domainCreateTxnCommitTimeLag") Duration domainCreateTxnCommitTimeLag,
BsaLock bsaLock,
Clock clock,
Response response) {
@@ -73,7 +73,7 @@ public class BsaRefreshAction implements Runnable {
this.gcsClient = gcsClient;
this.bsaReportSender = bsaReportSender;
this.transactionBatchSize = transactionBatchSize;
this.domainTxnMaxDuration = domainTxnMaxDuration;
this.domainCreateTxnCommitTimeLag = domainCreateTxnCommitTimeLag;
this.bsaLock = bsaLock;
this.clock = clock;
this.response = response;
@@ -109,17 +109,21 @@ public class BsaRefreshAction implements Runnable {
RefreshSchedule schedule = maybeSchedule.get();
DomainsRefresher refresher =
new DomainsRefresher(
schedule.prevRefreshTime(), clock.nowUtc(), domainTxnMaxDuration, transactionBatchSize);
schedule.prevRefreshTime(),
clock.nowUtc(),
domainCreateTxnCommitTimeLag,
transactionBatchSize);
switch (schedule.stage()) {
case CHECK_FOR_CHANGES:
ImmutableList<UnblockableDomainChange> blockabilityChanges =
refresher.checkForBlockabilityChanges();
// Always write even if no change. Easier for manual inspection of GCS bucket.
gcsClient.writeRefreshChanges(schedule.jobName(), blockabilityChanges.stream());
if (blockabilityChanges.isEmpty()) {
logger.atInfo().log("No change to Unblockable domains found.");
schedule.updateJobStage(RefreshStage.DONE);
return null;
}
gcsClient.writeRefreshChanges(schedule.jobName(), blockabilityChanges.stream());
schedule.updateJobStage(RefreshStage.APPLY_CHANGES);
// Fall through
case APPLY_CHANGES:
@@ -132,10 +136,12 @@ public class BsaRefreshAction implements Runnable {
case UPLOAD_REMOVALS:
try (Stream<UnblockableDomainChange> changes =
gcsClient.readRefreshChanges(schedule.jobName())) {
// Unblockables with changes in REASON are removed then added back. That is why they are
// included in this stage.
Optional<String> report =
JsonSerializations.toUnblockableDomainsRemovalReport(
changes
.filter(UnblockableDomainChange::isDelete)
.filter(UnblockableDomainChange::isChangeOrDelete)
.map(UnblockableDomainChange::domainName));
if (report.isPresent()) {
gcsClient.logRemovedUnblockableDomainsReport(
@@ -153,12 +159,12 @@ public class BsaRefreshAction implements Runnable {
Optional<String> report =
JsonSerializations.toUnblockableDomainsReport(
changes
.filter(UnblockableDomainChange::isAddOrChange)
.filter(UnblockableDomainChange::isNewOrChange)
.map(UnblockableDomainChange::newValue));
if (report.isPresent()) {
gcsClient.logRemovedUnblockableDomainsReport(
gcsClient.logAddedUnblockableDomainsReport(
schedule.jobName(), LINE_SPLITTER.splitToStream(report.get()));
bsaReportSender.removeUnblockableDomainsUpdates(report.get());
bsaReportSender.addUnblockableDomainsUpdates(report.get());
} else {
logger.atInfo().log("No new Unblockable domains to add.");
}

View File

@@ -30,7 +30,7 @@ public class BsaStringUtils {
public static final Splitter LINE_SPLITTER = Splitter.on('\n');
public static String getLabelInDomain(String domainName) {
List<String> parts = DOMAIN_SPLITTER.limit(1).splitToList(domainName);
List<String> parts = DOMAIN_SPLITTER.splitToList(domainName);
checkArgument(!parts.isEmpty(), "Not a valid domain: [%s]", domainName);
return parts.get(0);
}

View File

@@ -14,28 +14,42 @@
package google.registry.bsa;
import static com.google.common.base.Verify.verify;
import static google.registry.persistence.PersistenceModule.TransactionIsolationLevel.TRANSACTION_REPEATABLE_READ;
import static google.registry.persistence.transaction.JpaTransactionManagerImpl.isInTransaction;
import static google.registry.persistence.transaction.TransactionManagerFactory.replicaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.errorprone.annotations.CanIgnoreReturnValue;
import google.registry.persistence.transaction.TransactionManager.ThrowingRunnable;
import java.util.concurrent.Callable;
/**
* Helpers for executing JPA transactions for BSA processing.
*
* <p>All mutating transactions for BSA may be executed at the {@code TRANSACTION_REPEATABLE_READ}
* level.
* <p>All mutating transactions for BSA are executed at the {@code TRANSACTION_REPEATABLE_READ}
* level since the global {@link BsaLock} ensures there is a single writer at any time.
*
* <p>All domain and label queries can use the replica since all processing are snapshot based.
*/
public final class BsaTransactions {
@CanIgnoreReturnValue
public static <T> T bsaTransact(Callable<T> work) {
verify(!isInTransaction(), "May only be used for top-level transactions.");
return tm().transact(work, TRANSACTION_REPEATABLE_READ);
}
public static void bsaTransact(ThrowingRunnable work) {
verify(!isInTransaction(), "May only be used for top-level transactions.");
tm().transact(work, TRANSACTION_REPEATABLE_READ);
}
@CanIgnoreReturnValue
public static <T> T bsaQuery(Callable<T> work) {
return tm().transact(work, TRANSACTION_REPEATABLE_READ);
verify(!isInTransaction(), "May only be used for top-level transactions.");
// TRANSACTION_REPEATABLE_READ is default on replica.
return replicaTm().transact(work);
}
private BsaTransactions() {}

View File

@@ -161,7 +161,7 @@ public class GcsClient {
}
Stream<UnblockableDomainChange> readRefreshChanges(String jobName) {
BlobId blobId = getBlobId(jobName, UNBLOCKABLE_DOMAINS_FILE);
BlobId blobId = getBlobId(jobName, REFRESHED_UNBLOCKABLE_DOMAINS_FILE);
return readStream(blobId).map(UnblockableDomainChange::deserialize);
}

View File

@@ -19,6 +19,7 @@ import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.bsa.BsaStringUtils.DOMAIN_JOINER;
import static google.registry.flows.domain.DomainFlowUtils.isReserved;
import static google.registry.model.tld.Tlds.findTldForName;
import static google.registry.model.tld.label.ReservedList.loadReservedLists;
import com.google.common.collect.ImmutableSet;
import com.google.common.net.InternetDomainName;
@@ -51,12 +52,9 @@ public final class ReservedDomainsUtils {
.flatMap(ImmutableSet::stream);
}
/** Returns */
/** Returns all reserved domains in a given {@code tld} as of {@code now}. */
static ImmutableSet<String> getAllReservedDomainsInTld(Tld tld, DateTime now) {
return tld.getReservedListNames().stream()
.map(ReservedList::get)
.filter(Optional::isPresent)
.map(Optional::get)
return loadReservedLists(tld.getReservedListNames()).stream()
.map(ReservedList::getReservedListEntries)
.map(Map::keySet)
.flatMap(Set::stream)

View File

@@ -14,13 +14,13 @@
package google.registry.bsa;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static com.google.common.collect.Iterables.getLast;
import static google.registry.model.tld.Tld.isEnrolledWithBsa;
import static google.registry.model.tld.Tlds.getTldEntitiesOfType;
import static google.registry.model.tld.label.ReservedList.loadReservedLists;
import static google.registry.persistence.PersistenceModule.TransactionIsolationLevel.TRANSACTION_REPEATABLE_READ;
import static google.registry.persistence.transaction.TransactionManagerFactory.replicaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.request.Action.Method.GET;
import static google.registry.request.Action.Method.POST;
import static java.nio.charset.StandardCharsets.US_ASCII;
@@ -28,6 +28,7 @@ import static java.nio.charset.StandardCharsets.US_ASCII;
import com.google.api.client.http.HttpStatusCodes;
import com.google.cloud.storage.BlobId;
import com.google.common.base.Joiner;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedSet;
import com.google.common.collect.Ordering;
@@ -49,8 +50,10 @@ import java.io.IOException;
import java.io.OutputStream;
import java.io.OutputStreamWriter;
import java.io.Writer;
import java.util.Optional;
import java.util.zip.GZIPOutputStream;
import javax.inject.Inject;
import javax.persistence.TypedQuery;
import okhttp3.MediaType;
import okhttp3.MultipartBody;
import okhttp3.OkHttpClient;
@@ -77,6 +80,8 @@ public class UploadBsaUnavailableDomainsAction implements Runnable {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private static final int BATCH_SIZE = 50000;
Clock clock;
BsaCredential bsaCredential;
@@ -110,10 +115,7 @@ public class UploadBsaUnavailableDomainsAction implements Runnable {
// TODO(mcilwain): Implement a date Cursor, have the cronjob run frequently, and short-circuit
// the run if the daily upload is already completed.
DateTime runTime = clock.nowUtc();
// TODO(mcilwain): Batch this.
String unavailableDomains =
Joiner.on("\n")
.join(tm().transact(() -> getUnavailableDomains(runTime), TRANSACTION_REPEATABLE_READ));
String unavailableDomains = Joiner.on("\n").join(getUnavailableDomains(runTime));
if (unavailableDomains.isEmpty()) {
logger.atWarning().log("No unavailable domains found; terminating.");
} else {
@@ -193,6 +195,7 @@ public class UploadBsaUnavailableDomainsAction implements Runnable {
}
private ImmutableSortedSet<String> getUnavailableDomains(DateTime runTime) {
// Get list of TLDs to process.
ImmutableSet<Tld> bsaEnabledTlds =
getTldEntitiesOfType(TldType.REAL).stream()
.filter(tld -> isEnrolledWithBsa(tld, runTime))
@@ -204,28 +207,56 @@ public class UploadBsaUnavailableDomainsAction implements Runnable {
ImmutableSortedSet.Builder<String> unavailableDomains =
new ImmutableSortedSet.Builder<>(Ordering.natural());
for (Tld tld : bsaEnabledTlds) {
for (ReservedList reservedList : loadReservedLists(tld.getReservedListNames())) {
if (reservedList.getShouldPublish()) {
unavailableDomains.addAll(
reservedList.getReservedListEntries().keySet().stream()
.map(label -> toDomain(label, tld))
.collect(toImmutableSet()));
}
}
}
unavailableDomains.addAll(
replicaTm()
.query(
"SELECT domainName FROM Domain "
+ "WHERE tld IN :tlds "
+ "AND deletionTime > :now ",
String.class)
.setParameter(
"tlds", bsaEnabledTlds.stream().map(Tld::getTldStr).collect(toImmutableSet()))
.setParameter("now", runTime)
.getResultList());
// Add domains on reserved lists to unavailable names list.
replicaTm()
.transact(
() -> {
for (Tld tld : bsaEnabledTlds) {
for (ReservedList reservedList : loadReservedLists(tld.getReservedListNames())) {
unavailableDomains.addAll(
reservedList.getReservedListEntries().keySet().stream()
.map(label -> toDomain(label, tld))
.collect(toImmutableSet()));
}
}
});
// Add existing domains to unavailable names list, in batches so as to not time out on replica.
ImmutableSet<String> tldNames =
bsaEnabledTlds.stream().map(Tld::getTldStr).collect(toImmutableSet());
ImmutableList<String> domainsBatch;
Optional<String> lastDomain = Optional.empty();
do {
final Optional<String> lastDomainCopy = lastDomain;
domainsBatch =
replicaTm()
.transact(
() -> {
String sql =
String.format(
"SELECT domainName FROM Domain "
+ "WHERE tld IN :tlds "
+ "AND deletionTime > :now "
+ "%s ORDER BY domainName ASC",
lastDomainCopy.isPresent()
? "AND domainName > :lastInPreviousBatch"
: "");
TypedQuery<String> query =
replicaTm()
.query(sql, String.class)
.setParameter("tlds", tldNames)
.setParameter("now", runTime);
lastDomainCopy.ifPresent(l -> query.setParameter("lastInPreviousBatch", l));
return query
.setMaxResults(BATCH_SIZE)
.getResultStream()
.collect(toImmutableList());
});
unavailableDomains.addAll(domainsBatch);
lastDomain = Optional.ofNullable(domainsBatch.isEmpty() ? null : getLast(domainsBatch));
} while (domainsBatch.size() == BATCH_SIZE);
ImmutableSortedSet<String> result = unavailableDomains.build();
logger.atInfo().log("Found %d total unavailable domains.", result.size());
return result;

View File

@@ -28,9 +28,9 @@ import java.util.List;
// TODO(1/15/2024): rename to UnblockableDomain.
@AutoValue
public abstract class UnblockableDomain {
abstract String domainName();
public abstract String domainName();
abstract Reason reason();
public abstract Reason reason();
/** Reasons why a valid domain name cannot be blocked. */
public enum Reason {

View File

@@ -52,12 +52,16 @@ public abstract class UnblockableDomainChange {
return UnblockableDomain.of(unblockable().domainName(), newReason().get());
}
public boolean isAddOrChange() {
public boolean isNewOrChange() {
return newReason().isPresent();
}
public boolean isChangeOrDelete() {
return !isNew();
}
public boolean isDelete() {
return !this.isAddOrChange();
return !this.isNewOrChange();
}
public boolean isNew() {

View File

@@ -105,6 +105,10 @@ class BsaUnblockableDomain {
return new BsaUnblockableDomain(parts.get(0), parts.get(1), reason);
}
static BsaUnblockableDomain of(UnblockableDomain unblockable) {
return of(unblockable.domainName(), Reason.valueOf(unblockable.reason().name()));
}
static VKey<BsaUnblockableDomain> vKey(String domainName) {
ImmutableList<String> parts = ImmutableList.copyOf(DOMAIN_SPLITTER.splitToList(domainName));
verify(parts.size() == 2, "Invalid domain name: %s", domainName);

View File

@@ -17,9 +17,15 @@ package google.registry.bsa.persistence;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static com.google.common.collect.ImmutableMap.toImmutableMap;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.bsa.BsaTransactions.bsaQuery;
import static google.registry.bsa.BsaTransactions.bsaTransact;
import static google.registry.bsa.ReservedDomainsUtils.getAllReservedNames;
import static google.registry.bsa.ReservedDomainsUtils.isReservedDomain;
import static google.registry.bsa.persistence.Queries.queryLivesDomains;
import static google.registry.bsa.persistence.Queries.batchReadUnblockables;
import static google.registry.bsa.persistence.Queries.queryBsaLabelByLabels;
import static google.registry.bsa.persistence.Queries.queryNewlyCreatedDomains;
import static google.registry.model.tld.Tld.isEnrolledWithBsa;
import static google.registry.model.tld.Tlds.getTldEntitiesOfType;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static java.util.stream.Collectors.groupingBy;
@@ -36,7 +42,10 @@ import google.registry.bsa.api.UnblockableDomain.Reason;
import google.registry.bsa.api.UnblockableDomainChange;
import google.registry.model.ForeignKeyUtils;
import google.registry.model.domain.Domain;
import google.registry.model.tld.Tld;
import google.registry.model.tld.Tld.TldType;
import google.registry.util.BatchedStreams;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Map.Entry;
@@ -100,6 +109,8 @@ public final class DomainsRefresher {
ImmutableSet<String> upgradedDomains =
upgrades.stream().map(UnblockableDomainChange::domainName).collect(toImmutableSet());
// Upgrades are collected in its own txn after downgrades so need reconciliation. Conflicts are
// unlikely but possible: domains may be recreated or reserved names added during the interval.
ImmutableList<UnblockableDomainChange> trueDowngrades =
downgrades.stream()
.filter(c -> !upgradedDomains.contains(c.domainName()))
@@ -124,7 +135,7 @@ public final class DomainsRefresher {
ImmutableList<BsaUnblockableDomain> batch;
Optional<BsaUnblockableDomain> lastRead = Optional.empty();
do {
batch = Queries.batchReadUnblockables(lastRead, transactionBatchSize);
batch = batchReadUnblockables(lastRead, transactionBatchSize);
if (!batch.isEmpty()) {
lastRead = Optional.of(batch.get(batch.size() - 1));
changes.addAll(recheckStaleDomainsBatch(batch));
@@ -191,42 +202,81 @@ public final class DomainsRefresher {
}
public ImmutableList<UnblockableDomainChange> getNewUnblockables() {
ImmutableSet<String> newCreated = getNewlyCreatedUnblockables(prevRefreshStartTime, now);
ImmutableSet<String> newReserved = getNewlyReservedUnblockables(now, transactionBatchSize);
SetView<String> reservedNotCreated = Sets.difference(newReserved, newCreated);
return Streams.concat(
// TODO(weiminyu): both methods below use `queryBsaLabelByLabels`. Should combine in a single
// query.
HashSet<String> newCreated =
Sets.newHashSet(bsaQuery(() -> getNewlyCreatedUnblockables(prevRefreshStartTime, now)));
// We cannot identify new reserved unblockables so must look at all of them. There are not many
// of these.
HashSet<String> allReserved =
Sets.newHashSet(bsaQuery(() -> getAllReservedUnblockables(now, transactionBatchSize)));
HashSet<String> reservedNotCreated = Sets.newHashSet(Sets.difference(allReserved, newCreated));
ImmutableList.Builder<UnblockableDomainChange> changes = new ImmutableList.Builder<>();
ImmutableList<BsaUnblockableDomain> batch;
Optional<BsaUnblockableDomain> lastRead = Optional.empty();
do {
batch = batchReadUnblockables(lastRead, transactionBatchSize);
if (batch.isEmpty()) {
break;
}
lastRead = Optional.of(batch.get(batch.size() - 1));
for (BsaUnblockableDomain unblockable : batch) {
String domainName = unblockable.domainName();
if (unblockable.reason.equals(BsaUnblockableDomain.Reason.REGISTERED)) {
newCreated.remove(domainName);
reservedNotCreated.remove(domainName);
} else {
reservedNotCreated.remove(domainName);
if (newCreated.remove(domainName)) {
changes.add(
UnblockableDomainChange.ofChanged(
unblockable.toUnblockableDomain(), Reason.REGISTERED));
}
}
}
} while (batch.size() == transactionBatchSize);
Streams.concat(
newCreated.stream()
.map(name -> UnblockableDomain.of(name, Reason.REGISTERED))
.map(UnblockableDomainChange::ofNew),
reservedNotCreated.stream()
.map(name -> UnblockableDomain.of(name, Reason.RESERVED))
.map(UnblockableDomainChange::ofNew))
.collect(toImmutableList());
.forEach(changes::add);
return changes.build();
}
static ImmutableSet<String> getNewlyCreatedUnblockables(
DateTime prevRefreshStartTime, DateTime now) {
ImmutableSet<String> liveDomains = queryLivesDomains(prevRefreshStartTime, now);
return getUnblockedDomainNames(liveDomains);
ImmutableSet<String> bsaEnabledTlds =
getTldEntitiesOfType(TldType.REAL).stream()
.filter(tld -> isEnrolledWithBsa(tld, now))
.map(Tld::getTldStr)
.collect(toImmutableSet());
ImmutableSet<String> liveDomains =
queryNewlyCreatedDomains(bsaEnabledTlds, prevRefreshStartTime, now);
return getBlockedDomainNames(liveDomains);
}
static ImmutableSet<String> getNewlyReservedUnblockables(DateTime now, int batchSize) {
static ImmutableSet<String> getAllReservedUnblockables(DateTime now, int batchSize) {
Stream<String> allReserved = getAllReservedNames(now);
return BatchedStreams.toBatches(allReserved, batchSize)
.map(DomainsRefresher::getUnblockedDomainNames)
.map(DomainsRefresher::getBlockedDomainNames)
.flatMap(ImmutableSet::stream)
.collect(toImmutableSet());
}
static ImmutableSet<String> getUnblockedDomainNames(ImmutableCollection<String> domainNames) {
static ImmutableSet<String> getBlockedDomainNames(ImmutableCollection<String> domainNames) {
Map<String, List<String>> labelToNames =
domainNames.stream().collect(groupingBy(BsaStringUtils::getLabelInDomain));
ImmutableSet<String> bsaLabels =
Queries.queryBsaLabelByLabels(ImmutableSet.copyOf(labelToNames.keySet()))
queryBsaLabelByLabels(ImmutableSet.copyOf(labelToNames.keySet()))
.map(BsaLabel::getLabel)
.collect(toImmutableSet());
return labelToNames.entrySet().stream()
.filter(entry -> !bsaLabels.contains(entry.getKey()))
.filter(entry -> bsaLabels.contains(entry.getKey()))
.map(Entry::getValue)
.flatMap(List::stream)
.collect(toImmutableSet());
@@ -239,20 +289,21 @@ public final class DomainsRefresher {
.collect(
groupingBy(
change -> change.isDelete() ? "remove" : "change", toImmutableSet())));
tm().transact(
() -> {
if (changesByType.containsKey("remove")) {
tm().delete(
changesByType.get("remove").stream()
.map(c -> BsaUnblockableDomain.vKey(c.domainName()))
.collect(toImmutableSet()));
}
if (changesByType.containsKey("change")) {
tm().putAll(
changesByType.get("change").stream()
.map(UnblockableDomainChange::newValue)
.collect(toImmutableSet()));
}
});
bsaTransact(
() -> {
if (changesByType.containsKey("remove")) {
tm().delete(
changesByType.get("remove").stream()
.map(c -> BsaUnblockableDomain.vKey(c.domainName()))
.collect(toImmutableSet()));
}
if (changesByType.containsKey("change")) {
tm().putAll(
changesByType.get("change").stream()
.map(UnblockableDomainChange::newValue)
.map(BsaUnblockableDomain::of)
.collect(toImmutableSet()));
}
});
}
}

View File

@@ -16,14 +16,13 @@ package google.registry.bsa.persistence;
import static com.google.common.base.Verify.verify;
import static google.registry.bsa.BsaStringUtils.DOMAIN_SPLITTER;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.bsa.BsaTransactions.bsaQuery;
import static google.registry.persistence.transaction.JpaTransactionManagerImpl.em;
import com.google.common.collect.ImmutableCollection;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import google.registry.model.CreateAutoTimestamp;
import google.registry.model.ForeignKeyUtils;
import google.registry.model.domain.Domain;
import java.util.List;
import java.util.Optional;
import java.util.stream.Collectors;
@@ -36,15 +35,14 @@ class Queries {
private Queries() {}
private static Object detach(Object obj) {
tm().getEntityManager().detach(obj);
em().detach(obj);
return obj;
}
static Stream<BsaUnblockableDomain> queryBsaUnblockableDomainByLabels(
ImmutableCollection<String> labels) {
return ((Stream<?>)
tm().getEntityManager()
.createQuery("FROM BsaUnblockableDomain WHERE label in (:labels)")
em().createQuery("FROM BsaUnblockableDomain WHERE label in (:labels)")
.setParameter("labels", labels)
.getResultStream())
.map(Queries::detach)
@@ -53,8 +51,7 @@ class Queries {
static Stream<BsaLabel> queryBsaLabelByLabels(ImmutableCollection<String> labels) {
return ((Stream<?>)
tm().getEntityManager()
.createQuery("FROM BsaLabel where label in (:labels)")
em().createQuery("FROM BsaLabel where label in (:labels)")
.setParameter("labels", labels)
.getResultStream())
.map(Queries::detach)
@@ -62,8 +59,7 @@ class Queries {
}
static int deleteBsaLabelByLabels(ImmutableCollection<String> labels) {
return tm().getEntityManager()
.createQuery("DELETE FROM BsaLabel where label IN (:deleted_labels)")
return em().createQuery("DELETE FROM BsaLabel where label IN (:deleted_labels)")
.setParameter("deleted_labels", labels)
.executeUpdate();
}
@@ -71,14 +67,15 @@ class Queries {
static ImmutableList<BsaUnblockableDomain> batchReadUnblockables(
Optional<BsaUnblockableDomain> lastRead, int batchSize) {
return ImmutableList.copyOf(
tm().getEntityManager()
.createQuery(
"FROM BsaUnblockableDomain d WHERE d.label > :label OR (d.label = :label AND d.tld"
+ " > :tld) ORDER BY d.tld, d.label ")
.setParameter("label", lastRead.map(d -> d.label).orElse(""))
.setParameter("tld", lastRead.map(d -> d.tld).orElse(""))
.setMaxResults(batchSize)
.getResultList());
bsaQuery(
() ->
em().createQuery(
"FROM BsaUnblockableDomain d WHERE d.label > :label OR (d.label = :label"
+ " AND d.tld > :tld) ORDER BY d.tld, d.label ")
.setParameter("label", lastRead.map(d -> d.label).orElse(""))
.setParameter("tld", lastRead.map(d -> d.tld).orElse(""))
.setMaxResults(batchSize)
.getResultList()));
}
static ImmutableSet<String> queryUnblockablesByNames(ImmutableSet<String> domains) {
@@ -96,17 +93,20 @@ class Queries {
"SELECT CONCAT(d.label, '.', d.tld) FROM \"BsaUnblockableDomain\" d "
+ "WHERE (d.label, d.tld) IN (%s)",
labelTldParis);
return ImmutableSet.copyOf(tm().getEntityManager().createNativeQuery(sql).getResultList());
return ImmutableSet.copyOf(em().createNativeQuery(sql).getResultList());
}
static ImmutableSet<String> queryLivesDomains(DateTime minCreationTime, DateTime now) {
ImmutableSet<String> candidates =
ImmutableSet.copyOf(
tm().getEntityManager()
.createQuery(
"SELECT domainName FROM Domain WHERE creationTime >= :time ", String.class)
.setParameter("time", CreateAutoTimestamp.create(minCreationTime))
.getResultList());
return ImmutableSet.copyOf(ForeignKeyUtils.load(Domain.class, candidates, now).keySet());
static ImmutableSet<String> queryNewlyCreatedDomains(
ImmutableCollection<String> tlds, DateTime minCreationTime, DateTime now) {
return ImmutableSet.copyOf(
em().createQuery(
"SELECT domainName FROM Domain WHERE creationTime >= :minCreationTime "
+ "AND deletionTime > :now "
+ "AND tld in (:tlds)",
String.class)
.setParameter("minCreationTime", CreateAutoTimestamp.create(minCreationTime))
.setParameter("now", now)
.setParameter("tlds", tlds)
.getResultList());
}
}

View File

@@ -1472,9 +1472,9 @@ public final class RegistryConfig {
}
@Provides
@Config("domainTxnMaxDuration")
public static Duration provideDomainTxnMaxDuration(RegistryConfigSettings config) {
return Duration.standardSeconds(config.bsa.domainTxnMaxDurationSeconds);
@Config("domainCreateTxnCommitTimeLag")
public static Duration provideDomainCreateTxnCommitTimeLag(RegistryConfigSettings config) {
return Duration.standardSeconds(config.bsa.domainCreateTxnCommitTimeLagSeconds);
}
@Provides

View File

@@ -273,7 +273,7 @@ public class RegistryConfigSettings {
public int bsaDownloadIntervalMinutes;
public int bsaMaxNopIntervalHours;
public int bsaTxnBatchSize;
public int domainTxnMaxDurationSeconds;
public int domainCreateTxnCommitTimeLagSeconds;
public String authUrl;
public int authTokenExpirySeconds;
public Map<String, String> dataUrls;

View File

@@ -624,9 +624,18 @@ bsa:
# Max time period during which downloads can be skipped because checksums have
# not changed from the previous one.
bsaMaxNopIntervalHours: 24
# A very lax upper bound of the time it takes to execute a transaction that
# mutates a domain. Please See `BsaRefreshAction` for use case.
domainTxnMaxDurationSeconds: 60
# A very lax upper bound of the lag between a domain-creating transaction's
# recorded and actual commit time. In Nomulus, a domain's creation time is the
# start time of the transaction, while the domain is only visible after the
# transaction commits. Let `l` represents this lag, then at any point of time
# `t`, a query of domains by creation time is only guaranteed to find those
# created before `t - l`. Please See `BsaRefreshAction` for use case.
#
# The value below is decided by finding the longest domain-creation EPP
# request over the past 3 months (60 seconds, which is much longer than the
# transaction time), and add to it the maximum allowed replication lag (30
# seconds).
domainCreateTxnCommitTimeLagSeconds: 90
# Number of entities (labels and unblockable domains) to process in a single
# DB transaction.
bsaTxnBatchSize: 1000

View File

@@ -0,0 +1,60 @@
# Example of a TLD configuration file. TLD configuration files are stored in
# the internal repository and synced regularly to the database using the
# release/cloudbuild-tld-sync.yaml job. The file can be created using the exact
# output from the getTldCommand of an existing TLD.
addGracePeriodLength: "PT432000S"
allowedFullyQualifiedHostNames: []
allowedRegistrantContactIds: []
anchorTenantAddGracePeriodLength: "PT2592000S"
autoRenewGracePeriodLength: "PT3888000S"
automaticTransferLength: "PT432000S"
claimsPeriodEnd: "294247-01-10T04:00:54.775Z"
createBillingCost:
currency: "USD"
amount: 8.00
creationTime: "2017-01-03T19:52:47.770Z"
currency: "USD"
defaultPromoTokens: []
dnsAPlusAaaaTtl: null
dnsDsTtl: null
dnsNsTtl: null
dnsPaused: true
dnsWriters:
- "VoidDnsWriter"
driveFolderId: null
eapFeeSchedule:
"1970-01-01T00:00:00.000Z":
currency: "USD"
amount: 0.00
escrowEnabled: false
idnTables: []
invoicingEnabled: false
lordnUsername: null
numDnsPublishLocks: 1
pendingDeleteLength: "PT432000S"
premiumListName: null
pricingEngineClassName: "google.registry.model.pricing.StaticPremiumListPricingEngine"
redemptionGracePeriodLength: "PT2592000S"
registryLockOrUnlockBillingCost:
currency: "USD"
amount: 0.00
renewBillingCostTransitions:
"1970-01-01T00:00:00.000Z":
currency: "USD"
amount: 8.00
renewGracePeriodLength: "PT432000S"
reservedListNames: []
restoreBillingCost:
currency: "USD"
amount: 50.00
roidSuffix: "EXAMPLE"
serverStatusChangeBillingCost:
currency: "USD"
amount: 0.00
tldStateTransitions:
"1970-01-01T00:00:00.000Z": "GENERAL_AVAILABILITY"
tldStr: "example"
tldType: "TEST"
tldUnicode: "example"
transferGracePeriodLength: "PT432000S"

View File

@@ -1,10 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java8</runtime>
<runtime>java17</runtime>
<service>backend</service>
<!--app-engine-apis>true</app-engine-apis-->
<threadsafe>true</threadsafe>
<app-engine-apis>true</app-engine-apis>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4_1G</instance-class>
<basic-scaling>

View File

@@ -1,10 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java8</runtime>
<runtime>java17</runtime>
<service>bsa</service>
<!--app-engine-apis>true</app-engine-apis-->
<threadsafe>true</threadsafe>
<app-engine-apis>true</app-engine-apis>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4_1G</instance-class>
<basic-scaling>

View File

@@ -1,10 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java8</runtime>
<runtime>java17</runtime>
<service>default</service>
<!--app-engine-apis>true</app-engine-apis-->
<threadsafe>true</threadsafe>
<app-engine-apis>true</app-engine-apis>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4_1G</instance-class>
<manual-scaling>

View File

@@ -285,6 +285,18 @@
<schedule>0 * * * *</schedule>
</task>
<task>
<url><![CDATA[/_dr/task/bsaRefresh]]></url>
<name>bsaRefresh</name>
<service>bsa</service>
<description>
Checks for changes in registered domains and reserved labels, and updates
the unblockable domains list.
</description>
<!-- Runs every 30 minutes. -->
<schedule>15,45 * * * *</schedule>
</task>
<task>
<url><![CDATA[/_dr/task/uploadBsaUnavailableNames]]></url>
<name>uploadBsaUnavailableNames</name>

View File

@@ -1,10 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java8</runtime>
<runtime>java17</runtime>
<service>pubapi</service>
<!--app-engine-apis>true</app-engine-apis-->
<threadsafe>true</threadsafe>
<app-engine-apis>true</app-engine-apis>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4_1G</instance-class>
<manual-scaling>

View File

@@ -1,10 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java8</runtime>
<runtime>java17</runtime>
<service>tools</service>
<!--app-engine-apis>true</app-engine-apis-->
<threadsafe>true</threadsafe>
<app-engine-apis>true</app-engine-apis>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4_1G</instance-class>
<basic-scaling>

View File

@@ -1,10 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java8</runtime>
<runtime>java17</runtime>
<service>backend</service>
<!--app-engine-apis>true</app-engine-apis-->
<threadsafe>true</threadsafe>
<app-engine-apis>true</app-engine-apis>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4</instance-class>
<basic-scaling>

View File

@@ -1,10 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java8</runtime>
<runtime>java17</runtime>
<service>bsa</service>
<!--app-engine-apis>true</app-engine-apis-->
<threadsafe>true</threadsafe>
<app-engine-apis>true</app-engine-apis>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4</instance-class>
<basic-scaling>

View File

@@ -1,10 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java8</runtime>
<runtime>java17</runtime>
<service>default</service>
<!--app-engine-apis>true</app-engine-apis-->
<threadsafe>true</threadsafe>
<app-engine-apis>true</app-engine-apis>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4_1G</instance-class>
<manual-scaling>

View File

@@ -174,6 +174,18 @@
<schedule>0 * * * *</schedule>
</task>
<task>
<url><![CDATA[/_dr/task/bsaRefresh]]></url>
<name>bsaRefresh</name>
<service>bsa</service>
<description>
Checks for changes in registered domains and reserved labels, and updates
the unblockable domains list.
</description>
<!-- Runs every 30 minutes. -->
<schedule>15,45 * * * *</schedule>
</task>
<task>
<url><![CDATA[/_dr/task/uploadBsaUnavailableNames]]></url>
<name>uploadBsaUnavailableNames</name>

View File

@@ -1,10 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java8</runtime>
<runtime>java17</runtime>
<service>pubapi</service>
<!--app-engine-apis>true</app-engine-apis-->
<threadsafe>true</threadsafe>
<app-engine-apis>true</app-engine-apis>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4_1G</instance-class>
<manual-scaling>

View File

@@ -1,10 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<runtime>java8</runtime>
<runtime>java17</runtime>
<service>tools</service>
<!--app-engine-apis>true</app-engine-apis-->
<threadsafe>true</threadsafe>
<app-engine-apis>true</app-engine-apis>
<sessions-enabled>true</sessions-enabled>
<instance-class>B4</instance-class>
<basic-scaling>

View File

@@ -296,7 +296,7 @@ public class Tld extends ImmutableObject implements Buildable, UnsafeSerializabl
* this procedure to change this value:
*
* <ol>
* <li>Pause the DNS queue via {@link google.registry.tools.UpdateTldCommand}
* <li>Pause the DNS queue via {@link google.registry.tools.ConfigureTldCommand}
* <li>Change this number
* <li>Let the Tld caches expire (currently 5 minutes) and drain the DNS publish queue
* <li>Unpause the DNS queue

View File

@@ -35,7 +35,13 @@ public interface JpaTransactionManager extends TransactionManager {
* Returns the {@link EntityManager} for the current request.
*
* <p>The returned instance is closed when the current transaction completes.
*
* @deprecated Use the static {@link JpaTransactionManagerImpl#em} method for now. In current
* implementation the entity manager is obtained from a static {@code ThreadLocal} object that
* is set up by the outermost {@link #transact} call. As an instance method, this method gives
* the illusion that the call site has control over which database instance to use.
*/
@Deprecated // See Javadoc above.
EntityManager getEntityManager();
/**

View File

@@ -91,6 +91,26 @@ public class JpaTransactionManagerImpl implements JpaTransactionManager {
private static final ThreadLocal<TransactionInfo> transactionInfo =
ThreadLocal.withInitial(TransactionInfo::new);
/** Returns true if inside a transaction; returns false otherwise. */
public static boolean isInTransaction() {
return transactionInfo.get().inTransaction;
}
/**
* Returns the {@link EntityManager} for the current database transaction.
*
* <p>This method must be call from inside a transaction.
*/
public static EntityManager em() {
EntityManager entityManager = transactionInfo.get().entityManager;
if (entityManager == null) {
throw new PersistenceException(
"No EntityManager has been initialized. getEntityManager() must be invoked in the scope"
+ " of a transaction");
}
return entityManager;
}
public JpaTransactionManagerImpl(EntityManagerFactory emf, Clock clock, boolean readOnly) {
this.emf = emf;
this.clock = clock;
@@ -111,6 +131,7 @@ public class JpaTransactionManagerImpl implements JpaTransactionManager {
return emf.createEntityManager();
}
@Deprecated // See Javadoc of interface method.
@Override
public EntityManager getEntityManager() {
EntityManager entityManager = transactionInfo.get().entityManager;
@@ -137,6 +158,7 @@ public class JpaTransactionManagerImpl implements JpaTransactionManager {
return getEntityManager().createQuery(sqlString);
}
@Deprecated // See Javadoc of instance method.
@Override
public boolean inTransaction() {
return transactionInfo.get().inTransaction;

View File

@@ -36,7 +36,14 @@ public interface TransactionManager {
*
* <p>Note that this function is kept for backward compatibility. We will review the use case
* later when adding the cloud sql implementation.
*
* @deprecated Use the static {@link JpaTransactionManagerImpl#isInTransaction()} method for now.
* In current implementation the entity manager is obtained from a static {@code ThreadLocal}
* object that is set up by the outermost {@link #transact} call. As an instance method, this
* method gives the illusion that the call site has control over which database instance to
* use.
*/
@Deprecated // See Javadoc above.
boolean inTransaction();
/**

View File

@@ -64,12 +64,24 @@ public class ConfigureTldCommand extends MutatingCommand {
Path inputFile;
@Parameter(
names = {"-b", "--breakglass"},
names = {"-b", "--break_glass"},
description =
"Sets the breakglass field on the TLD to true, preventing Cloud Build from overwriting"
+ " these new changes until the TLD configuration file stored internally matches the"
+ " configuration in the database.")
boolean breakglass;
"Sets the breakGlass field on the TLD which prevents Cloud Build from overwriting new"
+ " TLD changes until the TLD configuration file stored internally matches the"
+ " configuration in the database. Setting this flag to false will indicate that"
+ " break glass mode should be turned off and any locally made changes to the"
+ " database should be overwritten by the internally stored configurations on the"
+ " next scheduled TLD sync.",
arity = 1)
Boolean breakGlass;
@Parameter(
names = {"--build_environment"},
description =
"DO NOT USE THIS FLAG ON THE COMMAND LINE! This flag indicates the command is being run"
+ " by the build environment tools. This flag should never be used by a human user"
+ " from the command line.")
boolean buildEnv;
@Parameter(
names = {"-d", "--dry_run"},
@@ -86,10 +98,10 @@ public class ConfigureTldCommand extends MutatingCommand {
boolean newDiff = true;
/**
* Indicates if the existing TLD is currently in breakglass mode and should not be modified unless
* the breakglass flag is used
* Indicates if the existing TLD is currently in break glass mode and should not be modified
* unless the --break_glass flag is used
*/
boolean oldTldInBreakglass = false;
boolean oldTldInBreakGlass = false;
@Override
protected void init() throws Exception {
@@ -100,23 +112,29 @@ public class ConfigureTldCommand extends MutatingCommand {
Tld oldTld = getTlds().contains(name) ? Tld.get(name) : null;
Tld newTld = mapper.readValue(inputFile.toFile(), Tld.class);
if (oldTld != null) {
oldTldInBreakglass = oldTld.getBreakglassMode();
oldTldInBreakGlass = oldTld.getBreakglassMode();
newDiff = !oldTld.equalYaml(newTld);
}
if (!newDiff && !oldTldInBreakglass) {
if (!newDiff && !oldTldInBreakGlass) {
// Don't construct a new object if there is no new diff
return;
}
if (oldTldInBreakglass && !breakglass) {
checkArgument(
oldTldInBreakGlass || !Boolean.FALSE.equals(breakGlass),
"The --break_glass flag cannot be set to false to end break glass mode because the TLD is"
+ " not currently in break glass mode");
if (oldTldInBreakGlass && !Boolean.TRUE.equals(breakGlass)) {
checkArgument(
!newDiff,
"Changes can not be applied since TLD is in breakglass mode but the breakglass flag was"
+ " not used");
!newDiff || breakGlass != null,
"Changes can not be applied since TLD is in break glass mode but the --break_glass flag"
+ " was not used");
// if there are no new diffs, then the YAML file has caught up to the database and the
// breakglass mode should be removed
logger.atInfo().log("Breakglass mode removed from TLD: %s", name);
// break glass mode should be removed. Also remove the break glass mode if the break glass
// flag was set to false.
logger.atInfo().log("Break glass mode removed from TLD: %s", name);
}
checkPremiumList(newTld);
@@ -129,8 +147,11 @@ public class ConfigureTldCommand extends MutatingCommand {
if (bsaEnrollTime.isPresent()) {
newTld = newTld.asBuilder().setBsaEnrollStartTime(bsaEnrollTime).build();
}
// Set the new TLD to breakglass mode if breakglass flag was used
if (breakglass) {
// Set the new TLD to break glass mode if break glass flag was used. Note that the break glass
// mode does not need to be set to false if the --break_glass flag was set to false since the
// field already defaults to false. Break glass mode will also automatically turn off if there
// is no new diff and the --break_glass flag is null.
if (Boolean.TRUE.equals(breakGlass)) {
newTld = newTld.asBuilder().setBreakglassMode(true).build();
}
stageEntityChange(oldTld, newTld);
@@ -142,14 +163,15 @@ public class ConfigureTldCommand extends MutatingCommand {
return true;
}
if (!newDiff) {
if (oldTldInBreakglass && !breakglass) {
// Run command to remove breakglass mode
if (oldTldInBreakGlass && (breakGlass == null || !breakGlass)) {
// Run command to remove break glass mode if there is no break glass flag or if the break
// glass flag is false.
return false;
}
logger.atInfo().log("TLD YAML file contains no new changes");
checkArgument(
!breakglass || oldTldInBreakglass,
"Breakglass mode can only be set when making new changes to a TLD configuration");
breakGlass == null || oldTldInBreakGlass,
"Break glass mode can only be set when making new changes to a TLD configuration");
return true;
}
return false;

View File

@@ -1,485 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.tools;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.tools.UpdateOrDeleteAllocationTokensCommand.getTokenKeys;
import static google.registry.util.CollectionUtils.findDuplicates;
import static google.registry.util.DomainNameUtils.canonicalizeHostname;
import com.beust.jcommander.Parameter;
import com.google.common.base.Joiner;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedMap;
import com.google.common.collect.Sets;
import com.google.common.collect.Sets.SetView;
import google.registry.model.pricing.StaticPremiumListPricingEngine;
import google.registry.model.tld.Tld;
import google.registry.model.tld.Tld.TldState;
import google.registry.model.tld.Tld.TldType;
import google.registry.model.tld.Tlds;
import google.registry.model.tld.label.PremiumList;
import google.registry.model.tld.label.PremiumListDao;
import google.registry.tldconfig.idn.IdnTableEnum;
import google.registry.tools.params.OptionalStringParameter;
import google.registry.tools.params.StringListParameter;
import google.registry.tools.params.TransitionListParameter.BillingCostTransitions;
import google.registry.tools.params.TransitionListParameter.TldStateTransitions;
import java.io.UnsupportedEncodingException;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.Set;
import javax.annotation.Nullable;
import javax.inject.Inject;
import javax.inject.Named;
import org.joda.money.Money;
import org.joda.time.DateTime;
import org.joda.time.Duration;
/** Shared base class for commands to create or update a TLD. */
abstract class CreateOrUpdateTldCommand extends MutatingCommand {
@Inject
@Named("dnsWriterNames")
Set<String> validDnsWriterNames;
@Parameter(description = "Names of the TLDs", required = true)
List<String> mainParameters;
@Parameter(
names = "--escrow",
description = "Whether to enable nightly RDE escrow deposits",
arity = 1)
private Boolean escrow;
@Parameter(
names = "--dns",
description = "Set to false to pause writing to the DNS queue",
arity = 1)
private Boolean dns;
@Nullable
@Parameter(
names = "--add_grace_period",
description = "Length of the add grace period (in ISO 8601 duration format)")
Duration addGracePeriod;
@Nullable
@Parameter(
names = "--redemption_grace_period",
description = "Length of the redemption grace period (in ISO 8601 duration format)")
Duration redemptionGracePeriod;
@Nullable
@Parameter(
names = "--pending_delete_length",
description = "Length of the pending delete period (in ISO 8601 duration format)")
Duration pendingDeleteLength;
@Nullable
@Parameter(
names = "--automatic_transfer_length",
description = "Length of the automatic transfer period (in ISO 8601 duration format)")
private Duration automaticTransferLength;
@Nullable
@Parameter(
names = "--restore_billing_cost",
description = "One-time billing cost for restoring a domain")
private Money restoreBillingCost;
@Nullable
@Parameter(
names = "--roid_suffix",
description = "The suffix to be used for ROIDs, e.g. COM for .com domains (which then "
+ "creates roids looking like 123ABC-COM)")
String roidSuffix;
@Nullable
@Parameter(
names = "--server_status_change_cost",
description = "One-time billing cost for a server status change")
private Money serverStatusChangeCost;
@Nullable
@Parameter(
names = "--registry_lock_or_unlock_cost",
description = "One-time billing cost for a registry lock or unlock")
private Money registryLockOrUnlockCost;
@Nullable
@Parameter(
names = "--tld_type",
description = "Tld type (REAL or TEST)")
private TldType tldType;
@Nullable
@Parameter(
names = "--invoicing_enabled",
description = "Whether invoicing is enabled for this tld.",
arity = 1)
private Boolean invoicingEnabled;
@Nullable
@Parameter(
names = "--create_billing_cost",
description = "Per-year billing cost for creating a domain")
Money createBillingCost;
@Nullable
@Parameter(
names = "--drive_folder_id",
description = "Id of the folder in drive used to publish information for this TLD",
converter = OptionalStringParameter.class,
validateWith = OptionalStringParameter.class)
Optional<String> driveFolderId;
@Nullable
@Parameter(
names = "--lordn_username",
description = "Username for LORDN uploads",
converter = OptionalStringParameter.class,
validateWith = OptionalStringParameter.class)
Optional<String> lordnUsername;
@Nullable
@Parameter(
names = "--premium_list",
description = "The name of the premium list to apply to the TLD",
converter = OptionalStringParameter.class,
validateWith = OptionalStringParameter.class)
Optional<String> premiumListName;
@Parameter(
names = "--tld_state_transitions",
converter = TldStateTransitions.class,
validateWith = TldStateTransitions.class,
description = "Comma-delimited list of TLD state transitions, of the form "
+ "<time>=<tld-state>[,<time>=<tld-state>]*")
ImmutableSortedMap<DateTime, TldState> tldStateTransitions = ImmutableSortedMap.of();
@Parameter(
names = "--renew_billing_cost_transitions",
converter = BillingCostTransitions.class,
validateWith = BillingCostTransitions.class,
description = "Comma-delimited list of renew billing cost transitions, of the form "
+ "<time>=<money-amount>[,<time>=<money-amount>]* where each amount "
+ "represents the per-year billing cost for renewing a domain")
ImmutableSortedMap<DateTime, Money> renewBillingCostTransitions =
ImmutableSortedMap.of();
@Parameter(
names = "--eap_fee_schedule",
converter = BillingCostTransitions.class,
validateWith = BillingCostTransitions.class,
description = "Comma-delimited list of EAP fees effective on specific dates, of the form "
+ "<time>=<money-amount>[,<time>=<money-amount>]* where each amount represents the "
+ "EAP fee for creating a new domain under the TLD.")
ImmutableSortedMap<DateTime, Money> eapFeeSchedule = ImmutableSortedMap.of();
@Nullable
@Parameter(
names = "--reserved_lists",
description = "A comma-separated list of reserved list names to be applied to the TLD",
listConverter = StringListParameter.class)
List<String> reservedListNames;
@Nullable
@Parameter(
names = "--allowed_registrants",
description = "A comma-separated list of allowed registrants for the TLD",
listConverter = StringListParameter.class)
List<String> allowedRegistrants;
@Nullable
@Parameter(
names = "--allowed_nameservers",
description = "A comma-separated list of allowed nameservers for the TLD",
listConverter = StringListParameter.class)
List<String> allowedNameservers;
@Parameter(
names = {"-o", "--override_reserved_list_rules"},
description = "Override restrictions on reserved list naming")
boolean overrideReservedListRules;
@Nullable
@Parameter(
names = "--claims_period_end",
description = "The end of the claims period")
DateTime claimsPeriodEnd;
@Nullable
@Parameter(
names = "--dns_writers",
description = "A comma-separated list of DnsWriter implementations to use",
listConverter = StringListParameter.class)
List<String> dnsWriters;
@Nullable
@Parameter(
names = {"--num_dns_publish_locks"},
description =
"The number of publish locks we allow in parallel for DNS updates under this tld "
+ "(1 for TLD-wide locks)",
arity = 1)
Integer numDnsPublishShards;
@Nullable
@Parameter(
names = {"--dns_a_plus_aaaa_ttl"},
description = "The time to live for DNS A and AAAA records (Ex: PT240S)")
Duration dnsAPlusAaaaTtl;
@Nullable
@Parameter(
names = {"--dns_ns_ttl"},
description = "The time to live for DNS NS records (Ex: PT240S)")
Duration dnsNsTtl;
@Nullable
@Parameter(
names = {"--dns_ds_ttl"},
description = "The time to live for DNS DS records (Ex: PT240S)")
Duration dnsDsTtl;
@Nullable
@Parameter(
names = "--default_tokens",
description =
"A comma-separated list of default allocation tokens to be applied to the TLD. The"
+ " ordering of this list will determine which token is used in the case where"
+ " multiple tokens are valid for a registration. Use an empty string to clear all"
+ " present default tokens.",
listConverter = StringListParameter.class)
List<String> defaultTokens;
@Nullable
@Parameter(
names = "--idn_tables",
description =
"A comma-separated list of the IDN tables to use for this TLD. Specify an empty list to"
+ " remove any previously-set tables and to use the default. All elements must be"
+ " IdnTableEnum values",
listConverter = StringListParameter.class)
List<String> idnTables;
/** Returns the existing tld (for update) or null (for creates). */
@Nullable
abstract Tld getOldTld(String tld);
abstract ImmutableSet<String> getAllowedRegistrants(Tld oldTld);
abstract ImmutableSet<String> getAllowedNameservers(Tld oldTld);
abstract ImmutableSet<String> getReservedLists(Tld oldTld);
abstract Optional<Map.Entry<DateTime, TldState>> getTldStateTransitionToAdd();
/** Subclasses can override this to set their own properties. */
void setCommandSpecificProperties(@SuppressWarnings("unused") Tld.Builder builder) {}
/** Subclasses can override this to assert that the command can be run in this environment. */
void assertAllowedEnvironment() {}
protected abstract void initTldCommand();
@Override
protected final void init() throws UnsupportedEncodingException {
assertAllowedEnvironment();
initTldCommand();
String duplicates = Joiner.on(", ").join(findDuplicates(mainParameters));
checkArgument(duplicates.isEmpty(), "Duplicate arguments found: '%s'", duplicates);
Set<String> tlds = ImmutableSet.copyOf(mainParameters);
checkArgument(roidSuffix == null || tlds.size() == 1,
"Can't update roid suffixes on multiple TLDs simultaneously");
for (String tld : tlds) {
checkArgument(
tld.equals(canonicalizeHostname(tld)),
"TLD '%s' should be given in the canonical form '%s'",
tld,
canonicalizeHostname(tld));
checkArgument(
!Character.isDigit(tld.charAt(0)),
"TLDs cannot begin with a number");
Tld oldTld = getOldTld(tld);
// TODO(b/26901539): Add a flag to set the pricing engine once we have more than one option.
Tld.Builder builder =
oldTld == null
? new Tld.Builder()
.setTldStr(tld)
.setPremiumPricingEngine(StaticPremiumListPricingEngine.NAME)
: oldTld.asBuilder();
if (escrow != null) {
builder.setEscrowEnabled(escrow);
}
if (dns != null) {
builder.setDnsPaused(!dns);
}
Optional<Map.Entry<DateTime, TldState>> tldStateTransitionToAdd =
getTldStateTransitionToAdd();
if (!tldStateTransitions.isEmpty()) {
builder.setTldStateTransitions(tldStateTransitions);
} else if (tldStateTransitionToAdd.isPresent()) {
ImmutableSortedMap.Builder<DateTime, TldState> newTldStateTransitions =
ImmutableSortedMap.naturalOrder();
if (oldTld != null) {
checkArgument(
oldTld
.getTldStateTransitions()
.lastKey()
.isBefore(tldStateTransitionToAdd.get().getKey()),
"Cannot add %s at %s when there is a later transition already scheduled",
tldStateTransitionToAdd.get().getValue(),
tldStateTransitionToAdd.get().getKey());
newTldStateTransitions.putAll(oldTld.getTldStateTransitions());
}
builder.setTldStateTransitions(
newTldStateTransitions.put(getTldStateTransitionToAdd().get()).build());
}
if (!renewBillingCostTransitions.isEmpty()) {
// TODO(b/20764952): need invoicing support for multiple renew billing costs.
if (renewBillingCostTransitions.size() > 1) {
errorPrintStream.println(
"----------------------\n"
+ "WARNING: Do not set multiple renew cost transitions "
+ "until b/20764952 is fixed.\n"
+ "----------------------\n");
}
builder.setRenewBillingCostTransitions(renewBillingCostTransitions);
}
if (!eapFeeSchedule.isEmpty()) {
builder.setEapFeeSchedule(eapFeeSchedule);
}
Optional.ofNullable(addGracePeriod).ifPresent(builder::setAddGracePeriodLength);
Optional.ofNullable(redemptionGracePeriod).ifPresent(builder::setRedemptionGracePeriodLength);
Optional.ofNullable(pendingDeleteLength).ifPresent(builder::setPendingDeleteLength);
Optional.ofNullable(automaticTransferLength).ifPresent(builder::setAutomaticTransferLength);
Optional.ofNullable(driveFolderId).ifPresent(id -> builder.setDriveFolderId(id.orElse(null)));
Optional.ofNullable(createBillingCost).ifPresent(builder::setCreateBillingCost);
Optional.ofNullable(restoreBillingCost).ifPresent(builder::setRestoreBillingCost);
Optional.ofNullable(roidSuffix).ifPresent(builder::setRoidSuffix);
Optional.ofNullable(serverStatusChangeCost)
.ifPresent(builder::setServerStatusChangeBillingCost);
Optional.ofNullable(registryLockOrUnlockCost)
.ifPresent(builder::setRegistryLockOrUnlockBillingCost);
Optional.ofNullable(tldType).ifPresent(builder::setTldType);
Optional.ofNullable(invoicingEnabled).ifPresent(builder::setInvoicingEnabled);
Optional.ofNullable(lordnUsername).ifPresent(u -> builder.setLordnUsername(u.orElse(null)));
Optional.ofNullable(claimsPeriodEnd).ifPresent(builder::setClaimsPeriodEnd);
Optional.ofNullable(numDnsPublishShards).ifPresent(builder::setNumDnsPublishLocks);
Optional.ofNullable(dnsAPlusAaaaTtl).ifPresent(builder::setDnsAPlusAaaaTtl);
Optional.ofNullable(dnsNsTtl).ifPresent(builder::setDnsNsTtl);
Optional.ofNullable(dnsDsTtl).ifPresent(builder::setDnsDsTtl);
if (premiumListName != null) {
if (premiumListName.isPresent()) {
Optional<PremiumList> premiumList =
PremiumListDao.getLatestRevision(premiumListName.get());
checkArgument(
premiumList.isPresent(),
String.format("The premium list '%s' doesn't exist", premiumListName.get()));
builder.setPremiumList(premiumList.get());
} else {
builder.setPremiumList(null);
}
}
if (dnsWriters != null) {
ImmutableSet<String> dnsWritersSet = ImmutableSet.copyOf(dnsWriters);
SetView<String> invalidDnsWriters = Sets.difference(dnsWritersSet, validDnsWriterNames);
checkArgument(
invalidDnsWriters.isEmpty(),
"Invalid DNS writer name(s) specified: %s",
invalidDnsWriters);
builder.setDnsWriters(dnsWritersSet);
}
ImmutableSet<String> newReservedListNames = getReservedLists(oldTld);
checkReservedListValidityForTld(tld, newReservedListNames);
builder.setReservedListsByName(newReservedListNames);
builder.setAllowedRegistrantContactIds(getAllowedRegistrants(oldTld));
builder.setAllowedFullyQualifiedHostNames(getAllowedNameservers(oldTld));
if (defaultTokens != null) {
builder.setDefaultPromoTokens(getTokenKeys(defaultTokens, null));
}
if (idnTables != null) {
if (idnTables.equals(ImmutableList.of(""))) {
builder.setIdnTables(ImmutableSet.of());
} else {
ImmutableSet<String> upperCaseIdnTables =
idnTables.stream().map(String::toUpperCase).collect(toImmutableSet());
ImmutableSet<String> validIdnStringValues =
Arrays.stream(IdnTableEnum.values()).map(Enum::name).collect(toImmutableSet());
checkArgument(
validIdnStringValues.containsAll(upperCaseIdnTables),
"IDN tables %s contained invalid value(s). Possible values: %s",
upperCaseIdnTables,
validIdnStringValues);
builder.setIdnTables(
upperCaseIdnTables.stream().map(IdnTableEnum::valueOf).collect(toImmutableSet()));
}
}
// Update the Registry object.
setCommandSpecificProperties(builder);
stageEntityChange(oldTld, builder.build());
}
}
@Override
public String execute() throws Exception {
try {
return super.execute();
} finally {
// Manually reset the cache here so that subsequent commands (e.g. in SetupOteCommand) see
// the latest version of the data.
// TODO(b/24903801): change all those places to use uncached code paths to get TLDs.
Tlds.resetCache();
}
}
private void checkReservedListValidityForTld(String tld, Set<String> reservedListNames) {
ImmutableList.Builder<String> builder = new ImmutableList.Builder<>();
for (String reservedListName : reservedListNames) {
if (!reservedListName.startsWith("common_") && !reservedListName.startsWith(tld + "_")) {
builder.add(reservedListName);
}
}
ImmutableList<String> invalidNames = builder.build();
if (!invalidNames.isEmpty()) {
String errMsg = String.format("The reserved list(s) %s cannot be applied to the tld %s",
Joiner.on(", ").join(invalidNames),
tld);
if (overrideReservedListRules) {
errorPrintStream.println("Error overridden: " + errMsg);
} else {
throw new IllegalArgumentException(errMsg);
}
}
}
}

View File

@@ -1,113 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.tools;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkState;
import static google.registry.model.tld.Tlds.getTlds;
import static google.registry.util.CollectionUtils.nullToEmpty;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import com.beust.jcommander.Parameter;
import com.beust.jcommander.Parameters;
import com.google.common.base.Strings;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedMap;
import com.google.common.collect.Maps;
import google.registry.model.tld.Tld;
import google.registry.model.tld.Tld.TldState;
import java.util.Map;
import java.util.Optional;
import javax.annotation.Nullable;
import org.joda.money.CurrencyUnit;
import org.joda.money.Money;
import org.joda.time.DateTime;
/** Command to create a TLD. */
@Parameters(separators = " =", commandDescription = "Create new TLD(s)")
class CreateTldCommand extends CreateOrUpdateTldCommand {
@Nullable
@Parameter(
names = "--initial_tld_state",
description = "Initial state of the TLD (cannot be combined with a transitions list)")
TldState initialTldState;
@Nullable
@Parameter(
names = "--initial_renew_billing_cost",
description = "Initial per-year billing cost for renewing a domain "
+ "(cannot be combined with a transitions list)")
private Money initialRenewBillingCost;
@Override
protected void initTldCommand() {
checkArgument(initialTldState == null || tldStateTransitions.isEmpty(),
"Don't pass both --initial_tld_state and --tld_state_transitions");
checkArgument(initialRenewBillingCost == null || renewBillingCostTransitions.isEmpty(),
"Don't pass both --initial_renew_billing_cost and --renew_billing_cost_transitions");
if (initialRenewBillingCost != null) {
renewBillingCostTransitions = ImmutableSortedMap.of(START_OF_TIME, initialRenewBillingCost);
}
checkArgument(mainParameters.size() == 1, "Can't create more than one TLD at a time");
checkArgument(
!Strings.isNullOrEmpty(roidSuffix),
"The roid suffix is required when creating a TLD");
}
@Override
void setCommandSpecificProperties(Tld.Builder builder) {
// Pick up the currency from the create cost. Since all costs must be in one currency, and that
// condition is enforced by the builder, it doesn't matter which cost we choose it from.
CurrencyUnit currency =
createBillingCost != null ? createBillingCost.getCurrencyUnit() : Tld.DEFAULT_CURRENCY;
builder.setCurrency(currency);
// If this is a non-default currency and the user hasn't specified an EAP fee schedule, set the
// EAP fee schedule to a matching currency.
if (!currency.equals(Tld.DEFAULT_CURRENCY) && eapFeeSchedule.isEmpty()) {
builder.setEapFeeSchedule(ImmutableSortedMap.of(START_OF_TIME, Money.zero(currency)));
}
}
@Override
Tld getOldTld(String tld) {
checkState(!getTlds().contains(tld), "TLD '%s' already exists", tld);
return null;
}
@Override
ImmutableSet<String> getAllowedRegistrants(Tld oldTld) {
return ImmutableSet.copyOf(nullToEmpty(allowedRegistrants));
}
@Override
ImmutableSet<String> getAllowedNameservers(Tld oldTld) {
return ImmutableSet.copyOf(nullToEmpty(allowedNameservers));
}
@Override
ImmutableSet<String> getReservedLists(Tld oldTld) {
return ImmutableSet.copyOf(nullToEmpty(reservedListNames));
}
@Override
Optional<Map.Entry<DateTime, TldState>> getTldStateTransitionToAdd() {
return initialTldState != null
? Optional.of(Maps.immutableEntry(START_OF_TIME, initialTldState))
: Optional.empty();
}
}

View File

@@ -49,7 +49,6 @@ public final class RegistryTool {
.put("create_registrar", CreateRegistrarCommand.class)
.put("create_registrar_groups", CreateRegistrarGroupsCommand.class)
.put("create_reserved_list", CreateReservedListCommand.class)
.put("create_tld", CreateTldCommand.class)
.put("create_user", CreateUserCommand.class)
.put("curl", CurlCommand.class)
.put("delete_allocation_tokens", DeleteAllocationTokensCommand.class)
@@ -114,7 +113,6 @@ public final class RegistryTool {
.put("update_registrar", UpdateRegistrarCommand.class)
.put("update_reserved_list", UpdateReservedListCommand.class)
.put("update_server_locks", UpdateServerLocksCommand.class)
.put("update_tld", UpdateTldCommand.class)
.put("update_user", UpdateUserCommand.class)
.put("upload_claims_list", UploadClaimsListCommand.class)
.put("validate_escrow_deposit", ValidateEscrowDepositCommand.class)

View File

@@ -102,8 +102,6 @@ interface RegistryToolComponent {
void inject(CreateRegistrarCommand command);
void inject(CreateTldCommand command);
void inject(EncryptEscrowDepositCommand command);
void inject(EnqueuePollMessageCommand command);
@@ -162,8 +160,6 @@ interface RegistryToolComponent {
void inject(UpdateRegistrarCommand command);
void inject(UpdateTldCommand command);
void inject(ValidateEscrowDepositCommand command);
void inject(ValidateLoginCredentialsCommand command);

View File

@@ -1,189 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.tools;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.Sets.difference;
import static com.google.common.collect.Sets.intersection;
import static com.google.common.collect.Sets.union;
import static google.registry.model.tld.Tlds.assertTldExists;
import static google.registry.util.CollectionUtils.nullToEmpty;
import com.beust.jcommander.Parameter;
import com.beust.jcommander.Parameters;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Maps;
import google.registry.model.tld.Tld;
import google.registry.model.tld.Tld.TldState;
import google.registry.tools.params.StringListParameter;
import google.registry.util.RegistryEnvironment;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.Set;
import javax.annotation.Nullable;
import org.joda.time.DateTime;
import org.joda.time.DateTimeZone;
/** Command to update a TLD. */
@Parameters(separators = " =", commandDescription = "Update existing TLD(s)")
public class UpdateTldCommand extends CreateOrUpdateTldCommand {
@Nullable
@Parameter(
names = "--add_reserved_lists",
description = "A comma-separated list of reserved list names to be added to the TLD",
listConverter = StringListParameter.class)
List<String> reservedListsAdd;
@Nullable
@Parameter(
names = "--remove_reserved_lists",
description = "A comma-separated list of reserved list names to be removed from the TLD",
listConverter = StringListParameter.class)
List<String> reservedListsRemove;
@Nullable
@Parameter(
names = "--add_allowed_registrants",
description = "A comma-separated list of allowed registrants to be added to the TLD",
listConverter = StringListParameter.class)
List<String> allowedRegistrantsAdd;
@Nullable
@Parameter(
names = "--remove_allowed_registrants",
description = "A comma-separated list of allowed registrants to be removed from the TLD",
listConverter = StringListParameter.class)
List<String> allowedRegistrantsRemove;
@Nullable
@Parameter(
names = "--add_allowed_nameservers",
description = "A comma-separated list of allowed nameservers to be added to the TLD",
listConverter = StringListParameter.class)
List<String> allowedNameserversAdd;
@Nullable
@Parameter(
names = "--remove_allowed_nameservers",
description = "A comma-separated list of allowed nameservers to be removed from the TLD",
listConverter = StringListParameter.class)
List<String> allowedNameserversRemove;
@Nullable
@Parameter(
names = "--set_current_tld_state",
description = "Set the current TLD state. Specifically, adds a TLD transition at the "
+ "current time for the specified state.")
TldState setCurrentTldState;
@Override
Tld getOldTld(String tld) {
return Tld.get(assertTldExists(tld));
}
@Override
ImmutableSet<String> getAllowedRegistrants(Tld oldTld) {
return formUpdatedList(
"allowed registrants",
oldTld.getAllowedRegistrantContactIds(),
allowedRegistrants,
allowedRegistrantsAdd,
allowedRegistrantsRemove);
}
@Override
ImmutableSet<String> getAllowedNameservers(Tld oldTld) {
return formUpdatedList(
"allowed nameservers",
oldTld.getAllowedFullyQualifiedHostNames(),
allowedNameservers,
allowedNameserversAdd,
allowedNameserversRemove);
}
@Override
ImmutableSet<String> getReservedLists(Tld oldTld) {
return formUpdatedList(
"reserved lists",
oldTld.getReservedListNames(),
reservedListNames,
reservedListsAdd,
reservedListsRemove);
}
@Override
Optional<Map.Entry<DateTime, TldState>> getTldStateTransitionToAdd() {
return setCurrentTldState != null
? Optional.of(Maps.immutableEntry(DateTime.now(DateTimeZone.UTC), setCurrentTldState))
: Optional.empty();
}
@Override
protected void initTldCommand() {
// Due to per-instance caching on Registry, different instances can end up in different TLD
// states at the same time, so --set_current_tld_state should never be used in production.
checkArgument(
!RegistryEnvironment.get().equals(RegistryEnvironment.PRODUCTION)
|| setCurrentTldState == null,
"--set_current_tld_state is not safe to use in production.");
checkConflicts("reserved_lists", reservedListNames, reservedListsAdd, reservedListsRemove);
checkConflicts(
"allowed_registrants", allowedRegistrants, allowedRegistrantsAdd, allowedRegistrantsRemove);
checkConflicts(
"allowed_nameservers", allowedNameservers, allowedNameserversAdd, allowedNameserversRemove);
checkArgument(setCurrentTldState == null || tldStateTransitions.isEmpty(),
"Don't pass both --set_current_tld_state and --tld_state_transitions");
}
private static ImmutableSet<String> formUpdatedList(
String description,
ImmutableSet<String> originals,
List<String> fullReplacement,
List<String> itemsToAdd,
List<String> itemsToRemove) {
if (fullReplacement != null) {
return ImmutableSet.copyOf(fullReplacement);
}
Set<String> toAdd = ImmutableSet.copyOf(nullToEmpty(itemsToAdd));
Set<String> toRemove = ImmutableSet.copyOf(nullToEmpty(itemsToRemove));
checkIsEmpty(
intersection(toAdd, toRemove),
String.format(
"Adding and removing the same %s simultaneously doesn't make sense", description));
checkIsEmpty(
intersection(originals, toAdd),
String.format("Cannot add %s that were previously present", description));
checkIsEmpty(
difference(toRemove, originals),
String.format("Cannot remove %s that were not previously present", description));
return ImmutableSet.copyOf(difference(union(originals, toAdd), toRemove));
}
private static void checkIsEmpty(Set<String> set, String errorString) {
checkArgument(set.isEmpty(), String.format("%s: %s", errorString, set));
}
private static void checkConflicts(
String baseFlagName, Object overwriteValue, Object addValue, Object removeValue) {
checkNotBoth(baseFlagName, overwriteValue, "add_" + baseFlagName, addValue);
checkNotBoth(baseFlagName, overwriteValue, "remove_" + baseFlagName, removeValue);
}
private static void checkNotBoth(String nameA, Object valueA, String nameB, Object valueB) {
checkArgument(valueA == null || valueB == null, "Don't pass both --%s and --%s", nameA, nameB);
}
}

View File

@@ -118,15 +118,8 @@ class BsaDownloadFunctionalTest {
gcsClient.readBlockList(downloadJob, BlockListType.BLOCK_PLUS)) {
assertThat(blockListFile).containsExactly(BSA_CSV_HEADER, "abc,2", "def,3");
}
ImmutableList<String> persistedLabels =
ImmutableList.copyOf(
tm().transact(
() ->
tm().getEntityManager()
.createNativeQuery("SELECT label from \"BsaLabel\"")
.getResultList()));
// TODO(weiminyu): check intermediate files
assertThat(persistedLabels).containsExactly("abc", "def");
assertThat(getPersistedLabels()).containsExactly("abc", "def");
}
@Test

View File

@@ -0,0 +1,319 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.bsa;
import static com.google.common.truth.Truth.assertThat;
import static com.google.common.truth.Truth8.assertThat;
import static google.registry.bsa.ReservedDomainsTestingUtils.addReservedDomainToList;
import static google.registry.bsa.ReservedDomainsTestingUtils.addReservedListsToTld;
import static google.registry.bsa.ReservedDomainsTestingUtils.createReservedList;
import static google.registry.bsa.ReservedDomainsTestingUtils.removeReservedDomainFromList;
import static google.registry.bsa.persistence.BsaTestingUtils.createDownloadScheduler;
import static google.registry.bsa.persistence.BsaTestingUtils.persistBsaLabel;
import static google.registry.bsa.persistence.BsaTestingUtils.queryUnblockableDomains;
import static google.registry.model.tld.Tlds.getTldEntitiesOfType;
import static google.registry.model.tld.label.ReservationType.RESERVED_FOR_SPECIFIC_USE;
import static google.registry.testing.DatabaseHelper.createTlds;
import static google.registry.testing.DatabaseHelper.deleteTestDomain;
import static google.registry.testing.DatabaseHelper.persistActiveDomain;
import static google.registry.testing.DatabaseHelper.persistResource;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.Mockito.never;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.verifyNoInteractions;
import com.google.cloud.storage.contrib.nio.testing.LocalStorageHelper;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import google.registry.bsa.api.BsaReportSender;
import google.registry.bsa.api.UnblockableDomain;
import google.registry.bsa.api.UnblockableDomain.Reason;
import google.registry.bsa.api.UnblockableDomainChange;
import google.registry.bsa.persistence.BsaTestingUtils;
import google.registry.gcs.GcsUtils;
import google.registry.model.domain.Domain;
import google.registry.model.tld.Tld.TldType;
import google.registry.persistence.transaction.JpaTestExtensions;
import google.registry.persistence.transaction.JpaTestExtensions.JpaIntegrationWithCoverageExtension;
import google.registry.request.Response;
import google.registry.testing.FakeClock;
import google.registry.testing.FakeLockHandler;
import google.registry.testing.FakeResponse;
import java.util.Optional;
import org.joda.time.DateTime;
import org.joda.time.Duration;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.junit.jupiter.api.extension.RegisterExtension;
import org.mockito.InOrder;
import org.mockito.Mock;
import org.mockito.Mockito;
import org.mockito.junit.jupiter.MockitoExtension;
/**
* Functional tests for refreshing the unblockable domains with recent registration and reservation
* changes.
*/
@ExtendWith(MockitoExtension.class)
class BsaRefreshFunctionalTest {
static final DateTime TEST_START_TIME = DateTime.parse("2024-01-01T00:00:00Z");
static final String RESERVED_LIST_NAME = "reserved";
private final FakeClock fakeClock = new FakeClock(TEST_START_TIME);
@RegisterExtension
JpaIntegrationWithCoverageExtension jpa =
new JpaTestExtensions.Builder().withClock(fakeClock).buildIntegrationWithCoverageExtension();
@Mock BsaReportSender bsaReportSender;
private GcsClient gcsClient;
private Response response;
private BsaRefreshAction action;
@BeforeEach
void setup() throws Exception {
gcsClient =
new GcsClient(new GcsUtils(LocalStorageHelper.getOptions()), "my-bucket", "SHA-256");
response = new FakeResponse();
action =
new BsaRefreshAction(
BsaTestingUtils.createRefreshScheduler(),
gcsClient,
bsaReportSender,
/* transactionBatchSize= */ 5,
/* domainCreateTxnCommitTimeLag= */ Duration.millis(1),
new BsaLock(
new FakeLockHandler(/* lockSucceeds= */ true), Duration.standardSeconds(30)),
fakeClock,
response);
initDb();
}
private String getRefreshJobName(DateTime jobStartTime) {
return jobStartTime.toString() + "-refresh";
}
private void initDb() {
createTlds("app", "dev");
getTldEntitiesOfType(TldType.REAL)
.forEach(
tld ->
persistResource(
tld.asBuilder().setBsaEnrollStartTime(Optional.of(START_OF_TIME)).build()));
createReservedList(RESERVED_LIST_NAME, "dummy", RESERVED_FOR_SPECIFIC_USE);
addReservedListsToTld("app", ImmutableList.of(RESERVED_LIST_NAME));
persistBsaLabel("blocked1");
persistBsaLabel("blocked2");
// Creates a download record so that refresher will not quit immediately.
createDownloadScheduler(fakeClock).schedule().get().updateJobStage(DownloadStage.DONE);
fakeClock.advanceOneMilli();
}
@Test
void newReservedDomain_addedAsUnblockable() throws Exception {
addReservedDomainToList(
RESERVED_LIST_NAME, ImmutableMap.of("blocked1", RESERVED_FOR_SPECIFIC_USE));
String jobName = getRefreshJobName(fakeClock.nowUtc());
action.run();
UnblockableDomain newUnblockable = UnblockableDomain.of("blocked1.app", Reason.RESERVED);
assertThat(queryUnblockableDomains()).containsExactly(newUnblockable);
assertThat(gcsClient.readRefreshChanges(jobName))
.containsExactly(UnblockableDomainChange.ofNew(newUnblockable));
verify(bsaReportSender, never()).removeUnblockableDomainsUpdates(anyString());
verify(bsaReportSender, times(1))
.addUnblockableDomainsUpdates("{\n \"reserved\": [\n \"blocked1.app\"\n ]\n}");
}
@Test
void newRegisteredDomain_addedAsUnblockable() throws Exception {
persistActiveDomain("blocked1.dev", fakeClock.nowUtc());
persistActiveDomain("dummy.dev", fakeClock.nowUtc());
String jobName = getRefreshJobName(fakeClock.nowUtc());
action.run();
UnblockableDomain newUnblockable = UnblockableDomain.of("blocked1.dev", Reason.REGISTERED);
assertThat(queryUnblockableDomains()).containsExactly(newUnblockable);
assertThat(gcsClient.readRefreshChanges(jobName))
.containsExactly(UnblockableDomainChange.ofNew(newUnblockable));
verify(bsaReportSender, never()).removeUnblockableDomainsUpdates(anyString());
verify(bsaReportSender, times(1))
.addUnblockableDomainsUpdates("{\n \"registered\": [\n \"blocked1.dev\"\n ]\n}");
}
@Test
void registeredUnblockable_unregistered() {
Domain domain = persistActiveDomain("blocked1.dev", fakeClock.nowUtc());
action.run();
assertThat(queryUnblockableDomains())
.containsExactly(UnblockableDomain.of("blocked1.dev", Reason.REGISTERED));
fakeClock.advanceOneMilli();
deleteTestDomain(domain, fakeClock.nowUtc());
fakeClock.advanceOneMilli();
String jobName = getRefreshJobName(fakeClock.nowUtc());
Mockito.reset(bsaReportSender);
action.run();
assertThat(queryUnblockableDomains()).isEmpty();
assertThat(gcsClient.readRefreshChanges(jobName))
.containsExactly(
UnblockableDomainChange.ofDeleted(
UnblockableDomain.of("blocked1.dev", Reason.REGISTERED)));
verify(bsaReportSender, never()).addUnblockableDomainsUpdates(anyString());
verify(bsaReportSender, times(1)).removeUnblockableDomainsUpdates("[\n \"blocked1.dev\"\n]");
}
@Test
void reservedUnblockable_noLongerReserved() {
addReservedDomainToList(
RESERVED_LIST_NAME, ImmutableMap.of("blocked1", RESERVED_FOR_SPECIFIC_USE));
action.run();
assertThat(queryUnblockableDomains())
.containsExactly(UnblockableDomain.of("blocked1.app", Reason.RESERVED));
fakeClock.advanceOneMilli();
removeReservedDomainFromList(RESERVED_LIST_NAME, ImmutableSet.of("blocked1"));
String jobName = getRefreshJobName(fakeClock.nowUtc());
Mockito.reset(bsaReportSender);
action.run();
assertThat(queryUnblockableDomains()).isEmpty();
assertThat(gcsClient.readRefreshChanges(jobName))
.containsExactly(
UnblockableDomainChange.ofDeleted(
UnblockableDomain.of("blocked1.app", Reason.RESERVED)));
verify(bsaReportSender, never()).addUnblockableDomainsUpdates(anyString());
verify(bsaReportSender, times(1)).removeUnblockableDomainsUpdates("[\n \"blocked1.app\"\n]");
}
@Test
void registeredAndReservedUnblockable_noLongerRegistered_stillUnblockable() throws Exception {
addReservedDomainToList(
RESERVED_LIST_NAME, ImmutableMap.of("blocked1", RESERVED_FOR_SPECIFIC_USE));
Domain domain = persistActiveDomain("blocked1.app", fakeClock.nowUtc());
action.run();
assertThat(queryUnblockableDomains())
.containsExactly(UnblockableDomain.of("blocked1.app", Reason.REGISTERED));
fakeClock.advanceOneMilli();
deleteTestDomain(domain, fakeClock.nowUtc());
fakeClock.advanceOneMilli();
String jobName = getRefreshJobName(fakeClock.nowUtc());
Mockito.reset(bsaReportSender);
action.run();
assertThat(queryUnblockableDomains())
.containsExactly(UnblockableDomain.of("blocked1.app", Reason.RESERVED));
assertThat(gcsClient.readRefreshChanges(jobName))
.containsExactly(
UnblockableDomainChange.ofChanged(
UnblockableDomain.of("blocked1.app", Reason.REGISTERED), Reason.RESERVED));
InOrder inOrder = Mockito.inOrder(bsaReportSender);
inOrder.verify(bsaReportSender).removeUnblockableDomainsUpdates("[\n \"blocked1.app\"\n]");
inOrder
.verify(bsaReportSender)
.addUnblockableDomainsUpdates("{\n \"reserved\": [\n \"blocked1.app\"\n ]\n}");
}
@Test
void reservedUblockable_becomesRegistered_changeToRegisterd() throws Exception {
addReservedDomainToList(
RESERVED_LIST_NAME, ImmutableMap.of("blocked1", RESERVED_FOR_SPECIFIC_USE));
action.run();
assertThat(queryUnblockableDomains())
.containsExactly(UnblockableDomain.of("blocked1.app", Reason.RESERVED));
fakeClock.advanceOneMilli();
persistActiveDomain("blocked1.app", fakeClock.nowUtc());
fakeClock.advanceOneMilli();
Mockito.reset(bsaReportSender);
String jobName = getRefreshJobName(fakeClock.nowUtc());
action.run();
UnblockableDomain changed = UnblockableDomain.of("blocked1.app", Reason.REGISTERED);
assertThat(queryUnblockableDomains()).containsExactly(changed);
assertThat(gcsClient.readRefreshChanges(jobName))
.containsExactly(
UnblockableDomainChange.ofChanged(
UnblockableDomain.of("blocked1.app", Reason.RESERVED), Reason.REGISTERED));
InOrder inOrder = Mockito.inOrder(bsaReportSender);
inOrder.verify(bsaReportSender).removeUnblockableDomainsUpdates("[\n \"blocked1.app\"\n]");
inOrder
.verify(bsaReportSender)
.addUnblockableDomainsUpdates("{\n \"registered\": [\n \"blocked1.app\"\n ]\n}");
}
@Test
void newRegisteredAndReservedDomain_addedAsRegisteredUnblockable() throws Exception {
addReservedDomainToList(
RESERVED_LIST_NAME, ImmutableMap.of("blocked1", RESERVED_FOR_SPECIFIC_USE));
persistActiveDomain("blocked1.app", fakeClock.nowUtc());
String jobName = getRefreshJobName(fakeClock.nowUtc());
action.run();
UnblockableDomain newUnblockable = UnblockableDomain.of("blocked1.app", Reason.REGISTERED);
assertThat(queryUnblockableDomains()).containsExactly(newUnblockable);
assertThat(gcsClient.readRefreshChanges(jobName))
.containsExactly(UnblockableDomainChange.ofNew(newUnblockable));
}
@Test
void registeredAndReservedUnblockable_noLongerReserved_noChange() throws Exception {
addReservedDomainToList(
RESERVED_LIST_NAME, ImmutableMap.of("blocked1", RESERVED_FOR_SPECIFIC_USE));
persistActiveDomain("blocked1.app", fakeClock.nowUtc());
action.run();
assertThat(queryUnblockableDomains())
.containsExactly(UnblockableDomain.of("blocked1.app", Reason.REGISTERED));
fakeClock.advanceOneMilli();
removeReservedDomainFromList(RESERVED_LIST_NAME, ImmutableSet.of("blocked1"));
fakeClock.advanceOneMilli();
Mockito.reset(bsaReportSender);
String jobName = getRefreshJobName(fakeClock.nowUtc());
action.run();
assertThat(queryUnblockableDomains())
.containsExactly(UnblockableDomain.of("blocked1.app", Reason.REGISTERED));
assertThat(gcsClient.readRefreshChanges(jobName)).isEmpty();
verifyNoInteractions(bsaReportSender);
}
@Test
void registeredUblockable_becomesReserved_noChange() throws Exception {
persistActiveDomain("blocked1.app", fakeClock.nowUtc());
action.run();
assertThat(queryUnblockableDomains())
.containsExactly(UnblockableDomain.of("blocked1.app", Reason.REGISTERED));
fakeClock.advanceOneMilli();
addReservedDomainToList(
RESERVED_LIST_NAME, ImmutableMap.of("blocked1", RESERVED_FOR_SPECIFIC_USE));
fakeClock.advanceOneMilli();
Mockito.reset(bsaReportSender);
String jobName = getRefreshJobName(fakeClock.nowUtc());
action.run();
assertThat(queryUnblockableDomains())
.containsExactly(UnblockableDomain.of("blocked1.app", Reason.REGISTERED));
assertThat(gcsClient.readRefreshChanges(jobName)).isEmpty();
verifyNoInteractions(bsaReportSender);
}
}

View File

@@ -0,0 +1,104 @@
// Copyright 2024 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.bsa;
import static google.registry.testing.DatabaseHelper.persistResource;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Maps;
import google.registry.model.tld.Tld;
import google.registry.model.tld.label.ReservationType;
import google.registry.model.tld.label.ReservedList;
import google.registry.model.tld.label.ReservedList.ReservedListEntry;
import google.registry.model.tld.label.ReservedListDao;
/** Helpers for setting up reserved lists in tests. */
public final class ReservedDomainsTestingUtils {
private ReservedDomainsTestingUtils() {}
public static void createReservedList(
String listName, ImmutableMap<String, ReservationType> reservedLabels) {
ImmutableMap<String, ReservedListEntry> entries =
ImmutableMap.copyOf(
Maps.transformEntries(
reservedLabels, (key, value) -> ReservedListEntry.create(key, value, "")));
ReservedListDao.save(
new ReservedList.Builder()
.setName(listName)
.setCreationTimestamp(START_OF_TIME)
.setShouldPublish(true)
.setReservedListMap(entries)
.build());
}
public static void createReservedList(
String listName, String label, ReservationType reservationType) {
createReservedList(listName, ImmutableMap.of(label, reservationType));
}
public static void addReservedListsToTld(String tldStr, ImmutableList<String> listNames) {
Tld tld = Tld.get(tldStr);
ImmutableSet<String> reservedLists =
new ImmutableSet.Builder<String>()
.addAll(tld.getReservedListNames())
.addAll(listNames)
.build();
persistResource(tld.asBuilder().setReservedListsByName(reservedLists).build());
}
public static void addReservedDomainToList(
String listName, ImmutableMap<String, ReservationType> reservedLabels) {
ImmutableMap<String, ReservedListEntry> existingEntries =
ReservedList.get(listName).get().getReservedListEntries();
ImmutableMap<String, ReservedListEntry> newEntries =
ImmutableMap.copyOf(
Maps.transformEntries(
reservedLabels, (key, value) -> ReservedListEntry.create(key, value, "")));
ReservedListDao.save(
new ReservedList.Builder()
.setName(listName)
.setCreationTimestamp(START_OF_TIME)
.setShouldPublish(true)
.setReservedListMap(
new ImmutableMap.Builder<String, ReservedListEntry>()
.putAll(existingEntries)
.putAll(newEntries)
.buildKeepingLast())
.build());
}
public static void removeReservedDomainFromList(
String listName, ImmutableSet<String> removedLabels) {
ImmutableMap<String, ReservedListEntry> existingEntries =
ReservedList.get(listName).get().getReservedListEntries();
ImmutableMap<String, ReservedListEntry> newEntries =
ImmutableMap.copyOf(
Maps.filterEntries(existingEntries, entry -> !removedLabels.contains(entry.getKey())));
ReservedListDao.save(
new ReservedList.Builder()
.setName(listName)
.setCreationTimestamp(START_OF_TIME)
.setShouldPublish(true)
.setReservedListMap(newEntries)
.build());
}
}

View File

@@ -15,6 +15,8 @@
package google.registry.bsa;
import static com.google.common.truth.Truth.assertThat;
import static google.registry.bsa.ReservedDomainsTestingUtils.addReservedListsToTld;
import static google.registry.bsa.ReservedDomainsTestingUtils.createReservedList;
import static google.registry.bsa.ReservedDomainsUtils.getAllReservedDomainsInTld;
import static google.registry.model.tld.Tld.TldState.GENERAL_AVAILABILITY;
import static google.registry.model.tld.Tld.TldState.START_DATE_SUNRISE;
@@ -26,13 +28,10 @@ import static google.registry.model.tld.label.ReservationType.RESERVED_FOR_SPECI
import static google.registry.testing.DatabaseHelper.createTld;
import static google.registry.testing.DatabaseHelper.persistResource;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedMap;
import google.registry.model.tld.Tld;
import google.registry.model.tld.label.ReservedList;
import google.registry.model.tld.label.ReservedList.ReservedListEntry;
import google.registry.model.tld.label.ReservedListDao;
import google.registry.persistence.transaction.JpaTestExtensions;
import google.registry.persistence.transaction.JpaTestExtensions.JpaIntegrationWithCoverageExtension;
import google.registry.testing.FakeClock;
@@ -51,41 +50,19 @@ class ReservedDomainsUtilsTest {
@BeforeEach
void setup() {
ImmutableMap<String, ReservedListEntry> byType =
createReservedList(
"testlist",
ImmutableMap.of(
"sunrise",
ReservedListEntry.create("sunrise", ALLOWED_IN_SUNRISE, ""),
"specific",
ReservedListEntry.create("specific", RESERVED_FOR_SPECIFIC_USE, ""),
"anchor",
ReservedListEntry.create("anchor", RESERVED_FOR_ANCHOR_TENANT, ""),
"fully",
ReservedListEntry.create("fully", FULLY_BLOCKED, ""),
"name",
ReservedListEntry.create("name", NAME_COLLISION, ""));
ImmutableMap<String, ReservedListEntry> altList =
"sunrise", ALLOWED_IN_SUNRISE,
"specific", RESERVED_FOR_SPECIFIC_USE,
"anchor", RESERVED_FOR_ANCHOR_TENANT,
"fully", FULLY_BLOCKED,
"name", NAME_COLLISION));
createReservedList(
"testlist2",
ImmutableMap.of(
"anchor",
ReservedListEntry.create("anchor", RESERVED_FOR_ANCHOR_TENANT, ""),
"somethingelse",
ReservedListEntry.create("somethingelse", RESERVED_FOR_ANCHOR_TENANT, ""));
ReservedListDao.save(
new ReservedList.Builder()
.setName("testlist")
.setCreationTimestamp(fakeClock.nowUtc())
.setShouldPublish(false)
.setReservedListMap(byType)
.build());
ReservedListDao.save(
new ReservedList.Builder()
.setName("testlist2")
.setCreationTimestamp(fakeClock.nowUtc())
.setShouldPublish(false)
.setReservedListMap(altList)
.build());
"anchor", RESERVED_FOR_ANCHOR_TENANT,
"somethingelse", RESERVED_FOR_ANCHOR_TENANT));
createTld("tld");
persistResource(
@@ -95,15 +72,11 @@ class ReservedDomainsUtilsTest {
ImmutableSortedMap.of(
fakeClock.nowUtc(), START_DATE_SUNRISE,
fakeClock.nowUtc().plusMillis(1), GENERAL_AVAILABILITY))
.setReservedListsByName(ImmutableSet.of("testlist"))
.build());
addReservedListsToTld("tld", ImmutableList.of("testlist"));
createTld("tld2");
persistResource(
Tld.get("tld2")
.asBuilder()
.setReservedListsByName(ImmutableSet.of("testlist", "testlist2"))
.build());
addReservedListsToTld("tld2", ImmutableList.of("testlist", "testlist2"));
}
@Test

View File

@@ -0,0 +1,107 @@
// Copyright 2024 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.bsa;
import static com.google.common.truth.Truth.assertThat;
import static google.registry.testing.DatabaseHelper.createTld;
import static google.registry.testing.DatabaseHelper.persistActiveDomain;
import static google.registry.testing.DatabaseHelper.persistDeletedDomain;
import static google.registry.testing.DatabaseHelper.persistReservedList;
import static google.registry.testing.DatabaseHelper.persistResource;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import static java.nio.charset.StandardCharsets.UTF_8;
import com.google.cloud.storage.BlobId;
import com.google.cloud.storage.contrib.nio.testing.LocalStorageHelper;
import google.registry.bsa.api.BsaCredential;
import google.registry.gcs.GcsUtils;
import google.registry.model.tld.Tld;
import google.registry.model.tld.Tld.TldType;
import google.registry.model.tld.label.ReservedList;
import google.registry.persistence.transaction.JpaTestExtensions;
import google.registry.persistence.transaction.JpaTestExtensions.JpaIntegrationTestExtension;
import google.registry.request.UrlConnectionService;
import google.registry.testing.FakeClock;
import google.registry.testing.FakeResponse;
import java.util.Optional;
import org.joda.time.DateTime;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.junit.jupiter.api.extension.RegisterExtension;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
/** Unit tests for {@link UploadBsaUnavailableDomainsAction}. */
@ExtendWith(MockitoExtension.class)
public class UploadBsaUnavailableDomainsActionTest {
private final FakeClock clock = new FakeClock(DateTime.parse("2024-02-02T02:02:02Z"));
@RegisterExtension
final JpaIntegrationTestExtension jpa =
new JpaTestExtensions.Builder().withClock(clock).buildIntegrationTestExtension();
private UploadBsaUnavailableDomainsAction action;
@Mock UrlConnectionService connectionService;
@Mock BsaCredential bsaCredential;
private final GcsUtils gcsUtils = new GcsUtils(LocalStorageHelper.getOptions());
private final String BUCKET = "domain-registry-bsa";
private final String API_URL = "https://upload.test/bsa";
private final FakeResponse response = new FakeResponse();
@BeforeEach
void beforeEach() {
ReservedList reservedList =
persistReservedList(
"tld-reserved_list",
true,
"tine,FULLY_BLOCKED",
"flagrant,NAME_COLLISION",
"jimmy,RESERVED_FOR_SPECIFIC_USE");
createTld("tld");
persistResource(
Tld.get("tld")
.asBuilder()
.setReservedLists(reservedList)
.setBsaEnrollStartTime(Optional.of(START_OF_TIME))
.setTldType(TldType.REAL)
.build());
action =
new UploadBsaUnavailableDomainsAction(
clock, bsaCredential, gcsUtils, BUCKET, API_URL, response);
}
@Test
void calculatesEntriesCorrectly() throws Exception {
persistActiveDomain("foobar.tld");
persistActiveDomain("ace.tld");
persistDeletedDomain("not-blocked.tld", clock.nowUtc().minusDays(1));
action.run();
BlobId existingFile =
BlobId.of(BUCKET, String.format("unavailable_domains_%s.txt", clock.nowUtc()));
String blockList = new String(gcsUtils.readBytesFrom(existingFile), UTF_8);
assertThat(blockList).isEqualTo("ace.tld\nflagrant.tld\nfoobar.tld\njimmy.tld\ntine.tld");
assertThat(blockList).doesNotContain("not-blocked.tld");
// TODO(mcilwain): Add test of BSA API upload as well.
}
}

View File

@@ -50,7 +50,7 @@ public class BsaLabelUtilsTest {
@Test
void isLabelBlocked_yes() {
persistBsaLabel("abc", fakeClock.nowUtc());
persistBsaLabel("abc");
assertThat(isLabelBlocked("abc")).isTrue();
}

View File

@@ -14,8 +14,11 @@
package google.registry.bsa.persistence;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableList;
import google.registry.bsa.api.UnblockableDomain;
import google.registry.util.Clock;
import org.joda.time.DateTime;
import org.joda.time.Duration;
@@ -26,13 +29,30 @@ public final class BsaTestingUtils {
public static final Duration DEFAULT_DOWNLOAD_INTERVAL = Duration.standardHours(1);
public static final Duration DEFAULT_NOP_INTERVAL = Duration.standardDays(1);
/** An arbitrary point of time used as BsaLabels' creation time. */
public static final DateTime BSA_LABEL_CREATION_TIME = DateTime.parse("2023-12-31T00:00:00Z");
private BsaTestingUtils() {}
public static void persistBsaLabel(String domainLabel, DateTime creationTime) {
tm().transact(() -> tm().put(new BsaLabel(domainLabel, creationTime)));
public static void persistBsaLabel(String domainLabel) {
tm().transact(() -> tm().put(new BsaLabel(domainLabel, BSA_LABEL_CREATION_TIME)));
}
public static void persistUnblockableDomain(UnblockableDomain unblockableDomain) {
tm().transact(() -> tm().put(BsaUnblockableDomain.of(unblockableDomain)));
}
public static DownloadScheduler createDownloadScheduler(Clock clock) {
return new DownloadScheduler(DEFAULT_DOWNLOAD_INTERVAL, DEFAULT_NOP_INTERVAL, clock);
}
public static RefreshScheduler createRefreshScheduler() {
return new RefreshScheduler();
}
public static ImmutableList<UnblockableDomain> queryUnblockableDomains() {
return tm().transact(() -> tm().loadAllOf(BsaUnblockableDomain.class)).stream()
.map(BsaUnblockableDomain::toUnblockableDomain)
.collect(toImmutableList());
}
}

View File

@@ -15,7 +15,8 @@
package google.registry.bsa.persistence;
import static com.google.common.truth.Truth.assertThat;
import static google.registry.bsa.BsaTransactions.bsaTransact;
import static google.registry.bsa.ReservedDomainsTestingUtils.addReservedListsToTld;
import static google.registry.bsa.ReservedDomainsTestingUtils.createReservedList;
import static google.registry.bsa.persistence.BsaTestingUtils.persistBsaLabel;
import static google.registry.model.tld.label.ReservationType.RESERVED_FOR_SPECIFIC_USE;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
@@ -24,15 +25,11 @@ import static google.registry.testing.DatabaseHelper.newDomain;
import static google.registry.testing.DatabaseHelper.persistResource;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableList;
import google.registry.bsa.api.UnblockableDomain;
import google.registry.bsa.api.UnblockableDomainChange;
import google.registry.bsa.persistence.BsaUnblockableDomain.Reason;
import google.registry.model.tld.Tld;
import google.registry.model.tld.label.ReservedList;
import google.registry.model.tld.label.ReservedList.ReservedListEntry;
import google.registry.model.tld.label.ReservedListDao;
import google.registry.persistence.transaction.JpaTestExtensions;
import google.registry.persistence.transaction.JpaTestExtensions.JpaIntegrationWithCoverageExtension;
import google.registry.testing.FakeClock;
@@ -44,7 +41,7 @@ import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;
/** Unit tests for {@link DomainsRefresher}. */
public class DomainRefresherTest {
public class DomainsRefresherTest {
FakeClock fakeClock = new FakeClock(DateTime.parse("2023-11-09T02:08:57.880Z"));
@@ -66,52 +63,54 @@ public class DomainRefresherTest {
}
@Test
void staleUnblockableRemoved_wasRegistered() {
persistBsaLabel("label", fakeClock.nowUtc().minus(Duration.standardDays(1)));
void registeredUnblockable_removed_afterDomainIsDeleted() {
persistBsaLabel("label");
tm().transact(() -> tm().insert(BsaUnblockableDomain.of("label.tld", Reason.REGISTERED)));
assertThat(bsaTransact(refresher::refreshStaleUnblockables))
assertThat(refresher.refreshStaleUnblockables())
.containsExactly(
UnblockableDomainChange.ofDeleted(
UnblockableDomain.of("label.tld", UnblockableDomain.Reason.REGISTERED)));
}
@Test
void staleUnblockableRemoved_wasReserved() {
persistBsaLabel("label", fakeClock.nowUtc().minus(Duration.standardDays(1)));
void reservedUnblockable_removed_whenReservedLabelIsRemoved() {
persistBsaLabel("label");
tm().transact(() -> tm().insert(BsaUnblockableDomain.of("label.tld", Reason.RESERVED)));
assertThat(bsaTransact(refresher::refreshStaleUnblockables))
assertThat(refresher.refreshStaleUnblockables())
.containsExactly(
UnblockableDomainChange.ofDeleted(
UnblockableDomain.of("label.tld", UnblockableDomain.Reason.RESERVED)));
}
@Test
void newUnblockableAdded_isRegistered() {
void regsiteredUnblockable_added_whenDomainIsAdded() {
persistResource(newDomain("label.tld"));
persistBsaLabel("label", fakeClock.nowUtc().minus(Duration.standardDays(1)));
assertThat(bsaTransact(refresher::getNewUnblockables))
persistBsaLabel("label");
assertThat(refresher.getNewUnblockables())
.containsExactly(
UnblockableDomainChange.ofNew(
UnblockableDomain.of("label.tld", UnblockableDomain.Reason.REGISTERED)));
}
@Test
void newUnblockableAdded_isReserved() {
persistBsaLabel("label", fakeClock.nowUtc().minus(Duration.standardDays(1)));
setReservedList("label");
assertThat(bsaTransact(refresher::getNewUnblockables))
void reservedUnblockable_added_whenReservedLabelIsAdded() {
persistBsaLabel("label");
createReservedList("reservedList", "label", RESERVED_FOR_SPECIFIC_USE);
addReservedListsToTld("tld", ImmutableList.of("reservedList"));
assertThat(refresher.getNewUnblockables())
.containsExactly(
UnblockableDomainChange.ofNew(
UnblockableDomain.of("label.tld", UnblockableDomain.Reason.RESERVED)));
}
@Test
void staleUnblockableDowngraded_registeredToReserved() {
persistBsaLabel("label", fakeClock.nowUtc().minus(Duration.standardDays(1)));
setReservedList("label");
void registeredUnblockable_changedToReserved_whenDomainIsDeletedButLabelIsReserved() {
persistBsaLabel("label");
createReservedList("reservedList", "label", RESERVED_FOR_SPECIFIC_USE);
addReservedListsToTld("tld", ImmutableList.of("reservedList"));
tm().transact(() -> tm().insert(BsaUnblockableDomain.of("label.tld", Reason.REGISTERED)));
assertThat(bsaTransact(refresher::refreshStaleUnblockables))
assertThat(refresher.refreshStaleUnblockables())
.containsExactly(
UnblockableDomainChange.ofChanged(
UnblockableDomain.of("label.tld", UnblockableDomain.Reason.REGISTERED),
@@ -119,12 +118,12 @@ public class DomainRefresherTest {
}
@Test
void staleUnblockableUpgraded_reservedToRegisteredButNotReserved() {
persistBsaLabel("label", fakeClock.nowUtc().minus(Duration.standardDays(1)));
void reservedUnblockableUpgraded_changedToRegistered_whenDomainIsCreatedButNoLongerReserved() {
persistBsaLabel("label");
tm().transact(() -> tm().insert(BsaUnblockableDomain.of("label.tld", Reason.RESERVED)));
persistResource(newDomain("label.tld"));
assertThat(bsaTransact(refresher::refreshStaleUnblockables))
assertThat(refresher.refreshStaleUnblockables())
.containsExactly(
UnblockableDomainChange.ofChanged(
UnblockableDomain.of("label.tld", UnblockableDomain.Reason.RESERVED),
@@ -132,31 +131,17 @@ public class DomainRefresherTest {
}
@Test
void staleUnblockableUpgraded_wasReserved_isReservedAndRegistered() {
persistBsaLabel("label", fakeClock.nowUtc().minus(Duration.standardDays(1)));
setReservedList("label");
void reservedUnblockableUpgraded_changedToRegistered_whenDomainIsCreatedAndStillReserved() {
persistBsaLabel("label");
createReservedList("reservedList", "label", RESERVED_FOR_SPECIFIC_USE);
addReservedListsToTld("tld", ImmutableList.of("reservedList"));
tm().transact(() -> tm().insert(BsaUnblockableDomain.of("label.tld", Reason.RESERVED)));
persistResource(newDomain("label.tld"));
assertThat(bsaTransact(refresher::refreshStaleUnblockables))
assertThat(refresher.refreshStaleUnblockables())
.containsExactly(
UnblockableDomainChange.ofChanged(
UnblockableDomain.of("label.tld", UnblockableDomain.Reason.RESERVED),
UnblockableDomain.Reason.REGISTERED));
}
private void setReservedList(String label) {
ImmutableMap<String, ReservedListEntry> reservedNameMap =
ImmutableMap.of(label, ReservedListEntry.create(label, RESERVED_FOR_SPECIFIC_USE, ""));
ReservedListDao.save(
new ReservedList.Builder()
.setName("testlist")
.setCreationTimestamp(fakeClock.nowUtc())
.setShouldPublish(false)
.setReservedListMap(reservedNameMap)
.build());
persistResource(
Tld.get("tld").asBuilder().setReservedListsByName(ImmutableSet.of("testlist")).build());
}
}

View File

@@ -16,6 +16,8 @@ package google.registry.bsa.persistence;
import static com.google.common.truth.Truth.assertThat;
import static com.google.common.truth.Truth8.assertThat;
import static google.registry.bsa.ReservedDomainsTestingUtils.addReservedListsToTld;
import static google.registry.bsa.ReservedDomainsTestingUtils.createReservedList;
import static google.registry.bsa.persistence.LabelDiffUpdates.applyLabelDiff;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.testing.DatabaseHelper.createTld;
@@ -26,7 +28,6 @@ import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.when;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Sets;
import google.registry.bsa.IdnChecker;
@@ -36,9 +37,6 @@ import google.registry.bsa.api.UnblockableDomain;
import google.registry.bsa.persistence.BsaUnblockableDomain.Reason;
import google.registry.model.tld.Tld;
import google.registry.model.tld.label.ReservationType;
import google.registry.model.tld.label.ReservedList;
import google.registry.model.tld.label.ReservedList.ReservedListEntry;
import google.registry.model.tld.label.ReservedListDao;
import google.registry.persistence.transaction.JpaTestExtensions;
import google.registry.persistence.transaction.JpaTestExtensions.JpaIntegrationWithCoverageExtension;
import google.registry.testing.FakeClock;
@@ -139,18 +137,8 @@ class LabelDiffUpdatesTest {
@Test
void applyLabelDiffs_newLabel() {
persistActiveDomain("label.app");
ReservedListDao.save(
new ReservedList.Builder()
.setReservedListMap(
ImmutableMap.of(
"label",
ReservedListEntry.create(
"label", ReservationType.RESERVED_FOR_SPECIFIC_USE, null)))
.setName("page_reserved")
.setCreationTimestamp(fakeClock.nowUtc())
.build());
ReservedList reservedList = ReservedList.get("page_reserved").get();
tm().transact(() -> tm().put(page.asBuilder().setReservedLists(reservedList).build()));
createReservedList("page_reserved", "label", ReservationType.RESERVED_FOR_SPECIFIC_USE);
addReservedListsToTld("page", ImmutableList.of("page_reserved"));
when(idnChecker.getForbiddingTlds(any()))
.thenReturn(Sets.difference(ImmutableSet.of(dev), ImmutableSet.of()).immutableCopy());

View File

@@ -16,10 +16,11 @@ package google.registry.bsa.persistence;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static com.google.common.truth.Truth.assertThat;
import static google.registry.bsa.BsaTransactions.bsaQuery;
import static google.registry.bsa.persistence.Queries.deleteBsaLabelByLabels;
import static google.registry.bsa.persistence.Queries.queryBsaLabelByLabels;
import static google.registry.bsa.persistence.Queries.queryBsaUnblockableDomainByLabels;
import static google.registry.bsa.persistence.Queries.queryLivesDomains;
import static google.registry.bsa.persistence.Queries.queryNewlyCreatedDomains;
import static google.registry.bsa.persistence.Queries.queryUnblockablesByNames;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.testing.DatabaseHelper.createTlds;
@@ -182,7 +183,7 @@ class QueriesTest {
}
@Test
void queryLivesDomains_onlyLiveDomainsReturned() {
void queryNewlyCreatedDomains_onlyLiveDomainsReturned() {
DateTime testStartTime = fakeClock.nowUtc();
createTlds("tld");
persistNewRegistrar("TheRegistrar");
@@ -199,7 +200,55 @@ class QueriesTest {
newDomain("d2.tld").asBuilder().setCreationTimeForTest(fakeClock.nowUtc()).build());
fakeClock.advanceOneMilli();
// Now is time 2
assertThat(tm().transact(() -> queryLivesDomains(testStartTime, fakeClock.nowUtc())))
assertThat(
bsaQuery(
() ->
queryNewlyCreatedDomains(
ImmutableList.of("tld"), testStartTime, fakeClock.nowUtc())))
.containsExactly("d1.tld", "d2.tld");
}
@Test
void queryNewlyCreatedDomains_onlyDomainsAfterMinCreationTimeReturned() {
DateTime testStartTime = fakeClock.nowUtc();
createTlds("tld");
persistNewRegistrar("TheRegistrar");
// time 0:
persistResource(
newDomain("d1.tld").asBuilder().setCreationTimeForTest(fakeClock.nowUtc()).build());
// time 0, deletion time 1
persistDomainAsDeleted(
newDomain("will-delete.tld").asBuilder().setCreationTimeForTest(fakeClock.nowUtc()).build(),
fakeClock.nowUtc().plusMillis(1));
fakeClock.advanceOneMilli();
// time 1
persistResource(
newDomain("d2.tld").asBuilder().setCreationTimeForTest(fakeClock.nowUtc()).build());
fakeClock.advanceOneMilli();
// Now is time 2, ask for domains created since time 1
assertThat(
bsaQuery(
() ->
queryNewlyCreatedDomains(
ImmutableList.of("tld"), testStartTime.plusMillis(1), fakeClock.nowUtc())))
.containsExactly("d2.tld");
}
@Test
void queryNewlyCreatedDomains_onlyDomainsInRequestedTldsReturned() {
DateTime testStartTime = fakeClock.nowUtc();
createTlds("tld", "tld2");
persistNewRegistrar("TheRegistrar");
persistResource(
newDomain("d1.tld").asBuilder().setCreationTimeForTest(fakeClock.nowUtc()).build());
persistResource(
newDomain("d2.tld2").asBuilder().setCreationTimeForTest(fakeClock.nowUtc()).build());
fakeClock.advanceOneMilli();
assertThat(
bsaQuery(
() ->
queryNewlyCreatedDomains(
ImmutableList.of("tld"), testStartTime, fakeClock.nowUtc())))
.containsExactly("d1.tld");
}
}

View File

@@ -288,7 +288,7 @@ class CheckApiActionTest {
@Test
void testSuccess_blockedByBsa() {
BsaTestingUtils.persistBsaLabel("rich", START_OF_TIME);
BsaTestingUtils.persistBsaLabel("rich");
persistResource(
Tld.get("example").asBuilder().setBsaEnrollStartTime(Optional.of(START_OF_TIME)).build());
assertThat(getCheckResponse("rich.example"))

View File

@@ -14,6 +14,7 @@
package google.registry.flows.domain;
import static google.registry.bsa.persistence.BsaTestingUtils.persistBsaLabel;
import static google.registry.model.billing.BillingBase.RenewalPriceBehavior.DEFAULT;
import static google.registry.model.billing.BillingBase.RenewalPriceBehavior.NONPREMIUM;
import static google.registry.model.billing.BillingBase.RenewalPriceBehavior.SPECIFIED;
@@ -44,7 +45,6 @@ import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedMap;
import com.google.common.collect.Ordering;
import google.registry.bsa.persistence.BsaTestingUtils;
import google.registry.flows.EppException;
import google.registry.flows.FlowUtils.NotLoggedInException;
import google.registry.flows.FlowUtils.UnknownCurrencyEppException;
@@ -160,7 +160,7 @@ class DomainCheckFlowTest extends ResourceCheckFlowTestCase<DomainCheckFlow, Dom
@Test
void testSuccess_bsaBlocked_otherwiseAvailable_blocked() throws Exception {
BsaTestingUtils.persistBsaLabel("example1", clock.nowUtc());
persistBsaLabel("example1");
doCheckTest(
create(false, "example1.tld", "Blocked by a GlobalBlock service"),
create(true, "example2.tld", null),
@@ -169,7 +169,7 @@ class DomainCheckFlowTest extends ResourceCheckFlowTestCase<DomainCheckFlow, Dom
@Test
void testSuccess_bsaBlocked_alsoRegistered_registered() throws Exception {
BsaTestingUtils.persistBsaLabel("example1", clock.nowUtc());
persistBsaLabel("example1");
persistActiveDomain("example1.tld");
doCheckTest(
create(false, "example1.tld", "In use"),
@@ -179,8 +179,8 @@ class DomainCheckFlowTest extends ResourceCheckFlowTestCase<DomainCheckFlow, Dom
@Test
void testSuccess_bsaBlocked_alsoReserved_reserved() throws Exception {
BsaTestingUtils.persistBsaLabel("reserved", clock.nowUtc());
BsaTestingUtils.persistBsaLabel("allowedinsunrise", clock.nowUtc());
persistBsaLabel("reserved");
persistBsaLabel("allowedinsunrise");
setEppInput("domain_check_one_tld_reserved.xml");
doCheckTest(
create(false, "reserved.tld", "Reserved"),

View File

@@ -2575,7 +2575,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
@Test
void testSuccess_bsaLabelMatch_notEnrolled() throws Exception {
persistResource(Tld.get("tld").asBuilder().setBsaEnrollStartTime(Optional.empty()).build());
persistBsaLabel("example", clock.nowUtc());
persistBsaLabel("example");
persistContactsAndHosts();
doSuccessfulTest();
}
@@ -2587,7 +2587,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
.asBuilder()
.setBsaEnrollStartTime(Optional.of(clock.nowUtc().plusSeconds(1)))
.build());
persistBsaLabel("example", clock.nowUtc());
persistBsaLabel("example");
persistContactsAndHosts();
doSuccessfulTest();
}
@@ -2599,7 +2599,7 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
.asBuilder()
.setBsaEnrollStartTime(Optional.of(clock.nowUtc().minusSeconds(1)))
.build());
persistBsaLabel("example", clock.nowUtc());
persistBsaLabel("example");
persistContactsAndHosts();
EppException thrown = assertThrows(DomainLabelBlockedByBsaException.class, this::runFlow);
assertAboutEppExceptions()

View File

@@ -506,7 +506,7 @@ public class ConfigureTldCommandTest extends CommandTestCase<ConfigureTldCommand
}
@Test
void testFailure_breakglassFlag_NoChanges() throws Exception {
void testFailure_breakGlassFlag_NoChanges() throws Exception {
Tld tld = createTld("idns");
persistResource(
tld.asBuilder()
@@ -517,19 +517,20 @@ public class ConfigureTldCommandTest extends CommandTestCase<ConfigureTldCommand
Files.asCharSink(tldFile, UTF_8).write(loadFile(getClass(), "idns.yaml"));
IllegalArgumentException thrown =
assertThrows(
IllegalArgumentException.class, () -> runCommandForced("--input=" + tldFile, "-b"));
IllegalArgumentException.class,
() -> runCommandForced("--input=" + tldFile, "-b=true"));
assertThat(thrown.getMessage())
.isEqualTo(
"Breakglass mode can only be set when making new changes to a TLD configuration");
"Break glass mode can only be set when making new changes to a TLD configuration");
}
@Test
void testSuccess_breakglassFlag_startsBreakglassMode() throws Exception {
void testSuccess_breakGlassFlag_startsBreakGlassMode() throws Exception {
Tld tld = createTld("tld");
assertThat(tld.getCreateBillingCost()).isEqualTo(Money.of(USD, 13));
File tldFile = tmpDir.resolve("tld.yaml").toFile();
Files.asCharSink(tldFile, UTF_8).write(loadFile(getClass(), "tld.yaml"));
runCommandForced("--input=" + tldFile, "--breakglass");
runCommandForced("--input=" + tldFile, "--break_glass=true");
Tld updatedTld = Tld.get("tld");
assertThat(updatedTld.getCreateBillingCost()).isEqualTo(Money.of(USD, 25));
testTldConfiguredSuccessfully(updatedTld, "tld.yaml");
@@ -537,13 +538,13 @@ public class ConfigureTldCommandTest extends CommandTestCase<ConfigureTldCommand
}
@Test
void testSuccess_breakglassFlag_continuesBreakglassMode() throws Exception {
void testSuccess_breakGlassFlag_continuesBreakGlassMode() throws Exception {
Tld tld = createTld("tld");
assertThat(tld.getCreateBillingCost()).isEqualTo(Money.of(USD, 13));
persistResource(tld.asBuilder().setBreakglassMode(true).build());
File tldFile = tmpDir.resolve("tld.yaml").toFile();
Files.asCharSink(tldFile, UTF_8).write(loadFile(getClass(), "tld.yaml"));
runCommandForced("--input=" + tldFile, "--breakglass");
runCommandForced("--input=" + tldFile, "--break_glass=true");
Tld updatedTld = Tld.get("tld");
assertThat(updatedTld.getCreateBillingCost()).isEqualTo(Money.of(USD, 25));
testTldConfiguredSuccessfully(updatedTld, "tld.yaml");
@@ -551,7 +552,7 @@ public class ConfigureTldCommandTest extends CommandTestCase<ConfigureTldCommand
}
@Test
void testSuccess_NoDiffNoBreakglassFlag_endsBreakglassMode() throws Exception {
void testSuccess_NoDiffNoBreakGlassFlag_endsBreakGlassMode() throws Exception {
Tld tld = createTld("idns");
persistResource(
tld.asBuilder()
@@ -566,11 +567,11 @@ public class ConfigureTldCommandTest extends CommandTestCase<ConfigureTldCommand
assertThat(updatedTld.getBreakglassMode()).isFalse();
assertAboutLogs()
.that(logHandler)
.hasLogAtLevelWithMessage(INFO, "Breakglass mode removed from TLD: idns");
.hasLogAtLevelWithMessage(INFO, "Break glass mode removed from TLD: idns");
}
@Test
void testSuccess_noDiffBreakglassFlag_continuesBreakglassMode() throws Exception {
void testSuccess_noDiffBreakGlassFlag_continuesBreakGlassMode() throws Exception {
Tld tld = createTld("idns");
persistResource(
tld.asBuilder()
@@ -580,7 +581,7 @@ public class ConfigureTldCommandTest extends CommandTestCase<ConfigureTldCommand
.build());
File tldFile = tmpDir.resolve("idns.yaml").toFile();
Files.asCharSink(tldFile, UTF_8).write(loadFile(getClass(), "idns.yaml"));
runCommandForced("--input=" + tldFile, "-b");
runCommandForced("--input=" + tldFile, "-b=true");
Tld updatedTld = Tld.get("idns");
assertThat(updatedTld.getBreakglassMode()).isTrue();
assertAboutLogs()
@@ -589,7 +590,7 @@ public class ConfigureTldCommandTest extends CommandTestCase<ConfigureTldCommand
}
@Test
void testFailure_noBreakglassFlag_inBreakglassMode() throws Exception {
void testFailure_noBreakGlassFlag_inBreakGlassMode() throws Exception {
Tld tld = createTld("tld");
persistResource(tld.asBuilder().setBreakglassMode(true).build());
File tldFile = tmpDir.resolve("tld.yaml").toFile();
@@ -598,10 +599,60 @@ public class ConfigureTldCommandTest extends CommandTestCase<ConfigureTldCommand
assertThrows(IllegalArgumentException.class, () -> runCommandForced("--input=" + tldFile));
assertThat(thrown.getMessage())
.isEqualTo(
"Changes can not be applied since TLD is in breakglass mode but the breakglass flag"
"Changes can not be applied since TLD is in break glass mode but the --break_glass flag"
+ " was not used");
}
@Test
void testFailure_breakGlassFlagFalse_notInBreakGlass() throws Exception {
createTld("tld");
File tldFile = tmpDir.resolve("tld.yaml").toFile();
Files.asCharSink(tldFile, UTF_8).write(loadFile(getClass(), "tld.yaml"));
IllegalArgumentException thrown =
assertThrows(
IllegalArgumentException.class,
() -> runCommandForced("--input=" + tldFile, "--break_glass=false"));
assertThat(thrown.getMessage())
.isEqualTo(
"The --break_glass flag cannot be set to false to end break glass mode because the TLD"
+ " is not currently in break glass mode");
}
@Test
void testSuccess_breakGlassFlagFalse_endsBreakGlassMode() throws Exception {
Tld tld = createTld("tld");
assertThat(tld.getCreateBillingCost()).isEqualTo(Money.of(USD, 13));
persistResource(tld.asBuilder().setBreakglassMode(true).build());
File tldFile = tmpDir.resolve("tld.yaml").toFile();
Files.asCharSink(tldFile, UTF_8).write(loadFile(getClass(), "tld.yaml"));
runCommandForced("--break_glass=false", "--input=" + tldFile);
Tld updatedTld = Tld.get("tld");
assertThat(updatedTld.getCreateBillingCost()).isEqualTo(Money.of(USD, 25));
testTldConfiguredSuccessfully(updatedTld, "tld.yaml");
assertThat(updatedTld.getBreakglassMode()).isFalse();
}
@Test
void testSuccess_noDiffBreakGlassFlagFalse_endsBreakGlassMode() throws Exception {
Tld tld = createTld("idns");
persistResource(
tld.asBuilder()
.setIdnTables(ImmutableSet.of(JA, UNCONFUSABLE_LATIN, EXTENDED_LATIN))
.setAllowedFullyQualifiedHostNames(ImmutableSet.of("beta", "zeta", "alpha", "gamma"))
.setBreakglassMode(true)
.build());
File tldFile = tmpDir.resolve("idns.yaml").toFile();
Files.asCharSink(tldFile, UTF_8).write(loadFile(getClass(), "idns.yaml"));
// This update contains no diffs from whats in the config file
runCommandForced("--input=" + tldFile, "--break_glass=false");
Tld updatedTld = Tld.get("idns");
assertThat(updatedTld.getBreakglassMode()).isFalse();
testTldConfiguredSuccessfully(updatedTld, "idns.yaml");
assertAboutLogs()
.that(logHandler)
.hasLogAtLevelWithMessage(INFO, "Break glass mode removed from TLD: idns");
}
@Test
void testSuccess_dryRunOnCreate_noChanges() throws Exception {
File tldFile = tmpDir.resolve("tld.yaml").toFile();

View File

@@ -1,733 +0,0 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.tools;
import static com.google.common.truth.Truth.assertThat;
import static com.google.common.truth.Truth8.assertThat;
import static google.registry.model.domain.token.AllocationToken.TokenType.DEFAULT_PROMO;
import static google.registry.model.tld.Tld.TldState.GENERAL_AVAILABILITY;
import static google.registry.model.tld.Tld.TldState.PREDELEGATION;
import static google.registry.testing.DatabaseHelper.createTld;
import static google.registry.testing.DatabaseHelper.persistPremiumList;
import static google.registry.testing.DatabaseHelper.persistReservedList;
import static google.registry.testing.DatabaseHelper.persistResource;
import static google.registry.util.DateTimeUtils.START_OF_TIME;
import static java.math.BigDecimal.ROUND_UNNECESSARY;
import static org.joda.money.CurrencyUnit.JPY;
import static org.joda.money.CurrencyUnit.USD;
import static org.joda.time.DateTimeZone.UTC;
import static org.joda.time.Duration.standardMinutes;
import static org.junit.jupiter.api.Assertions.assertThrows;
import com.beust.jcommander.ParameterException;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Range;
import google.registry.model.domain.token.AllocationToken;
import google.registry.model.tld.Tld;
import google.registry.tldconfig.idn.IdnTableEnum;
import java.math.BigDecimal;
import org.joda.money.Money;
import org.joda.time.DateTime;
import org.joda.time.Duration;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
/** Unit tests for {@link CreateTldCommand}. */
class CreateTldCommandTest extends CommandTestCase<CreateTldCommand> {
@BeforeEach
void beforeEach() {
persistReservedList("common_abuse", "baa,FULLY_BLOCKED");
persistReservedList("xn--q9jyb4c_abuse", "lamb,FULLY_BLOCKED");
persistReservedList("tld_banned", "kilo,FULLY_BLOCKED", "lima,FULLY_BLOCKED");
persistReservedList("soy_expurgated", "fireflies,FULLY_BLOCKED");
persistPremiumList("xn--q9jyb4c", USD, "minecraft,USD 1000");
command.validDnsWriterNames = ImmutableSet.of("VoidDnsWriter", "FooDnsWriter");
}
@Test
void testSuccess() throws Exception {
DateTime before = fakeClock.nowUtc();
runCommandForced("xn--q9jyb4c", "--roid_suffix=Q9JYB4C", "--dns_writers=FooDnsWriter");
DateTime after = fakeClock.nowUtc();
Tld registry = Tld.get("xn--q9jyb4c");
assertThat(registry).isNotNull();
assertThat(registry.getAddGracePeriodLength()).isEqualTo(Tld.DEFAULT_ADD_GRACE_PERIOD);
assertThat(registry.getCreationTime()).isIn(Range.closed(before, after));
assertThat(registry.getDnsWriters()).containsExactly("FooDnsWriter");
assertThat(registry.getTldState(registry.getCreationTime())).isEqualTo(PREDELEGATION);
assertThat(registry.getRedemptionGracePeriodLength())
.isEqualTo(Tld.DEFAULT_REDEMPTION_GRACE_PERIOD);
assertThat(registry.getPendingDeleteLength()).isEqualTo(Tld.DEFAULT_PENDING_DELETE_LENGTH);
assertThat(registry.getRegistryLockOrUnlockBillingCost())
.isEqualTo(Tld.DEFAULT_REGISTRY_LOCK_OR_UNLOCK_BILLING_COST);
}
@Test
void testSuccess_ttls() throws Exception {
runCommandForced(
"xn--q9jyb4c",
"--roid_suffix=Q9JYB4C",
"--dns_writers=FooDnsWriter",
"--dns_a_plus_aaaa_ttl=PT300S",
"--dns_ds_ttl=PT240S",
"--dns_ns_ttl=PT180S");
Tld registry = Tld.get("xn--q9jyb4c");
assertThat(registry).isNotNull();
assertThat(registry.getDnsAPlusAaaaTtl().get()).isEqualTo(standardMinutes(5));
assertThat(registry.getDnsDsTtl().get()).isEqualTo(standardMinutes(4));
assertThat(registry.getDnsNsTtl().get()).isEqualTo(standardMinutes(3));
}
@Test
void testFailure_multipleArguments() {
IllegalArgumentException thrown =
assertThrows(
IllegalArgumentException.class,
() ->
runCommandForced(
"--roid_suffix=BLAH", "--dns_writers=VoidDnsWriter", "xn--q9jyb4c", "test"));
assertThat(thrown).hasMessageThat().contains("Can't create more than one TLD at a time");
}
@Test
void testFailure_multipleDuplicateArguments() {
IllegalArgumentException thrown =
assertThrows(
IllegalArgumentException.class,
() ->
runCommandForced(
"--roid_suffix=BLAH", "--dns_writers=VoidDnsWriter", "test", "test"));
assertThat(thrown).hasMessageThat().contains("Can't create more than one TLD at a time");
}
@Test
void testSuccess_initialTldStateFlag() throws Exception {
runCommandForced(
"--initial_tld_state=GENERAL_AVAILABILITY",
"--roid_suffix=Q9JYB4C",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c");
assertThat(Tld.get("xn--q9jyb4c").getTldState(DateTime.now(UTC)))
.isEqualTo(GENERAL_AVAILABILITY);
}
@Test
void testSuccess_initialRenewBillingCostFlag() throws Exception {
runCommandForced(
"--initial_renew_billing_cost=\"USD 42.42\"",
"--roid_suffix=Q9JYB4C",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c");
assertThat(Tld.get("xn--q9jyb4c").getStandardRenewCost(DateTime.now(UTC)))
.isEqualTo(Money.of(USD, 42.42));
}
@Test
void testSuccess_eapFeeSchedule() throws Exception {
DateTime now = DateTime.now(UTC);
DateTime tomorrow = now.plusDays(1);
runCommandForced(
String.format(
"--eap_fee_schedule=\"%s=USD 0.00,%s=USD 50.00,%s=USD 10.00\"",
START_OF_TIME, now, tomorrow),
"--roid_suffix=Q9JYB4C",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c");
Tld registry = Tld.get("xn--q9jyb4c");
assertThat(registry.getEapFeeFor(now.minusHours(1)).getCost())
.isEqualTo(BigDecimal.ZERO.setScale(2, ROUND_UNNECESSARY));
assertThat(registry.getEapFeeFor(now.plusHours(1)).getCost())
.isEqualTo(new BigDecimal("50.00"));
assertThat(registry.getEapFeeFor(now.plusDays(1).plusHours(1)).getCost())
.isEqualTo(new BigDecimal("10.00"));
}
@Test
void testSuccess_addGracePeriodFlag() throws Exception {
runCommandForced(
"--add_grace_period=PT300S",
"--roid_suffix=Q9JYB4C",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c");
assertThat(Tld.get("xn--q9jyb4c").getAddGracePeriodLength()).isEqualTo(standardMinutes(5));
}
@Test
void testSuccess_roidSuffixWorks() throws Exception {
runCommandForced("--roid_suffix=RSUFFIX", "--dns_writers=VoidDnsWriter", "tld");
assertThat(Tld.get("tld").getRoidSuffix()).isEqualTo("RSUFFIX");
}
@Test
void testSuccess_escrow() throws Exception {
runCommandForced(
"--escrow=true", "--roid_suffix=Q9JYB4C", "--dns_writers=VoidDnsWriter", "xn--q9jyb4c");
assertThat(Tld.get("xn--q9jyb4c").getEscrowEnabled()).isTrue();
}
@Test
void testSuccess_noEscrow() throws Exception {
runCommandForced(
"--escrow=false", "--roid_suffix=Q9JYB4C", "--dns_writers=VoidDnsWriter", "xn--q9jyb4c");
assertThat(Tld.get("xn--q9jyb4c").getEscrowEnabled()).isFalse();
}
@Test
void testSuccess_redemptionGracePeriodFlag() throws Exception {
runCommandForced(
"--redemption_grace_period=PT300S",
"--roid_suffix=Q9JYB4C",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c");
assertThat(Tld.get("xn--q9jyb4c").getRedemptionGracePeriodLength())
.isEqualTo(standardMinutes(5));
}
@Test
void testSuccess_pendingDeleteLengthFlag() throws Exception {
runCommandForced(
"--pending_delete_length=PT300S",
"--roid_suffix=Q9JYB4C",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c");
assertThat(Tld.get("xn--q9jyb4c").getPendingDeleteLength()).isEqualTo(standardMinutes(5));
}
@Test
void testSuccess_automaticTransferLengthFlag() throws Exception {
runCommandForced(
"--automatic_transfer_length=PT300S",
"--roid_suffix=Q9JYB4C",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c");
assertThat(Tld.get("xn--q9jyb4c").getAutomaticTransferLength()).isEqualTo(standardMinutes(5));
}
@Test
void testSuccess_createBillingCostFlag() throws Exception {
runCommandForced(
"--create_billing_cost=\"USD 42.42\"",
"--roid_suffix=Q9JYB4C",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c");
assertThat(Tld.get("xn--q9jyb4c").getCreateBillingCost()).isEqualTo(Money.of(USD, 42.42));
}
@Test
void testSuccess_restoreBillingCostFlag() throws Exception {
runCommandForced(
"--restore_billing_cost=\"USD 42.42\"",
"--roid_suffix=Q9JYB4C",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c");
assertThat(Tld.get("xn--q9jyb4c").getRestoreBillingCost()).isEqualTo(Money.of(USD, 42.42));
}
@Test
void testSuccess_serverStatusChangeCostFlag() throws Exception {
runCommandForced(
"--server_status_change_cost=\"USD 42.42\"",
"--roid_suffix=Q9JYB4C",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c");
assertThat(Tld.get("xn--q9jyb4c").getServerStatusChangeBillingCost())
.isEqualTo(Money.of(USD, 42.42));
}
@Test
void testSuccess_registryLockOrUnlockCostFlag() throws Exception {
runCommandForced(
"--registry_lock_or_unlock_cost=\"USD 42.42\"",
"--roid_suffix=Q9JYB4C",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c");
assertThat(Tld.get("xn--q9jyb4c").getRegistryLockOrUnlockBillingCost())
.isEqualTo(Money.of(USD, 42.42));
}
@Test
void testSuccess_nonUsdBillingCostFlag() throws Exception {
runCommandForced(
"--create_billing_cost=\"JPY 12345\"",
"--restore_billing_cost=\"JPY 67890\"",
"--initial_renew_billing_cost=\"JPY 101112\"",
"--server_status_change_cost=\"JPY 97865\"",
"--registry_lock_or_unlock_cost=\"JPY 9001\"",
"--roid_suffix=Q9JYB4C",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c");
Tld registry = Tld.get("xn--q9jyb4c");
assertThat(registry.getCreateBillingCost()).isEqualTo(Money.ofMajor(JPY, 12345));
assertThat(registry.getRestoreBillingCost()).isEqualTo(Money.ofMajor(JPY, 67890));
assertThat(registry.getStandardRenewCost(START_OF_TIME)).isEqualTo(Money.ofMajor(JPY, 101112));
}
@Test
void testSuccess_multipartTld() throws Exception {
runCommandForced("co.uk", "--roid_suffix=COUK", "--dns_writers=VoidDnsWriter");
Tld registry = Tld.get("co.uk");
assertThat(registry.getTldState(DateTime.now(UTC))).isEqualTo(PREDELEGATION);
assertThat(registry.getAddGracePeriodLength()).isEqualTo(Tld.DEFAULT_ADD_GRACE_PERIOD);
assertThat(registry.getRedemptionGracePeriodLength())
.isEqualTo(Tld.DEFAULT_REDEMPTION_GRACE_PERIOD);
assertThat(registry.getPendingDeleteLength()).isEqualTo(Tld.DEFAULT_PENDING_DELETE_LENGTH);
}
@Test
void testSuccess_setReservedLists() throws Exception {
runCommandForced(
"--reserved_lists=xn--q9jyb4c_abuse,common_abuse",
"--roid_suffix=Q9JYB4C",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c");
assertThat(Tld.get("xn--q9jyb4c").getReservedListNames())
.containsExactly("xn--q9jyb4c_abuse", "common_abuse");
}
@Test
void testFailure_invalidAddGracePeriod() {
IllegalArgumentException thrown =
assertThrows(
IllegalArgumentException.class,
() ->
runCommandForced(
"--add_grace_period=5m",
"--roid_suffix=Q9JYB4C",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c"));
assertThat(thrown).hasMessageThat().contains("Invalid format: \"5m\"");
}
@Test
void testFailure_invalidRedemptionGracePeriod() {
IllegalArgumentException thrown =
assertThrows(
IllegalArgumentException.class,
() ->
runCommandForced(
"--redemption_grace_period=5m",
"--roid_suffix=Q9JYB4C",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c"));
assertThat(thrown).hasMessageThat().contains("Invalid format: \"5m\"");
}
@Test
void testFailure_invalidPendingDeleteLength() {
IllegalArgumentException thrown =
assertThrows(
IllegalArgumentException.class,
() ->
runCommandForced(
"--pending_delete_length=5m",
"--roid_suffix=Q9JYB4C",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c"));
assertThat(thrown).hasMessageThat().contains("Invalid format: \"5m\"");
}
@Test
void testFailure_invalidTldState() {
ParameterException thrown =
assertThrows(
ParameterException.class,
() ->
runCommandForced(
"--initial_tld_state=INVALID_STATE",
"--roid_suffix=Q9JYB4C",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c"));
assertThat(thrown).hasMessageThat().contains("Invalid value for --initial_tld_state parameter");
}
@Test
void testFailure_bothTldStateFlags() {
DateTime now = DateTime.now(UTC);
IllegalArgumentException thrown =
assertThrows(
IllegalArgumentException.class,
() ->
runCommandForced(
String.format(
"--tld_state_transitions=%s=PREDELEGATION,%s=START_DATE_SUNRISE",
now, now.plus(Duration.millis(1))),
"--initial_tld_state=GENERAL_AVAILABILITY",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c"));
assertThat(thrown)
.hasMessageThat()
.contains("Don't pass both --initial_tld_state and --tld_state_transitions");
}
@Test
void testFailure_negativeInitialRenewBillingCost() {
IllegalArgumentException thrown =
assertThrows(
IllegalArgumentException.class,
() ->
runCommandForced(
"--initial_renew_billing_cost=USD -42",
"--roid_suffix=Q9JYB4C",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c"));
assertThat(thrown).hasMessageThat().contains("Renew billing cost cannot be negative");
}
@Test
void testFailure_invalidEapCurrency() {
IllegalArgumentException thrown =
assertThrows(
IllegalArgumentException.class,
() ->
runCommandForced(
String.format("--eap_fee_schedule=\"%s=JPY 123456\"", START_OF_TIME),
"--roid_suffix=Q9JYB4C",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c"));
assertThat(thrown).hasMessageThat().contains("All EAP fees must be in the TLD's currency");
}
@Test
void testFailure_noTldName() {
ParameterException thrown = assertThrows(ParameterException.class, this::runCommandForced);
assertThat(thrown)
.hasMessageThat()
.contains("Main parameters are required (\"Names of the TLDs\")");
}
@Test
void testFailure_noDnsWriter() {
IllegalArgumentException thrown =
assertThrows(
IllegalArgumentException.class,
() -> runCommandForced("xn--q9jyb4c", "--roid_suffix=Q9JYB4C"));
assertThat(thrown).hasMessageThat().contains("At least one DNS writer must be specified");
}
@Test
void testFailure_alreadyExists() {
createTld("xn--q9jyb4c");
IllegalStateException thrown =
assertThrows(
IllegalStateException.class,
() ->
runCommandForced(
"--roid_suffix=NOTDUPE", "--dns_writers=VoidDnsWriter", "xn--q9jyb4c"));
assertThat(thrown).hasMessageThat().contains("TLD 'xn--q9jyb4c' already exists");
}
@Test
void testFailure_tldStartsWithDigit() {
IllegalArgumentException thrown =
assertThrows(
IllegalArgumentException.class,
() -> runCommandForced("1foo", "--roid_suffix=1FOO", "--dns_writers=VoidDnsWriter"));
assertThat(thrown).hasMessageThat().contains("TLDs cannot begin with a number");
}
@Test
void testSuccess_setAllowedRegistrants() throws Exception {
runCommandForced(
"--allowed_registrants=alice,bob",
"--roid_suffix=Q9JYB4C",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c");
assertThat(Tld.get("xn--q9jyb4c").getAllowedRegistrantContactIds())
.containsExactly("alice", "bob");
}
@Test
void testSuccess_emptyAllowedRegistrants() throws Exception {
runCommandForced(
"--allowed_registrants=",
"--roid_suffix=Q9JYB4C",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c");
assertThat(Tld.get("xn--q9jyb4c").getAllowedRegistrantContactIds()).isEmpty();
}
@Test
void testSuccess_setAllowedNameservers() throws Exception {
runCommandForced(
"--allowed_nameservers=ns1.example.com,ns2.example.com",
"--roid_suffix=Q9JYB4C",
"--dns_writers=FooDnsWriter",
"xn--q9jyb4c");
assertThat(Tld.get("xn--q9jyb4c").getAllowedFullyQualifiedHostNames())
.containsExactly("ns1.example.com", "ns2.example.com");
}
@Test
void testSuccess_emptyAllowedNameservers() throws Exception {
runCommandForced(
"--allowed_nameservers=",
"--roid_suffix=Q9JYB4C",
"--dns_writers=FooDnsWriter",
"xn--q9jyb4c");
assertThat(Tld.get("xn--q9jyb4c").getAllowedFullyQualifiedHostNames()).isEmpty();
}
@Test
void testSuccess_setCommonReservedListOnTld() throws Exception {
runSuccessfulReservedListsTest("common_abuse");
}
@Test
void testSuccess_setTldSpecificReservedListOnTld() throws Exception {
runSuccessfulReservedListsTest("xn--q9jyb4c_abuse");
}
@Test
void testSuccess_setCommonReservedListAndTldSpecificReservedListOnTld() throws Exception {
runSuccessfulReservedListsTest("common_abuse,xn--q9jyb4c_abuse");
}
@Test
void testFailure_setReservedListFromOtherTld() {
runFailureReservedListsTest(
"tld_banned",
IllegalArgumentException.class,
"The reserved list(s) tld_banned cannot be applied to the tld xn--q9jyb4c");
}
@Test
void testSuccess_setReservedListFromOtherTld_withOverride() throws Exception {
runReservedListsTestOverride("tld_banned");
}
@Test
void testFailure_setCommonAndReservedListFromOtherTld() {
runFailureReservedListsTest(
"common_abuse,tld_banned",
IllegalArgumentException.class,
"The reserved list(s) tld_banned cannot be applied to the tld xn--q9jyb4c");
}
@Test
void testSuccess_setCommonAndReservedListFromOtherTld_withOverride() throws Exception {
command.errorPrintStream = System.err;
runReservedListsTestOverride("common_abuse,tld_banned");
String errMsg =
"Error overridden: The reserved list(s) tld_banned "
+ "cannot be applied to the tld xn--q9jyb4c";
assertThat(getStderrAsString()).contains(errMsg);
}
@Test
void testFailure_setMultipleReservedListsFromOtherTld() {
runFailureReservedListsTest(
"tld_banned,soy_expurgated",
IllegalArgumentException.class,
"The reserved list(s) tld_banned, soy_expurgated cannot be applied to the tld xn--q9jyb4c");
}
@Test
void testSuccess_setMultipleReservedListsFromOtherTld_withOverride() throws Exception {
runReservedListsTestOverride("tld_banned,soy_expurgated");
}
@Test
void testFailure_setNonExistentReservedLists() {
runFailureReservedListsTest(
"xn--q9jyb4c_asdf,common_asdsdgh",
IllegalArgumentException.class,
"Could not find reserved list xn--q9jyb4c_asdf to add to the tld");
}
@Test
void testSuccess_setPremiumList() throws Exception {
runCommandForced(
"--premium_list=xn--q9jyb4c",
"--roid_suffix=Q9JYB4C",
"--dns_writers=FooDnsWriter",
"xn--q9jyb4c");
assertThat(Tld.get("xn--q9jyb4c").getPremiumListName()).hasValue("xn--q9jyb4c");
}
@Test
void testSuccess_setDriveFolderIdToValue() throws Exception {
runCommandForced(
"--drive_folder_id=madmax2030",
"--roid_suffix=Q9JYB4C",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c");
assertThat(Tld.get("xn--q9jyb4c").getDriveFolderId()).isEqualTo("madmax2030");
}
@Test
void testSuccess_setDriveFolderIdToNull() throws Exception {
runCommandForced(
"--drive_folder_id=null",
"--roid_suffix=Q9JYB4C",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c");
assertThat(Tld.get("xn--q9jyb4c").getDriveFolderId()).isNull();
}
@Test
void testSuccess_setsIdnTables() throws Exception {
runCommandForced(
"--idn_tables=extended_latin,ja",
"--roid_suffix=ASDF",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c");
assertThat(Tld.get("xn--q9jyb4c").getIdnTables())
.containsExactly(IdnTableEnum.EXTENDED_LATIN, IdnTableEnum.JA);
}
@Test
void testFailure_invalidIdnTable() throws Exception {
IllegalArgumentException thrown =
assertThrows(
IllegalArgumentException.class,
() ->
runCommandForced(
"--idn_tables=extended_latin,bad_value",
"--roid_suffix=ASDF",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c"));
assertThat(thrown)
.hasMessageThat()
.isEqualTo(
"IDN tables [EXTENDED_LATIN, BAD_VALUE] contained invalid value(s). Possible values:"
+ " [EXTENDED_LATIN, UNCONFUSABLE_LATIN, JA]");
}
@Test
void testFailure_setPremiumListThatDoesntExist() {
IllegalArgumentException thrown =
assertThrows(
IllegalArgumentException.class,
() ->
runCommandForced(
"--premium_list=phonies",
"--roid_suffix=Q9JYB4C",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c"));
assertThat(thrown).hasMessageThat().contains("The premium list 'phonies' doesn't exist");
}
@Test
void testFailure_specifiedDnsWriters_dontExist() {
IllegalArgumentException thrown =
assertThrows(
IllegalArgumentException.class,
() ->
runCommandForced(
"xn--q9jyb4c", "--roid_suffix=Q9JYB4C", "--dns_writers=Invalid,Deadbeef"));
assertThat(thrown)
.hasMessageThat()
.contains("Invalid DNS writer name(s) specified: [Invalid, Deadbeef]");
}
@Test
void testSuccess_defaultToken() throws Exception {
AllocationToken token =
persistResource(
new AllocationToken()
.asBuilder()
.setToken("abc123")
.setTokenType(DEFAULT_PROMO)
.setAllowedTlds(ImmutableSet.of("xn--q9jyb4c"))
.build());
runCommandForced(
"--default_tokens=abc123",
"--roid_suffix=Q9JYB4C",
"--dns_writers=FooDnsWriter",
"xn--q9jyb4c");
assertThat(Tld.get("xn--q9jyb4c").getDefaultPromoTokens()).containsExactly(token.createVKey());
}
@Test
void testSuccess_multipleDefaultTokens() throws Exception {
AllocationToken token =
persistResource(
new AllocationToken()
.asBuilder()
.setToken("abc123")
.setTokenType(DEFAULT_PROMO)
.setAllowedTlds(ImmutableSet.of("xn--q9jyb4c"))
.build());
AllocationToken token2 =
persistResource(
new AllocationToken()
.asBuilder()
.setToken("token")
.setTokenType(DEFAULT_PROMO)
.setAllowedTlds(ImmutableSet.of("xn--q9jyb4c"))
.build());
runCommandForced(
"--default_tokens=abc123,token",
"--roid_suffix=Q9JYB4C",
"--dns_writers=FooDnsWriter",
"xn--q9jyb4c");
assertThat(Tld.get("xn--q9jyb4c").getDefaultPromoTokens())
.containsExactly(token.createVKey(), token2.createVKey());
}
@Test
void testFailure_specifiedDefaultToken_doesntExist() {
IllegalStateException thrown =
assertThrows(
IllegalStateException.class,
() ->
runCommandForced(
"xn--q9jyb4c",
"--default_tokens=InvalidToken",
"--roid_suffix=Q9JYB4C",
"--dns_writers=FooDnsWriter"));
assertThat(thrown)
.hasMessageThat()
.contains("Tokens with keys [VKey<AllocationToken>(sql:InvalidToken)] did not exist");
}
private void runSuccessfulReservedListsTest(String reservedLists) throws Exception {
runCommandForced(
"--reserved_lists",
reservedLists,
"--roid_suffix=Q9JYB4C",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c");
}
private void runReservedListsTestOverride(String reservedLists) throws Exception {
runCommandForced(
"--override_reserved_list_rules",
"--reserved_lists",
reservedLists,
"--roid_suffix=Q9JYB4C",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c");
}
private void runFailureReservedListsTest(
String reservedLists, Class<? extends Exception> errorClass, String errorMsg) {
Exception e =
assertThrows(
errorClass,
() ->
runCommandForced(
"--reserved_lists",
reservedLists,
"--roid_suffix=Q9JYB4C",
"--dns_writers=VoidDnsWriter",
"xn--q9jyb4c"));
assertThat(e).hasMessageThat().isEqualTo(errorMsg);
}
}

View File

@@ -174,7 +174,7 @@ public class WhoisActionTest {
.asBuilder()
.setBsaEnrollStartTime(Optional.of(clock.nowUtc().minusDays(1)))
.build());
persistBsaLabel("cat", clock.nowUtc());
persistBsaLabel("cat");
Registrar registrar = persistResource(makeRegistrar("evilregistrar", "Yes Virginia", ACTIVE));
persistResource(makeDomainWithRegistrar(registrar));
@@ -191,7 +191,7 @@ public class WhoisActionTest {
.asBuilder()
.setBsaEnrollStartTime(Optional.of(clock.nowUtc().minusDays(1)))
.build());
persistBsaLabel("cat", clock.nowUtc());
persistBsaLabel("cat");
newWhoisAction("domain cat.lol\r\n").run();
assertThat(response.getStatus()).isEqualTo(200);

View File

@@ -225,7 +225,7 @@ class WhoisHttpActionTest {
.asBuilder()
.setBsaEnrollStartTime(Optional.of(clock.nowUtc().minusDays(1)))
.build());
persistBsaLabel("cat", clock.nowUtc());
persistBsaLabel("cat");
Registrar registrar = persistResource(makeRegistrar("evilregistrar", "Yes Virginia", ACTIVE));
persistResource(
@@ -256,7 +256,7 @@ class WhoisHttpActionTest {
.asBuilder()
.setBsaEnrollStartTime(Optional.of(clock.nowUtc().minusDays(1)))
.build());
persistBsaLabel("cat", clock.nowUtc());
persistBsaLabel("cat");
newWhoisHttpAction("domain cat.lol\r\n").run();
assertThat(response.getStatus()).isEqualTo(404);

View File

@@ -1,9 +1,9 @@
addGracePeriodLength: "PT432000S"
allowedFullyQualifiedHostNames:
- "beta"
- "zeta"
- "alpha"
- "beta"
- "gamma"
- "zeta"
allowedRegistrantContactIds: []
anchorTenantAddGracePeriodLength: "PT2592000S"
autoRenewGracePeriodLength: "PT3888000S"

View File

@@ -120,9 +120,9 @@
"dev": true
},
"node_modules/follow-redirects": {
"version": "1.15.2",
"resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.15.2.tgz",
"integrity": "sha512-VQLG33o04KaQ8uYi2tVNbdrWp1QWxNNea+nmIB4EVM28v0hmP17z7aG1+wAkNzVq4KeXTq3221ye5qTJP91JwA==",
"version": "1.15.4",
"resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.15.4.tgz",
"integrity": "sha512-Cr4D/5wlrb0z9dgERpUL3LrmPKVDsETIJhaCMeDfuFYcqa5bldGV6wBsAN6X/vxlXQtFBMrXdXxdL8CbDTGniw==",
"dev": true,
"funding": [
{
@@ -537,9 +537,9 @@
"dev": true
},
"follow-redirects": {
"version": "1.15.2",
"resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.15.2.tgz",
"integrity": "sha512-VQLG33o04KaQ8uYi2tVNbdrWp1QWxNNea+nmIB4EVM28v0hmP17z7aG1+wAkNzVq4KeXTq3221ye5qTJP91JwA==",
"version": "1.15.4",
"resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.15.4.tgz",
"integrity": "sha512-Cr4D/5wlrb0z9dgERpUL3LrmPKVDsETIJhaCMeDfuFYcqa5bldGV6wBsAN6X/vxlXQtFBMrXdXxdL8CbDTGniw==",
"dev": true
},
"function-bind": {

View File

@@ -22,7 +22,7 @@ apt-get upgrade -y
# Install GPG2 (in case it was not included)
apt-get install gnupg2 -y
# Install Java
apt-get install openjdk-11-jdk-headless -y
apt-get install openjdk-17-jdk-headless -y
# Install Python
apt-get install python3 -y
# As of March 2021 python3 is at v3.6. Get pip then install dataclasses
@@ -37,7 +37,7 @@ npm cache clean -f
npm install -g n
# Retrying because fails are possible for node.js intallation. See -
# https://github.com/nodejs/build/issues/1993
for i in {1..5}; do n 16.19.0 && break || sleep 15; done
for i in {1..5}; do n 20.10.0 && break || sleep 15; done
# Install gcloud
# Cribbed from https://cloud.google.com/sdk/docs/quickstart-debian-ubuntu
apt-get install lsb-release -y