mcilwain c2ee453745 Make the verify OT&E action more performant ...

As previously written, it loaded up all history entries into memory and then
processed them. This was OOMing for some registrars on sandbox who had performed
a large number of testing actions, most of them long OT&E was passed.

This commit changes the verify OT&E action to stream the history entries in
batches, ordered by when they were made, and then terminates once all tests have
passed. This prevents OOMing because only a single batch of history entries need
reside in memory at once.

This does necessitate the creation of a new composite Datastore index on
HistoryEntry, so we'll need to run the ResaveAllHistoryEntriesAction in sandbox
after this change is deployed before the new verify OT&E code will work.

Note that the "history viewer" is long dead, but that the pre-existing index
on HistoryEntries is still used for many other purposes.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=223163337

2018-12-03 19:07:36 -05:00

..

javascrap

Remove unnecessary "throws" declarations

2018-06-18 18:17:56 -04:00

params

Add GenerateSpec11Action and SafeBrowsing evaluation

2018-08-10 13:46:48 -04:00

resources

…

server

Make the verify OT&E action more performant

2018-12-03 19:07:36 -05:00

soy

Add a nomulus renew_domain command

2018-06-06 15:16:09 -04:00

AllocateDomainCommand.java

Migrate away from VoidWorks

2018-06-27 15:28:52 -04:00

AppEngineAdminApiModule.java

Add option --non_live_versions to set_num_instances command

2018-12-03 18:58:05 -05:00

AppEngineConnection.java

Add option --non_live_versions to set_num_instances command

2018-12-03 18:58:05 -05:00

AuthModule.java

…

BigqueryCommand.java

Use @DefaultCredential for Cloud API access in GAE

2018-09-20 11:19:36 -04:00

BigqueryParameters.java

Use @DefaultCredential for Cloud API access in GAE

2018-09-20 11:19:36 -04:00

BUILD

Add runtime dependencies to the tools target

2018-12-03 19:04:24 -05:00

CanonicalizeLabelsCommand.java

…

CheckDomainClaimsCommand.java

Default to admin registrar in check_domain commands

2018-09-08 00:19:53 -04:00

CheckDomainCommand.java

Default to admin registrar in check_domain commands

2018-09-08 00:19:53 -04:00

CheckSnapshotCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

Command.java

Allow AppEngineConnection to target services other than "tools"

2018-09-08 00:10:06 -04:00

CommandRunner.java

Add a "shell" pseudo-command to nomulus tool

2018-03-19 18:22:01 -04:00

CommandUtilities.java

…

CommandWithConnection.java

Refactor AppEngineConnection

2018-10-29 15:34:12 -04:00

CommandWithRemoteApi.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

ComparableEntity.java

…

CompareDbBackups.java

Remove unnecessary "throws" declarations

2018-06-18 18:17:56 -04:00

ConfirmingCommand.java

Remove unnecessary "throws" declarations

2018-06-18 18:17:56 -04:00

ConvertIdnCommand.java

Remove unnecessary "throws" declarations

2018-06-18 18:17:56 -04:00

CountDomainsCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

CreateAnchorTenantCommand.java

Refactor StringGenerator bindings

2018-10-22 19:06:35 -04:00

CreateCdnsTld.java

Use @DefaultCredential for Cloud API access in GAE

2018-09-20 11:19:36 -04:00

CreateContactCommand.java

Refactor StringGenerator bindings

2018-10-22 19:06:35 -04:00

CreateDomainCommand.java

Refactor StringGenerator bindings

2018-10-22 19:06:35 -04:00

CreateHostCommand.java

…

CreateOrUpdateDomainCommand.java

Allow square bracket expansion when specifying nameservers

2018-08-10 13:44:25 -04:00

CreateOrUpdatePremiumListCommand.java

Refactor AppEngineConnection

2018-10-29 15:34:12 -04:00

CreateOrUpdateRegistrarCommand.java

Handle Purchase Order numbers explicitly in billing pipeline

2018-09-20 11:19:36 -04:00

CreateOrUpdateReservedListCommand.java

…

CreateOrUpdateTldCommand.java

Delete all Limited Release Program (LRP) code

2018-08-20 14:28:16 -04:00

CreatePremiumListCommand.java

…

CreateRegistrarCommand.java

Refactor AppEngineConnection

2018-10-29 15:34:12 -04:00

CreateRegistrarGroupsCommand.java

Refactor AppEngineConnection

2018-10-29 15:34:12 -04:00

CreateReservedListCommand.java

…

CreateSandboxTldCommand.java

Add a command-line tool suitable for tech support usage

2018-09-20 11:19:36 -04:00

CreateTldCommand.java

Remove unnecessary "throws" declarations

2018-06-18 18:17:56 -04:00

CurlCommand.java

Refactor AppEngineConnection

2018-10-29 15:34:12 -04:00

DefaultRequestFactoryModule.java

Refactor AppEngineConnection

2018-10-29 15:34:12 -04:00

DeleteAllocationTokensCommand.java

Add nomulus command for deleting AllocationTokens

2018-10-03 11:54:10 -04:00

DeleteDomainCommand.java

Add flag to delete domain immediately in nomulus too

2018-05-30 12:18:54 -04:00

DeleteHostCommand.java

…

DeletePremiumListCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

DeleteReservedListCommand.java

Remove unnecessary "throws" declarations

2018-06-18 18:17:56 -04:00

DeleteTldCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

DeployInvoicingPipelineCommand.java

Refactor beam invoicing pipeline into its own package

2018-07-17 22:08:13 -04:00

DeploySpec11PipelineCommand.java

Correct wrong description of command

2018-09-14 11:50:43 -04:00

DomainApplicationInfoCommand.java

Don't prompt to confirm non-mutating nomulus EPP tool commands

2018-04-10 16:53:18 -04:00

EncryptEscrowDepositCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

EppToolCommand.java

Refactor AppEngineConnection

2018-10-29 15:34:12 -04:00

EscrowDepositEncryptor.java

Simplify the RyDE API

2018-07-17 22:03:53 -04:00

ExecuteEppCommand.java

Do not use InjectRule to mock stdin in ExecuteEppCommandTest

2018-10-22 18:46:36 -04:00

GenerateAllocationTokensCommand.java

Refactor StringGenerator bindings

2018-10-22 19:06:35 -04:00

GenerateApplicationsReportCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

GenerateAuctionDataCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

GenerateDnsReportCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

GenerateEscrowDepositCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

GenerateLordnCommand.java

Refactor LordnTask to LordnTaskUtils

2018-10-22 18:59:48 -04:00

GenerateZoneFilesCommand.java

Refactor AppEngineConnection

2018-10-29 15:34:12 -04:00

GetApplicationCommand.java

…

GetApplicationIdsCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

GetAppliedLabelsCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

GetClaimsListCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

GetContactCommand.java

…

GetDomainCommand.java

…

GetEppResourceCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

GetHistoryEntriesCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

GetHostCommand.java

…

GetKeyringSecretCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

GetRegistrarCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

GetResourceByKeyCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

GetRoutingMapCommand.java

…

GetSchemaCommand.java

Remove unnecessary "throws" declarations

2018-06-18 18:17:56 -04:00

GetSchemaTreeCommand.java

Remove unnecessary "throws" declarations

2018-06-18 18:17:56 -04:00

GetTldCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

GhostrydeCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

GtechTool.java

Add a command-line tool suitable for tech support usage

2018-09-20 11:19:36 -04:00

HashCertificateCommand.java

…

HelpCommand.java

Remove unnecessary "throws" declarations

2018-06-18 18:17:56 -04:00

Injector.java

…

LevelDbLogReader.java

…

ListCursorsCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

ListDomainsCommand.java

Default to REAL TLDs in nomulus list_domains command

2018-10-22 18:53:16 -04:00

ListHostsCommand.java

…

ListObjectsCommand.java

Refactor AppEngineConnection

2018-10-29 15:34:12 -04:00

ListPremiumListsCommand.java

…

ListRegistrarsCommand.java

…

ListReservedListsCommand.java

…

ListTldsCommand.java

…

LoadSnapshotCommand.java

Remove more unnecessary "throws" declarations

2018-06-27 15:28:52 -04:00

LoadTestCommand.java

Refactor AppEngineConnection

2018-10-29 15:34:12 -04:00

LockDomainCommand.java

Remove unnecessary "throws" declarations

2018-06-18 18:17:56 -04:00

LockOrUnlockDomainCommand.java

Default clientId to registry's registrar in domain lock commands

2018-06-06 15:17:36 -04:00

logging.properties

Automated g4 rollback of changelist 212879670.

2018-10-03 11:50:47 -04:00

LoggingParameters.java

…

LoginCommand.java

…

LogoutCommand.java

Migrate to Flogger (green)

2018-05-30 12:18:54 -04:00

MutatingCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

MutatingEppToolCommand.java

Remove unnecessary "throws" declarations

2018-06-18 18:17:56 -04:00

NonMutatingEppToolCommand.java

Don't prompt to confirm non-mutating nomulus EPP tool commands

2018-04-10 16:53:18 -04:00

package-info.java

…

PendingEscrowCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

RecordAccumulator.java

…

RegistrarContactCommand.java

Add registrar contact to OTE registrars

2018-10-22 18:58:06 -04:00

RegistryCli.java

Fix the tool that was broken in []

2018-11-02 14:39:36 -04:00

RegistryTool.java

Add SetNumInstancesCommand in Nomulus tool to adjust the number of instances

2018-11-12 14:51:40 -05:00

RegistryToolComponent.java

Add option --non_live_versions to set_num_instances command

2018-12-03 18:58:05 -05:00

RegistryToolEnvironment.java

Restore original System Properties after tests

2018-11-14 11:57:23 -05:00

RenewDomainCommand.java

Add a nomulus renew_domain command

2018-06-06 15:16:09 -04:00

ResaveEntitiesCommand.java

Remove unnecessary "throws" declarations

2018-06-18 18:17:56 -04:00

ResaveEnvironmentEntitiesCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

ResaveEppResourceCommand.java

Remove unnecessary "throws" declarations

2018-06-18 18:17:56 -04:00

SendEscrowReportToIcannCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

SetNumInstancesCommand.java

Add option --non_live_versions to set_num_instances command

2018-12-03 18:58:05 -05:00

SetupOteCommand.java

Change OTE "fake" address state to a valid value

2018-11-14 11:59:04 -05:00

ShellCommand.java

Make OutputEncapsulator a CommandRunner

2018-11-12 14:51:40 -05:00

UniformRapidSuspensionCommand.java

Remove unnecessary "throws" declarations

2018-06-18 18:17:56 -04:00

UnlockDomainCommand.java

Remove unnecessary "throws" declarations

2018-06-18 18:17:56 -04:00

UpdateApplicationStatusCommand.java

Remove unnecessary "throws" declarations

2018-06-18 18:17:56 -04:00

UpdateClaimsNoticeCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

UpdateCursorsCommand.java

Remove unnecessary "throws" declarations

2018-06-18 18:17:56 -04:00

UpdateDomainCommand.java

Allow square bracket expansion when specifying nameservers

2018-08-10 13:44:25 -04:00

UpdateKmsKeyringCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

UpdatePremiumListCommand.java

…

UpdateRegistrarCommand.java

…

UpdateReservedListCommand.java

…

UpdateSandboxTldCommand.java

Add a command-line tool suitable for tech support usage

2018-09-20 11:19:36 -04:00

UpdateServerLocksCommand.java

…

UpdateSmdCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

UpdateTldCommand.java

Remove unnecessary "throws" declarations

2018-06-18 18:17:56 -04:00

UploadClaimsListCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00

UserIdProvider.java

…

ValidateEscrowDepositCommand.java

Simplify the Ghostryde API

2018-06-27 15:28:53 -04:00

ValidateLoginCredentialsCommand.java

Remove checking of SNI headers

2018-09-14 21:34:44 -04:00

VerifyOteCommand.java

Make the verify OT&E action more performant

2018-12-03 19:07:36 -05:00

WhoisQueryCommand.java

Rename RemoteApiCommand to CommandWithRemoteApi

2018-09-08 00:12:31 -04:00