LogsExportCursor was only used by ExportLogsTaskServlet, which we removed a long time ago. It's just dead code. The PersistedRangeLong type was only written for use by LogsExportCursor, and since it hasn't picked up new users in 3+ years I don't think we need to keep it around.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=169264994
Sometimes requests "die" suddenly, without going through catch/finally blocks.
If this happens, any lock they own will remain locked until it times out (which
can take hours in some cases).
This cl implicitly unlocks any lock if the owner of the lock isn't running
anymore.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=168880938
Allow superusers to change the transfer period to zero years and allow
superusers to change the automatic transfer length.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=167598314
This is the last of many cls adding explicit logging in all our domain
mutation flows to facilitate transaction reporting.
The transfer process is as follows:
GAINING sends a TransferRequest to LOSING
LOSING either acks (TransferApprove), nacks (TransferReject) or does nothing
(auto approve). For acks and autoapproves, we produce a +1 counter for GAINING
and LOSING for domain-gaining/losing-successful for each registrar, to be
reported on the approve date + the transfer grace period. For nacks, we produce
a +1 counter for domain-gaining/losing-nacked for each registrar.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=166535579
To log autorenews, we currently run a mapreduce daily that creates synthetic
billing events for each recurring event past its due time. These are all
parented under the original recurring event, which allows these synthetic events to incorrectly stack on the original mutating entry.
We now explicitly create a new HistoryEntry of type DOMAIN_AUTORENEW to log
autorenews alongside other mutating EPP flows. These also parent DomainTransactionRecords for the NET_RENEWS_1_YEAR field, with the reporting time equal to the billing time (which accounts for the autorenew grace period).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=166379700
After working further with domain deletes, I realized we'll need to record multiple reportingTimes under a single historyEntry when issuing a -1 counter to cancel grace-period adds. Since the TLD would be the only shared component within a record, we'll just duplicate it across all records to save an unnecessary layer of hierarchy.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=166261413
This change adds the persisted data model necessary to facilitate transaction
reporting. TransactionRecord is an embedded repeated class within HistoryEntry
which is only added to when a HistoryEntry is created that counts towards
transaction reporting.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=165619552
It was buggy (didn't work) and was never actually used.
Why never actually used: for it to be used executeWithLock has to be called
with different requesters on the same lockId. That never happend in the code.
How it was buggy: Logically, the queue is deleted on release of the lock (meaning it was
meaningless the only time it mattered - when the lock isn't taken). In
addition, a different bug meant that having items in the queue prevented the
lock from being released forcing all other tasks to have to wait for lock
timeout even if the task that acquired the lock is long done.
Alternative: fix the queue. This would mean we don't want to delete the lock on release (since we want to keep the queue). Instead, we resave the same lock with expiration date being START_OF_TIME. In addition - we need to fix the .equals used to determine if the lock the same as the acquired lock - instead use some isSame function that ignores the queue.
Note: the queue is dangerous! An item (calling class / action) in the first place of a queue means no other calling class can get that lock. Everything is waiting for the first calling class to be re-run - but that might take a long time (depending on that action's rerun policy) and even might never happen (if for some reason that action decided it was no longer needed without acquiring the lock) - causing all other actions to stall forever!
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=163705463
Note that even though the nomulus command line tool now supports multiple
DNS writers for all subcommands, this still won't work quite yet because
the DNS task queue format migration from [] is still in progress.
After next week's push that migration will be complete and we can remove
the final restriction against only having one DNS writer per TLD.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162490399
This is the first step in a multi-step data migration to allow multiple
DNS writers per TLD. The overall process looks like this:
1. Add a plural DNS writers field with backfill (this commit).
2. Deploy it.
3. Run the ResaveEnvironmentEntitiesCommand to populate this new field
on all entities.
4. Update the code to use the new field everywhere.
5. Deploy it.
6. Delete the now-unreferenced, old deprecated singular value field.
This process is rollback-safe.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161253436
Also added corresponding getters and setters for the new field. Note that
nothing has changed on the RDAP front for now, as the CL&D only concerns WHOIS.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155116134
We now send PendingActionNotificationResponses in our poll messages upon completion of an asynchronous contact or host deletion. This is part 1 of 2, which begins logging Trid in all enqueued Host/Contact deletion flows for use in batch deletions, and optionally consuming the resultant Trid info to emit a Host/ContactPendingActionNotifcationResponse.
Part 2 will make this response emission non-optional, which will happen once the queue is cleared of all non-Trid containing tasks.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=153084197
This is a follow-up to Lai's refactoring of the get reservation types
code to return a set rather than a single type. Since we're always
returning a set now, the more natural way to represent a label that is
not reserved is to return an empty set rather than a set containing
UNRESERVED.
Also fixes some minor style issues I ran across regarding static
importing and test method naming that I ran across (no logic
implications).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=151132116
A CurrencyUnit-to-BillingAccountEntry map is persisted in the Registrar entity. It provides flexibility for billing systems that assign different account ids for accounts under different currencies of the same registrar.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=151022753
During domain create/applicationcreate/allocate, domains that are on the reserved list(s) with nameserver restricted reservation type must set nameservers that are part of the allowed nameservers for that domain in the reserved list(s) applied to that TLD.
Additionally a boolean is added to Registry to indicate if a TLD is restricting domain create. If it is, only domains that are nameserver restricted can be registered.
For consistency with a similar feature that validates a TLD-wide nameserver whitelist, the per-domain nameserver validation is performed even when the operation is in super-user mode. Similarly, if a domain is nameserver restricted, nameservers must be supplied (i. e. the nameservers set cannot be empty) when registering the domain.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=150641269
Now that transfers are always restricted to 1 year, it's unnecessary to store
extendedRegistrationYears on TransferData - it will always be equal to 1. This
simplifies logic in a few other places, e.g. RdeDomainImportAction.
I verified in BigQuery that no DomainBases exist with extendedRegistrationYears
values that aren't either null or equal to 1. At some point we should remove
the persisted fields from datastore via e.g. resaving all those domains, but
it's low priority and can wait until we have some more pressing migration.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=150373897
A new field (allowedNameservers) is added to ReservedListEntry that stores the allow nameservers for the label. The field itself is a comma separated string, but the actual lines within a reserved list file (from which the field is parsed) uses colon to separate nameservers, to avoid conflicting with the commas used as primary separators in a CSV file.
Combined with upcoming update(s) that enables locking down an entire TLD to only delegate domains with a nameserver restricted reservation type, this change will enable us to restrict domain delegation to nameservers specifically specified in the allowed nameservers list, in order to prevent malicious delegation in case the registrar for a brand TLD is compromised.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149989330
This is the first step in the migration to remove the need to load all of
the premium list entries every time the cache expires (which causes slow-
downs). Once this is deployed, we can re-save all premium lists, creating
the bloom filters, and then the next step will be to read from them to
more efficiently determine if a label might be premium.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147525017
and HostResource.
DomainApplication is not transferable and has no need for this
field. HostResource needs it because it can be transferred with
a domain.
This is all in service of removing the ofy().load() inside of
host's cloneProjectedAtTime.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=139346925
HostResource and DomainApplication are not transferable, (or at
least, not directly in the case of hosts) and have no need for
the TransferData field. In a flat-flow world, we can push it down
to where it's actually used.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=139201423
This is the third and final phase in the migration away from ReferenceUnions.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=138778148
Right now, DomainApplicationCreateFlow checks to make sure that the registration period is in years, but doesn't store it in the DomainApplication explicitly. Instead, when DomainAllocateFlow runs later, it goes back to the XML in the HistoryEntry to get the number of years. Corey suggests that it would be cleaner to store the number of years in the DomainApplication. This is stage one of a data migration; we store the value, but don't actually read it anywhere except in tests. If we have any outstanding domain applications, we will then need to write a scrap tool to populate the years field, after which we can start relying on the field.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=137317739
It is important to get at least this one commit in before the public Nomulus
release so that none of our public users will have to go through this data
migration (although we will have to).
The migration strategy is as follows:
1. Dual-write to non-ReferenceUnion fields in addition to the current
ReferenceUnion fields in use, and add new indexes (this commit). Deploy.
2. Run the ResaveAllEppResourcesAction backfill [].
3. Switch all code over to using the new fields. Dual-write is still in effect,
except it is now copying over the values of the new fields to the old
fields. Switch over all BigQuery reporting scripts to use the new
fields. Deploy.
4. Remove all of the old code and indexes. Deploy.
5. (Optional, at our leisure) Re-run the ResaveAllEppResourcesAction backfill
[] to delete the old obsolete fields.
Note that this migration strategy is rollback-safe at every step -- new data is
not read until it has already been written out in the previous step, and old
data is not removed immediately following a step in which it was still being
read, so the previous step is safe to roll back to.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136196988
This is to better distinguish between an LRP "token" (the string passed along in EPP) and the datastore entity that contains the token and all metadata.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=135943480
The reason field is 1:1 mapped to skus in billing reports. Need to add a new reason for EAP for this type of billing event for reporting to work correctly. Also map that reason to the correct SKU.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=134005364
When EAP is involed we current have one billing event for domain create that
has the create fee and EAP fee lumped together. Change it to record two
separate billing events for each.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132335349
This change replaces all Ref objects in the code with Key objects. These are
stored in datastore as the same object (raw datastore keys), so this is not
a model change.
Our best practices doc says to use Keys not Refs because:
* The .get() method obscures what's actually going on
- Much harder to visually audit the code for datastore loads
- Hard to distinguish Ref<T> get()'s from Optional get()'s and Supplier get()'s
* Implicit ofy().load() offers much less control
- Antipattern for ultimate goal of making Ofy injectable
- Can't control cache use or batch loading without making ofy() explicit anyway
* Serialization behavior is surprising and could be quite dangerous/incorrect
- Can lead to serialization errors. If it actually worked "as intended",
it would lead to a Ref<> on a serialized object being replaced upon
deserialization with a stale copy of the old value, which could potentially
break all kinds of transactional expectations
* Having both Ref<T> and Key<T> introduces extra boilerplate everywhere
- E.g. helper methods all need to have Ref and Key overloads, or you need to
call .key() to get the Key<T> for every Ref<T> you want to pass in
- Creating a Ref<T> is more cumbersome, since it doesn't have all the create()
overloads that Key<T> has, only create(Key<T>) and create(Entity) - no way to
create directly from kind+ID/name, raw Key, websafe key string, etc.
(Note that Refs are treated specially by Objectify's @Load method and Keys are not;
we don't use that feature, but it is the one advantage Refs have over Keys.)
The direct impetus for this change is that I am trying to audit our use of memcache,
and the implicit .get() calls to datastore were making that very hard.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=131965491
Adds the ability to search for entities (contacts and registrars) by name.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130305930
Also had to add an EnumParameter class to support
List<T extends Enum<T>>, as these aren't natively supported by
JCommander (although single Enum parameters are.)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=129464699
This is one of several CLs in a sequence for allowing per-TLD DNS
implementations.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=129445641
The dark lord Gosling designed the Java package naming system so that
ownership flows from the DNS system. Since we own the domain name
registry.google, it seems only appropriate that we should use
google.registry as our package name.
This change renames directories in preparation for the great package
rename. The repository is now in a broken state because the code
itself hasn't been updated. However this should ensure that git
correctly preserves history for each file.
