Use the parent to store the history repo ID and fill in the base object (#830 )

* Use the parent to store the history repo ID and fill in the base object Storing the repo ID in the parent and in the base object has two primary benefits. First, it unifies the parent information in the HistoryEntry's `parent` object. This simplifies the builders and the data flow. Second, when possible (which should be always, post-migration) we fill out the DomainContent's repo ID (similarly for the other EPP resources) which means that when reconstituting the ofy keys we don't need to pass the repo ID in from a separate object. This way, all the data are encapsulated where they should be. The primary downside here is that it further reduces the "immutability" of the history objects (since we're using the Hibernate setter for the parent repo ID) but we weren't immutable anyway. * Respond to CR - compare the entire vkeys in tests - always return the parent for repo ID * Simplify creation of parent VKeys * Fix flipped isAssignableFrom check in VKey * Merge remote-tracking branch 'origin/master' into historyRepoId
Revert package-lock.json to version from #676 (#834 )
2026-06-09 16:33:02 +00:00 · 2020-10-09 16:01:51 -04:00 · 2020-10-09 15:57:23 -04:00 · 2020-10-09 15:02:11 -04:00 · 2020-10-09 14:50:21 -04:00 · 2020-10-09 13:32:52 -04:00
221 changed files with 4038 additions and 1071 deletions
@@ -191,7 +191,7 @@ allprojects {
 }

 task runPresubmits(type: Exec) {
-  executable '/usr/bin/python'
+  executable '/usr/bin/python3'
  args('config/presubmits.py')
 }

@@ -260,7 +260,7 @@ subprojects {
      // in the 'configurations' block, the following code must run after
      // project evaluation, when all configurations have been created.
      configurations.each {
-        if (it.name != 'dependencyLicenseReport') {
+        if (it.name != 'dependencyLicenseReport' && it.name != 'integration') {
          it.resolutionStrategy.activateDependencyLocking()
        }
      }
@@ -16,7 +16,7 @@
 """

 import argparse
-import attr
+import dataclasses
 import io
 import os
 import shutil
@@ -25,7 +25,7 @@ import sys
 from typing import List, Union


-@attr.s(auto_attribs=True)
+@dataclasses.dataclass
 class Property:
    name : str = ''
    desc : str = ''
@@ -39,7 +39,7 @@ class Property:
                raise ValidationError('value of {self.name} must be "true" or '
                                      '"false".')

-@attr.s(auto_attribs=True)
+@dataclasses.dataclass
 class GradleFlag:
    flags : Union[str, List[str]]
    desc : str
@@ -75,6 +75,7 @@ Pseudo-commands:
 """

 # Define all of our special gradle properties here.
+# TODO(b/169318491): use consistent naming style for properties and variables.
 PROPERTIES = [
    Property('mavenUrl',
             'URL to use for the main maven repository (defaults to maven '
@@ -124,6 +125,9 @@ PROPERTIES = [
             'server/schema integration tests. Please refer to <a '
             'href="./integration/README.md">integration project</a> for more '
             'information.'),
+    Property('baseSchemaTag',
+             'The nomulus version tag of the schema for use in the schema'
+             'deployment integration test (:db:schemaIncrementalDeployTest)'),
    Property('schema_version',
             'The nomulus version tag of the schema for use in a database'
             'integration test.'),
@@ -18,6 +18,7 @@ Error Prone) so we must write them manually.
 """

 import os
+from typing import List, Tuple
 import sys
 import re

@@ -178,6 +179,90 @@ PRESUBMITS = {
        "JavaScript files should not include console logging."
 }

+# Note that this regex only works for one kind of Flyway file.  If we want to
+# start using "R" and "U" files we'll need to update this script.
+FLYWAY_FILE_RX = re.compile(r'V(\d+)__.*')
+
+
+def get_seqnum(filename: str, location: str) -> int:
+  """Extracts the sequence number from a filename."""
+  m = FLYWAY_FILE_RX.match(filename)
+  if m is None:
+    raise ValueError('Illegal Flyway filename: %s in %s' % (filename, location))
+  return int(m.group(1))
+
+
+def files_by_seqnum(files: List[str], location: str) -> List[Tuple[int, str]]:
+  """Returns the list of seqnum, filename sorted by sequence number."""
+  return [(get_seqnum(filename, location), filename) for filename in files]
+
+
+def has_valid_order(indexed_files: List[Tuple[int, str]], location: str) -> bool:
+  """Verify that sequence numbers are in order without gaps or duplicates.
+
+  Args:
+    files: List of seqnum, filename for a list of Flyway files.
+    location: Where the list of files came from (for error reporting).
+
+  Returns:
+    True if the file list is valid.
+  """
+  last_index = 0
+  valid = True
+  for seqnum, filename in indexed_files:
+    if seqnum == last_index:
+      print('duplicate Flyway file sequence number found in %s: %s' %
+            (location, filename))
+      valid = False
+    elif seqnum < last_index:
+      print('File %s in %s is out of order.' % (filename, location))
+      valid = False
+    elif seqnum != last_index + 1:
+      print('Missing Flyway sequence number %d in %s.  Next file is %s' %
+            (last_index + 1, location, filename))
+      valid = False
+    last_index = seqnum
+  return valid
+
+
+def verify_flyway_index():
+  """Verifies that the Flyway index file is in sync with the directory."""
+  success = True
+
+  # Sort the files in the Flyway directory by their sequence number.
+  files = sorted(
+      files_by_seqnum(os.listdir('db/src/main/resources/sql/flyway'),
+                      'Flyway directory'))
+
+  # Make sure that there are no gaps and no duplicate sequence numbers in the
+  # files themselves.
+  if not has_valid_order(files, 'Flyway directory'):
+    success = False
+
+  # Remove the sequence numbers and compare against the index file contents.
+  files = [filename[1] for filename in sorted(files)]
+  with open('db/src/main/resources/sql/flyway.txt') as index:
+    indexed_files = index.read().splitlines()
+  if files != indexed_files:
+    unindexed = set(files) - set(indexed_files)
+    if unindexed:
+      print('The following Flyway files are not in flyway.txt: %s' % unindexed)
+
+    nonexistent = set(indexed_files) - set(files)
+    if nonexistent:
+      print('The following files are in flyway.txt but not in the Flyway '
+            'directory: %s' % nonexistent)
+
+    # Do an ordering check on the index file (ignore the result, we're failing
+    # anyway).
+    has_valid_order(files_by_seqnum(indexed_files, 'flyway.txt'), 'flyway.txt')
+    success = False
+
+  if not success:
+    print('Please fix any conflicts and run "./nom_build :db:generateFlywayIndex"')
+
+  return not success
+

 def get_files():
  for root, dirnames, filenames in os.walk("."):
@@ -197,5 +282,10 @@ if __name__ == "__main__":
      failed = True
      print("%s had errors: \n  %s" % (file, "\n  ".join(error_messages)))

+  # And now for something completely different: check to see if the Flyway
+  # index is up-to-date.  It's quicker to do it here than in the unit tests:
+  # when we put it here it fails fast before all of the tests are run.
+  failed |= verify_flyway_index()
+
  if failed:
    sys.exit(1)
@@ -245,6 +245,7 @@ dependencies {
  testCompile deps['org.apache.ftpserver:ftpserver-core']
  compile deps['org.apache.httpcomponents:httpclient']
  compile deps['org.apache.httpcomponents:httpcore']
+  runtime deps['org.apache.logging.log4j:log4j-core']
  testCompile deps['org.apache.sshd:sshd-core']
  testCompile deps['org.apache.sshd:sshd-scp']
  testCompile deps['org.apache.sshd:sshd-sftp']
@@ -269,6 +270,8 @@ dependencies {
  compile deps['org.testcontainers:postgresql']
  testCompile deps['org.testcontainers:selenium']
  testCompile deps['org.testcontainers:testcontainers']
+  compile deps['us.fatehi:schemacrawler']
+  compile deps['us.fatehi:schemacrawler-diagram']
  compile deps['xerces:xmlParserAPIs']
  compile deps['xpp3:xpp3']
  // This dependency must come after javax.mail:mail as it would otherwise
@@ -202,6 +202,7 @@ org.apache.httpcomponents:httpclient:4.5.11
 org.apache.httpcomponents:httpcore:4.4.13
 org.apache.logging.log4j:log4j-api:2.6.2
 org.bouncycastle:bcpg-jdk15on:1.61
+org.bouncycastle:bcpkix-jdk15on:1.61
 org.bouncycastle:bcprov-jdk15on:1.61
 org.checkerframework:checker-compat-qual:2.5.5
 org.checkerframework:checker-qual:2.11.1
@@ -249,5 +250,10 @@ org.tukaani:xz:1.8
 org.w3c.css:sac:1.3
 org.xerial.snappy:snappy-java:1.1.4
 org.yaml:snakeyaml:1.17
+us.fatehi:schemacrawler-api:16.10.1
+us.fatehi:schemacrawler-diagram:16.10.1
+us.fatehi:schemacrawler-tools:16.10.1
+us.fatehi:schemacrawler-utility:16.10.1
+us.fatehi:schemacrawler:16.10.1
 xerces:xmlParserAPIs:2.6.2
 xpp3:xpp3:1.1.4c
@@ -200,6 +200,7 @@ org.apache.httpcomponents:httpclient:4.5.11
 org.apache.httpcomponents:httpcore:4.4.13
 org.apache.logging.log4j:log4j-api:2.6.2
 org.bouncycastle:bcpg-jdk15on:1.61
+org.bouncycastle:bcpkix-jdk15on:1.61
 org.bouncycastle:bcprov-jdk15on:1.61
 org.checkerframework:checker-compat-qual:2.5.5
 org.checkerframework:checker-qual:2.11.1
@@ -246,5 +247,10 @@ org.tukaani:xz:1.8
 org.w3c.css:sac:1.3
 org.xerial.snappy:snappy-java:1.1.4
 org.yaml:snakeyaml:1.17
+us.fatehi:schemacrawler-api:16.10.1
+us.fatehi:schemacrawler-diagram:16.10.1
+us.fatehi:schemacrawler-tools:16.10.1
+us.fatehi:schemacrawler-utility:16.10.1
+us.fatehi:schemacrawler:16.10.1
 xerces:xmlParserAPIs:2.6.2
 xpp3:xpp3:1.1.4c
@@ -201,8 +201,10 @@ org.apache.commons:commons-compress:1.20
 org.apache.commons:commons-lang3:3.5
 org.apache.httpcomponents:httpclient:4.5.11
 org.apache.httpcomponents:httpcore:4.4.13
-org.apache.logging.log4j:log4j-api:2.6.2
+org.apache.logging.log4j:log4j-api:2.13.3
+org.apache.logging.log4j:log4j-core:2.13.3
 org.bouncycastle:bcpg-jdk15on:1.61
+org.bouncycastle:bcpkix-jdk15on:1.61
 org.bouncycastle:bcprov-jdk15on:1.61
 org.checkerframework:checker-compat-qual:2.5.5
 org.checkerframework:checker-qual:2.11.1
@@ -251,5 +253,10 @@ org.tukaani:xz:1.8
 org.w3c.css:sac:1.3
 org.xerial.snappy:snappy-java:1.1.4
 org.yaml:snakeyaml:1.17
+us.fatehi:schemacrawler-api:16.10.1
+us.fatehi:schemacrawler-diagram:16.10.1
+us.fatehi:schemacrawler-tools:16.10.1
+us.fatehi:schemacrawler-utility:16.10.1
+us.fatehi:schemacrawler:16.10.1
 xerces:xmlParserAPIs:2.6.2
 xpp3:xpp3:1.1.4c
@@ -200,8 +200,10 @@ org.apache.commons:commons-compress:1.20
 org.apache.commons:commons-lang3:3.5
 org.apache.httpcomponents:httpclient:4.5.11
 org.apache.httpcomponents:httpcore:4.4.13
-org.apache.logging.log4j:log4j-api:2.6.2
+org.apache.logging.log4j:log4j-api:2.13.3
+org.apache.logging.log4j:log4j-core:2.13.3
 org.bouncycastle:bcpg-jdk15on:1.61
+org.bouncycastle:bcpkix-jdk15on:1.61
 org.bouncycastle:bcprov-jdk15on:1.61
 org.checkerframework:checker-compat-qual:2.5.5
 org.checkerframework:checker-qual:2.11.1
@@ -249,5 +251,10 @@ org.tukaani:xz:1.8
 org.w3c.css:sac:1.3
 org.xerial.snappy:snappy-java:1.1.4
 org.yaml:snakeyaml:1.17
+us.fatehi:schemacrawler-api:16.10.1
+us.fatehi:schemacrawler-diagram:16.10.1
+us.fatehi:schemacrawler-tools:16.10.1
+us.fatehi:schemacrawler-utility:16.10.1
+us.fatehi:schemacrawler:16.10.1
 xerces:xmlParserAPIs:2.6.2
 xpp3:xpp3:1.1.4c
@@ -202,6 +202,7 @@ org.apache.httpcomponents:httpclient:4.5.11
 org.apache.httpcomponents:httpcore:4.4.13
 org.apache.logging.log4j:log4j-api:2.6.2
 org.bouncycastle:bcpg-jdk15on:1.61
+org.bouncycastle:bcpkix-jdk15on:1.61
 org.bouncycastle:bcprov-jdk15on:1.61
 org.checkerframework:checker-compat-qual:2.5.5
 org.checkerframework:checker-qual:2.11.1
@@ -249,5 +250,10 @@ org.tukaani:xz:1.8
 org.w3c.css:sac:1.3
 org.xerial.snappy:snappy-java:1.1.4
 org.yaml:snakeyaml:1.17
+us.fatehi:schemacrawler-api:16.10.1
+us.fatehi:schemacrawler-diagram:16.10.1
+us.fatehi:schemacrawler-tools:16.10.1
+us.fatehi:schemacrawler-utility:16.10.1
+us.fatehi:schemacrawler:16.10.1
 xerces:xmlParserAPIs:2.6.2
 xpp3:xpp3:1.1.4c
@@ -201,6 +201,7 @@ org.apache.httpcomponents:httpclient:4.5.11
 org.apache.httpcomponents:httpcore:4.4.13
 org.apache.logging.log4j:log4j-api:2.6.2
 org.bouncycastle:bcpg-jdk15on:1.61
+org.bouncycastle:bcpkix-jdk15on:1.61
 org.bouncycastle:bcprov-jdk15on:1.61
 org.checkerframework:checker-compat-qual:2.5.5
 org.checkerframework:checker-qual:2.11.1
@@ -248,5 +249,10 @@ org.tukaani:xz:1.8
 org.w3c.css:sac:1.3
 org.xerial.snappy:snappy-java:1.1.4
 org.yaml:snakeyaml:1.17
+us.fatehi:schemacrawler-api:16.10.1
+us.fatehi:schemacrawler-diagram:16.10.1
+us.fatehi:schemacrawler-tools:16.10.1
+us.fatehi:schemacrawler-utility:16.10.1
+us.fatehi:schemacrawler:16.10.1
 xerces:xmlParserAPIs:2.6.2
 xpp3:xpp3:1.1.4c
@@ -200,8 +200,10 @@ org.apache.commons:commons-compress:1.20
 org.apache.commons:commons-lang3:3.5
 org.apache.httpcomponents:httpclient:4.5.11
 org.apache.httpcomponents:httpcore:4.4.13
-org.apache.logging.log4j:log4j-api:2.6.2
+org.apache.logging.log4j:log4j-api:2.13.3
+org.apache.logging.log4j:log4j-core:2.13.3
 org.bouncycastle:bcpg-jdk15on:1.61
+org.bouncycastle:bcpkix-jdk15on:1.61
 org.bouncycastle:bcprov-jdk15on:1.61
 org.checkerframework:checker-compat-qual:2.5.5
 org.checkerframework:checker-qual:2.11.1
@@ -249,5 +251,10 @@ org.tukaani:xz:1.8
 org.w3c.css:sac:1.3
 org.xerial.snappy:snappy-java:1.1.4
 org.yaml:snakeyaml:1.17
+us.fatehi:schemacrawler-api:16.10.1
+us.fatehi:schemacrawler-diagram:16.10.1
+us.fatehi:schemacrawler-tools:16.10.1
+us.fatehi:schemacrawler-utility:16.10.1
+us.fatehi:schemacrawler:16.10.1
 xerces:xmlParserAPIs:2.6.2
 xpp3:xpp3:1.1.4c
@@ -200,8 +200,10 @@ org.apache.commons:commons-compress:1.20
 org.apache.commons:commons-lang3:3.5
 org.apache.httpcomponents:httpclient:4.5.11
 org.apache.httpcomponents:httpcore:4.4.13
-org.apache.logging.log4j:log4j-api:2.6.2
+org.apache.logging.log4j:log4j-api:2.13.3
+org.apache.logging.log4j:log4j-core:2.13.3
 org.bouncycastle:bcpg-jdk15on:1.61
+org.bouncycastle:bcpkix-jdk15on:1.61
 org.bouncycastle:bcprov-jdk15on:1.61
 org.checkerframework:checker-compat-qual:2.5.5
 org.checkerframework:checker-qual:2.11.1
@@ -249,5 +251,10 @@ org.tukaani:xz:1.8
 org.w3c.css:sac:1.3
 org.xerial.snappy:snappy-java:1.1.4
 org.yaml:snakeyaml:1.17
+us.fatehi:schemacrawler-api:16.10.1
+us.fatehi:schemacrawler-diagram:16.10.1
+us.fatehi:schemacrawler-tools:16.10.1
+us.fatehi:schemacrawler-utility:16.10.1
+us.fatehi:schemacrawler:16.10.1
 xerces:xmlParserAPIs:2.6.2
 xpp3:xpp3:1.1.4c
@@ -200,8 +200,10 @@ org.apache.commons:commons-compress:1.20
 org.apache.commons:commons-lang3:3.5
 org.apache.httpcomponents:httpclient:4.5.11
 org.apache.httpcomponents:httpcore:4.4.13
-org.apache.logging.log4j:log4j-api:2.6.2
+org.apache.logging.log4j:log4j-api:2.13.3
+org.apache.logging.log4j:log4j-core:2.13.3
 org.bouncycastle:bcpg-jdk15on:1.61
+org.bouncycastle:bcpkix-jdk15on:1.61
 org.bouncycastle:bcprov-jdk15on:1.61
 org.checkerframework:checker-compat-qual:2.5.5
 org.checkerframework:checker-qual:2.11.1
@@ -249,5 +251,10 @@ org.tukaani:xz:1.8
 org.w3c.css:sac:1.3
 org.xerial.snappy:snappy-java:1.1.4
 org.yaml:snakeyaml:1.17
+us.fatehi:schemacrawler-api:16.10.1
+us.fatehi:schemacrawler-diagram:16.10.1
+us.fatehi:schemacrawler-tools:16.10.1
+us.fatehi:schemacrawler-utility:16.10.1
+us.fatehi:schemacrawler:16.10.1
 xerces:xmlParserAPIs:2.6.2
 xpp3:xpp3:1.1.4c
@@ -200,8 +200,10 @@ org.apache.commons:commons-compress:1.20
 org.apache.commons:commons-lang3:3.5
 org.apache.httpcomponents:httpclient:4.5.11
 org.apache.httpcomponents:httpcore:4.4.13
-org.apache.logging.log4j:log4j-api:2.6.2
+org.apache.logging.log4j:log4j-api:2.13.3
+org.apache.logging.log4j:log4j-core:2.13.3
 org.bouncycastle:bcpg-jdk15on:1.61
+org.bouncycastle:bcpkix-jdk15on:1.61
 org.bouncycastle:bcprov-jdk15on:1.61
 org.checkerframework:checker-compat-qual:2.5.5
 org.checkerframework:checker-qual:2.11.1
@@ -249,5 +251,10 @@ org.tukaani:xz:1.8
 org.w3c.css:sac:1.3
 org.xerial.snappy:snappy-java:1.1.4
 org.yaml:snakeyaml:1.17
+us.fatehi:schemacrawler-api:16.10.1
+us.fatehi:schemacrawler-diagram:16.10.1
+us.fatehi:schemacrawler-tools:16.10.1
+us.fatehi:schemacrawler-utility:16.10.1
+us.fatehi:schemacrawler:16.10.1
 xerces:xmlParserAPIs:2.6.2
 xpp3:xpp3:1.1.4c
@@ -298,5 +298,10 @@ org.tukaani:xz:1.8
 org.w3c.css:sac:1.3
 org.xerial.snappy:snappy-java:1.1.4
 org.yaml:snakeyaml:1.17
+us.fatehi:schemacrawler-api:16.10.1
+us.fatehi:schemacrawler-diagram:16.10.1
+us.fatehi:schemacrawler-tools:16.10.1
+us.fatehi:schemacrawler-utility:16.10.1
+us.fatehi:schemacrawler:16.10.1
 xerces:xmlParserAPIs:2.6.2
 xpp3:xpp3:1.1.4c
@@ -297,5 +297,10 @@ org.tukaani:xz:1.8
 org.w3c.css:sac:1.3
 org.xerial.snappy:snappy-java:1.1.4
 org.yaml:snakeyaml:1.17
+us.fatehi:schemacrawler-api:16.10.1
+us.fatehi:schemacrawler-diagram:16.10.1
+us.fatehi:schemacrawler-tools:16.10.1
+us.fatehi:schemacrawler-utility:16.10.1
+us.fatehi:schemacrawler:16.10.1
 xerces:xmlParserAPIs:2.6.2
 xpp3:xpp3:1.1.4c
@@ -216,7 +216,8 @@ org.apache.ftpserver:ftplet-api:1.0.6
 org.apache.ftpserver:ftpserver-core:1.0.6
 org.apache.httpcomponents:httpclient:4.5.11
 org.apache.httpcomponents:httpcore:4.4.13
-org.apache.logging.log4j:log4j-api:2.6.2
+org.apache.logging.log4j:log4j-api:2.13.3
+org.apache.logging.log4j:log4j-core:2.13.3
 org.apache.mina:mina-core:2.0.4
 org.apache.sshd:sshd-core:2.0.0
 org.apache.sshd:sshd-scp:2.0.0
@@ -301,5 +302,10 @@ org.tukaani:xz:1.8
 org.w3c.css:sac:1.3
 org.xerial.snappy:snappy-java:1.1.4
 org.yaml:snakeyaml:1.17
+us.fatehi:schemacrawler-api:16.10.1
+us.fatehi:schemacrawler-diagram:16.10.1
+us.fatehi:schemacrawler-tools:16.10.1
+us.fatehi:schemacrawler-utility:16.10.1
+us.fatehi:schemacrawler:16.10.1
 xerces:xmlParserAPIs:2.6.2
 xpp3:xpp3:1.1.4c
@@ -216,7 +216,8 @@ org.apache.ftpserver:ftplet-api:1.0.6
 org.apache.ftpserver:ftpserver-core:1.0.6
 org.apache.httpcomponents:httpclient:4.5.11
 org.apache.httpcomponents:httpcore:4.4.13
-org.apache.logging.log4j:log4j-api:2.6.2
+org.apache.logging.log4j:log4j-api:2.13.3
+org.apache.logging.log4j:log4j-core:2.13.3
 org.apache.mina:mina-core:2.0.4
 org.apache.sshd:sshd-core:2.0.0
 org.apache.sshd:sshd-scp:2.0.0
@@ -302,5 +303,10 @@ org.tukaani:xz:1.8
 org.w3c.css:sac:1.3
 org.xerial.snappy:snappy-java:1.1.4
 org.yaml:snakeyaml:1.17
+us.fatehi:schemacrawler-api:16.10.1
+us.fatehi:schemacrawler-diagram:16.10.1
+us.fatehi:schemacrawler-tools:16.10.1
+us.fatehi:schemacrawler-utility:16.10.1
+us.fatehi:schemacrawler:16.10.1
 xerces:xmlParserAPIs:2.6.2
 xpp3:xpp3:1.1.4c
@@ -464,7 +464,7 @@ public class DeleteContactsAndHostsAction implements Runnable {
        HostResource host = (HostResource) existingResource;
        if (host.isSubordinate()) {
          dnsQueue.addHostRefreshTask(host.getHostName());
-          tm().saveNewOrUpdate(
+          tm().put(
                  tm().load(host.getSuperordinateDomain())
                      .asBuilder()
                      .removeSubordinateHost(host.getHostName())
@@ -437,7 +437,7 @@ public final class Transforms {
                .map(Optional::get)
                .map(ofy::toPojo)
                .collect(ImmutableList.toImmutableList());
-        retry(() -> jpaTm().transact(() -> jpaTm().saveNewOrUpdateAll(ofyEntities)));
+        retry(() -> jpaTm().transact(() -> jpaTm().putAll(ofyEntities)));
      }
    }

@@ -55,7 +55,7 @@ FROM (
    FROM
      `%PROJECT_ID%.%DATASTORE_EXPORT_DATA_SET%.%REGISTRY_TABLE%`
    WHERE
-      enableInvoicing IS TRUE) ) AS BillingEvent
+      invoicingEnabled IS TRUE) ) AS BillingEvent
  -- Gather billing ID from registrar table
  -- This is a 'JOIN' as opposed to 'LEFT JOIN' to filter out
  -- non-billable registrars
@@ -215,7 +215,7 @@ public class Spec11Pipeline implements Serializable {
                          .setRegistrarId(subdomain.registrarId())
                          .build();
                  JpaTransactionManager jpaTransactionManager = jpaSupplierFactory.get();
-                  jpaTransactionManager.transact(() -> jpaTransactionManager.saveNew(threatMatch));
+                  jpaTransactionManager.transact(() -> jpaTransactionManager.insert(threatMatch));
                }
              }
            }));
@@ -173,7 +173,7 @@ public final class DomainRestoreRequestFlow implements TransactionalFlow  {
            existingDomain, newExpirationTime, autorenewEvent, autorenewPollMessage, now, clientId);
    updateForeignKeyIndexDeletionTime(newDomain);
    entitiesToSave.add(newDomain, historyEntry, autorenewEvent, autorenewPollMessage);
-    tm().saveNewOrUpdateAll(entitiesToSave.build());
+    tm().putAll(entitiesToSave.build());
    tm().delete(existingDomain.getDeletePollMessage());
    dnsQueue.addDomainRefreshTask(existingDomain.getDomainName());
    return responseBuilder
@@ -67,6 +67,7 @@ import google.registry.model.domain.DomainCommand.Update.AddRemove;
 import google.registry.model.domain.DomainCommand.Update.Change;
 import google.registry.model.domain.fee.FeeUpdateCommandExtension;
 import google.registry.model.domain.metadata.MetadataExtension;
+import google.registry.model.domain.secdns.DelegationSignerData;
 import google.registry.model.domain.secdns.SecDnsUpdateExtension;
 import google.registry.model.domain.superuser.DomainUpdateSuperuserExtension;
 import google.registry.model.eppcommon.AuthInfo;
@@ -238,10 +239,16 @@ public final class DomainUpdateFlow implements TransactionalFlow {
    DomainBase.Builder domainBuilder =
        domain
            .asBuilder()
-            // Handle the secDNS extension.
+            // Handle the secDNS extension. As dsData in secDnsUpdate is read from EPP input and
+            // does not have domainRepoId set, we create a copy of the existing dsData without
+            // domainRepoId for comparison.
            .setDsData(
                secDnsUpdate.isPresent()
-                    ? updateDsData(domain.getDsData(), secDnsUpdate.get())
+                    ? updateDsData(
+                        domain.getDsData().stream()
+                            .map(DelegationSignerData::cloneWithoutDomainRepoId)
+                            .collect(toImmutableSet()),
+                        secDnsUpdate.get())
                    : domain.getDsData())
            .setLastEppUpdateTime(now)
            .setLastEppUpdateClientId(clientId)
@@ -285,7 +285,7 @@ public final class HostUpdateFlow implements TransactionalFlow {
        && newHost.isSubordinate()
        && Objects.equals(
            existingHost.getSuperordinateDomain(), newHost.getSuperordinateDomain())) {
-      tm().saveNewOrUpdate(
+      tm().put(
              tm().load(existingHost.getSuperordinateDomain())
                  .asBuilder()
                  .removeSubordinateHost(existingHost.getHostName())
@@ -294,14 +294,14 @@ public final class HostUpdateFlow implements TransactionalFlow {
      return;
    }
    if (existingHost.isSubordinate()) {
-      tm().saveNewOrUpdate(
+      tm().put(
              tm().load(existingHost.getSuperordinateDomain())
                  .asBuilder()
                  .removeSubordinateHost(existingHost.getHostName())
                  .build());
    }
    if (newHost.isSubordinate()) {
-      tm().saveNewOrUpdate(
+      tm().put(
              tm().load(newHost.getSuperordinateDomain())
                  .asBuilder()
                  .addSubordinateHost(newHost.getHostName())
@@ -64,7 +64,7 @@ public final class PollFlowUtils {
      // and re-save it for future autorenew poll messages to be delivered. Otherwise, this
      // autorenew poll message has no more events to deliver and should be deleted.
      if (nextEventTime.isBefore(autorenewPollMessage.getAutorenewEndTime())) {
-        tm().saveNewOrUpdate(autorenewPollMessage.asBuilder().setEventTime(nextEventTime).build());
+        tm().put(autorenewPollMessage.asBuilder().setEventTime(nextEventTime).build());
        includeAckedMessageInCount = isBeforeOrAt(nextEventTime, tm().getTransactionTime());
      } else {
        tm().delete(autorenewPollMessage.createVKey());
@@ -2,7 +2,7 @@
 # Script: Arab
 # Version: 1.0
 # Effective Date: 04-12-2012
-# Contact: tas-contact.google.com
+# Contact: iana-contact@google.com
 # Address: 1600 Amphitheatre Parkway Mountain View, CA 94043, USA
 # Telephone: +1 (650) 253-0000
 # Website: www.google.com
@@ -2,7 +2,7 @@
 # Script: Armn
 # Version: 1.0
 # Effective Date: 04-12-2012
-# Contact: tas-contact.google.com
+# Contact: iana-contact@google.com
 # Address: 1600 Amphitheatre Parkway Mountain View, CA 94043, USA
 # Telephone: +1 (650) 253-0000
 # Website: www.google.com
@@ -2,7 +2,7 @@
 # Script: Beng
 # Version: 1.0
 # Effective Date: 04-12-2012
-# Contact: tas-contact.google.com
+# Contact: iana-contact@google.com
 # Address: 1600 Amphitheatre Parkway Mountain View, CA 94043, USA
 # Telephone: +1 (650) 253-0000
 # Website: www.google.com
@@ -1,24 +1,24 @@
 # Registry: Charleston Road Registry Inc.
-# Script: Chinese
+# Script: zh-Hans
 # Version: 1.0
 # Effective Date: 04-12-2012
 #
-# Contact: tas-contact.google.com
+# Contact: iana-contact@google.com
 # Address: 1600 Amphitheatre Parkway Mountain View, CA 94043, USA
 # Telephone: +1 (650) 253-0000
 # Website: www.google.com
 # Notes: This table describes codepoints allowed for the Chinese script.

-Reference 0 Unicode 3.2
-Reference 1 A Complete Set of Simplified Chinese Characters
-Reference 2 Chinese Variants Collation Table
-Reference 3 Chinese Big Dictionary
-Reference 4 Chinese Relationship Table for Unihan Project
-Reference 5 GB2312
-Reference 6 General Table for Modern Chinese
-Reference 7 International Chinese Standard Big Dictionary
-Reference 8 Unihan Database
-Reference 9 BIG5
+# Reference 0 Unicode 3.2
+# Reference 1 A Complete Set of Simplified Chinese Characters
+# Reference 2 Chinese Variants Collation Table
+# Reference 3 Chinese Big Dictionary
+# Reference 4 Chinese Relationship Table for Unihan Project
+# Reference 5 GB2312
+# Reference 6 General Table for Modern Chinese
+# Reference 7 International Chinese Standard Big Dictionary
+# Reference 8 Unihan Database
+# Reference 9 BIG5

 U+002D(0);U+002D(0);
 U+0030(0);U+0030(0);
@@ -2,7 +2,7 @@
 # Script: Cyrl
 # Version: 1.0
 # Effective Date: 04-12-2012
-# Contact: tas-contact.google.com
+# Contact: iana-contact@google.com
 # Address: 1600 Amphitheatre Parkway Mountain View, CA 94043, USA
 # Telephone: +1 (650) 253-0000
 # Website: www.google.com
@@ -2,7 +2,7 @@
 # Script: Deva
 # Version: 1.0
 # Effective Date: 04-12-2012
-# Contact: tas-contact.google.com
+# Contact: iana-contact@google.com
 # Address: 1600 Amphitheatre Parkway Mountain View, CA 94043, USA
 # Telephone: +1 (650) 253-0000
 # Website: www.google.com
@@ -2,7 +2,7 @@
 # Script: Ethi
 # Version: 1.0
 # Effective Date: 04-12-2012
-# Contact: tas-contact.google.com
+# Contact: iana-contact@google.com
 # Address: 1600 Amphitheatre Parkway Mountain View, CA 94043, USA
 # Telephone: +1 (650) 253-0000
 # Website: www.google.com
@@ -3,7 +3,7 @@
 #
 # Version: 1.0
 # Effective Date: 04-12-2012
-# Contact: tas-contact.google.com
+# Contact: iana-contact@google.com
 # Address: 1600 Amphitheatre Parkway Mountain View, CA 94043, USA
 # Telephone: +1 (650) 253-0000
 # Website: www.google.com
@@ -3,7 +3,7 @@
 # Script: Grek
 # Version: 1.0
 # Effective Date: 04-12-2012
-# Contact: tas-contact.google.com
+# Contact: iana-contact@google.com
 # Address: 1600 Amphitheatre Parkway Mountain View, CA 94043, USA
 # Telephone: +1 (650) 253-0000
 # Website: www.google.com
@@ -3,7 +3,7 @@
 # Script: Guru
 # Version: 1.0
 # Effective Date: 04-12-2012
-# Contact: tas-contact.google.com
+# Contact: iana-contact@google.com
 # Address: 1600 Amphitheatre Parkway Mountain View, CA 94043, USA
 # Telephone: +1 (650) 253-0000
 # Website: www.google.com
@@ -2,7 +2,7 @@
 # Script: Hebr
 # Version: 1.0
 # Effective Date: 04-12-2012
-# Contact: tas-contact.google.com
+# Contact: iana-contact@google.com
 # Address: 1600 Amphitheatre Parkway Mountain View, CA 94043, USA
 # Telephone: +1 (650) 253-0000
 # Website: www.google.com
@@ -2,7 +2,7 @@
 # Script: Knda
 # Version: 1.0
 # Effective Date: 04-12-2012
-# Contact: tas-contact.google.com
+# Contact: iana-contact@google.com
 # Address: 1600 Amphitheatre Parkway Mountain View, CA 94043, USA
 # Telephone: +1 (650) 253-0000
 # Website: www.google.com
@@ -2,7 +2,7 @@
 # Script: Khmr
 # Version: 1.0
 # Effective Date: 04-12-2012
-# Contact: tas-contact.google.com
+# Contact: iana-contact@google.com
 # Address: 1600 Amphitheatre Parkway Mountain View, CA 94043, USA
 #
 # Telephone: +1 (650) 253-0000
@@ -2,7 +2,7 @@
 # Script: Kore
 # Version: 1.0
 # Effective Date: 04-12-2012
-# Contact: tas-contact.google.com
+# Contact: iana-contact@google.com
 # Address: 1600 Amphitheatre Parkway Mountain View, CA 94043, USA
 # Telephone: +1 (650) 253-0000
 # Website: www.google.com
@@ -2,7 +2,7 @@
 # Script: Mlym
 # Version: 1.0
 # Effective Date: 04-12-2012
-# Contact: tas-contact.google.com
+# Contact: iana-contact@google.com
 # Address: 1600 Amphitheatre Parkway Mountain View, CA 94043, USA
 # Telephone: +1 (650) 253-0000
 # Website: www.google.com
@@ -3,7 +3,7 @@
 # Version: 1.0
 # Effective Date: 04-12-2012
 #
-# Contact: tas-contact.google.com
+# Contact: iana-contact@google.com
 # Address: 1600 Amphitheatre Parkway Mountain View, CA 94043, USA
 # Telephone: +1 (650) 253-0000
 # Website: www.google.com
@@ -2,7 +2,7 @@
 # Script: Orya
 # Version: 1.0
 # Effective Date: 04-12-2012
-# Contact: tas-contact.google.com
+# Contact: iana-contact@google.com
 # Address: 1600 Amphitheatre Parkway Mountain View, CA 94043, USA
 # Telephone: +1 (650) 253-0000
 # Website: www.google.com
@@ -2,7 +2,7 @@
 # Script: Sinh
 # Version: 1.0
 # Effective Date: 04-12-2012
-# Contact: tas-contact.google.com
+# Contact: iana-contact@google.com
 # Address: 1600 Amphitheatre Parkway Mountain View, CA 94043, USA
 # Telephone: +1 (650) 253-0000
 # Website: www.google.com
@@ -2,7 +2,7 @@
 # Script: Taml
 # Version: 1.0
 # Effective Date: 04-12-2012
-# Contact: tas-contact.google.com
+# Contact: iana-contact@google.com
 # Address: 1600 Amphitheatre Parkway Mountain View, CA 94043, USA
 # Telephone: +1 (650) 253-0000
 # Website: www.google.com
@@ -2,7 +2,7 @@
 # Script: Telu
 # Version: 1.0
 # Effective Date: 04-12-2012
-# Contact: tas-contact.google.com
+# Contact: iana-contact@google.com
 # Address: 1600 Amphitheatre Parkway Mountain View, CA 94043, USA
 # Telephone: +1 (650) 253-0000
 # Website: www.google.com
@@ -2,7 +2,7 @@
 # Script: Thai
 # Version: 1.0
 # Effective Date: 04-12-2012
-# Contact: tas-contact.google.com
+# Contact: iana-contact@google.com
 # Address: 1600 Amphitheatre Parkway Mountain View, CA 94043, USA
 # Telephone: +1 (650) 253-0000
 # Website: www.google.com
@@ -2,7 +2,7 @@
 # Script: Tibt
 # Version: 1.0
 # Effective Date: 04-12-2012
-# Contact: tas-contact.google.com
+# Contact: iana-contact@google.com
 # Address: 1600 Amphitheatre Parkway Mountain View, CA 94043, USA
 # Telephone: +1 (650) 253-0000
 # Website: www.google.com
@@ -1,23 +1,23 @@
 # Registry: Charleston Road Registry Inc.
-# Script: Traditional Chinese
+# Script: zh-Hant
 # Version: 1.0
 # Effective Date: 04-12-2012
-# Contact: tas-contact.google.com
+# Contact: iana-contact@google.com
 # Address: 1600 Amphitheatre Parkway Mountain View, CA 94043, USA
 # Telephone: +1 (650) 253-0000
 # Website: www.google.com
 # Notes: This table describes codepoints allowed for the Traditional Chinese script.

-Reference 0 Unicode 3.2
-Reference 1 A Complete Set of Simplified Chinese Characters
-Reference 2 Chinese Variants Collation Table
-Reference 3 Chinese Big Dictionary
-Reference 4 Chinese Relationship Table for Unihan Project
-Reference 5 GB2312
-Reference 6 General Table for Modern Chinese
-Reference 7 International Chinese Standard Big Dictionary
-Reference 8 Unihan Database
-Reference 9 BIG5
+# Reference 0 Unicode 3.2
+# Reference 1 A Complete Set of Simplified Chinese Characters
+# Reference 2 Chinese Variants Collation Table
+# Reference 3 Chinese Big Dictionary
+# Reference 4 Chinese Relationship Table for Unihan Project
+# Reference 5 GB2312
+# Reference 6 General Table for Modern Chinese
+# Reference 7 International Chinese Standard Big Dictionary
+# Reference 8 Unihan Database
+# Reference 9 BIG5

 U+002D(0);U+002D(0);
 U+0030(0);U+0030(0);
@@ -62,23 +62,32 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
  /**
   * Unique identifier in the registry for this resource.
   *
+   * <p>Not persisted so that we can store these in references to other objects. Subclasses that
+   * wish to use this as the primary key should create a getter method annotated with @Id
+   *
   * <p>This is in the (\w|_){1,80}-\w{1,8} format specified by RFC 5730 for roidType.
   *
   * @see <a href="https://tools.ietf.org/html/rfc5730">RFC 5730</a>
   */
-  @Id
-  // not persisted so that we can store these in references to other objects. Subclasses that wish
-  // to use this as the primary key should create a getter method annotated with @Id
-  @Transient
-  String repoId;
+  @Id @Transient String repoId;

-  /** The ID of the registrar that is currently sponsoring this resource. */
+  /**
+   * The ID of the registrar that is currently sponsoring this resource.
+   *
+   * <p>This can be null in the case of pre-Registry-3.0-migration history objects with null
+   * resource fields.
+   */
  @Index
-  @Column(name = "currentSponsorRegistrarId", nullable = false)
+  @Column(name = "currentSponsorRegistrarId")
  String currentSponsorClientId;

-  /** The ID of the registrar that created this resource. */
-  @Column(name = "creationRegistrarId", nullable = false)
+  /**
+   * The ID of the registrar that created this resource.
+   *
+   * <p>This can be null in the case of pre-Registry-3.0-migration history objects with null
+   * resource fields.
+   */
+  @Column(name = "creationRegistrarId")
  String creationClientId;

  /**
@@ -91,13 +100,17 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
  @Column(name = "lastEppUpdateRegistrarId")
  String lastEppUpdateClientId;

-  /** The time when this resource was created. */
-  // Map the method to XML, not the field, because if we map the field (with an adaptor class) it
-  // will never be omitted from the xml even if the timestamp inside creationTime is null and we
-  // return null from the adaptor. (Instead it gets written as an empty tag.)
-  @Column(nullable = false)
-  @Index
-  CreateAutoTimestamp creationTime = CreateAutoTimestamp.create(null);
+  /**
+   * The time when this resource was created.
+   *
+   * <p>Map the method to XML, not the field, because if we map the field (with an adaptor class) it
+   * will never be omitted from the xml even if the timestamp inside creationTime is null and we
+   * return null from the adaptor (instead it gets written as an empty tag).
+   *
+   * <p>This can be null in the case of pre-Registry-3.0-migration history objects with null
+   * resource fields.
+   */
+  @Index CreateAutoTimestamp creationTime = CreateAutoTimestamp.create(null);

  /**
   * The time when this resource was or will be deleted.
@@ -112,8 +125,7 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
   * out of the index at that time, as long as we query for resources whose deletion time is before
   * now.
   */
-  @Index
-  DateTime deletionTime;
+  @Index DateTime deletionTime;

  /**
   * The time that this resource was last updated.
@@ -144,7 +156,7 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
    return repoId;
  }

-  // Hibernate needs this to populate the repo ID, but no one else should ever use it
+  /** This method exists solely to satisfy Hibernate. Use {@link Builder} instead. */
  @SuppressWarnings("UnusedMethod")
  private void setRepoId(String repoId) {
    this.repoId = repoId;
@@ -252,7 +252,8 @@ public final class OteAccountBuilder {
  private void saveAllEntities() {
    tm().assertInTransaction();

-    ImmutableList<Registry> registries = ImmutableList.of(sunriseTld, gaTld, eapTld);
+    // use ImmutableObject instead of Registry so that the Key generation doesn't break
+    ImmutableList<ImmutableObject> registries = ImmutableList.of(sunriseTld, gaTld, eapTld);
    ImmutableList<RegistrarContact> contacts = contactsBuilder.build();

    if (!replaceExisting) {
@@ -33,12 +33,14 @@ import com.googlecode.objectify.annotation.Id;
 import com.googlecode.objectify.annotation.Ignore;
 import com.googlecode.objectify.annotation.IgnoreSave;
 import com.googlecode.objectify.annotation.Index;
+import com.googlecode.objectify.annotation.OnLoad;
 import com.googlecode.objectify.annotation.Parent;
 import com.googlecode.objectify.condition.IfNull;
 import google.registry.model.Buildable;
 import google.registry.model.ImmutableObject;
 import google.registry.model.annotations.ReportedOn;
 import google.registry.model.common.TimeOfYear;
+import google.registry.model.domain.DomainBase;
 import google.registry.model.domain.GracePeriod;
 import google.registry.model.domain.rgp.GracePeriodStatus;
 import google.registry.model.domain.token.AllocationToken;
@@ -57,16 +59,14 @@ import javax.persistence.Column;
 import javax.persistence.Embedded;
 import javax.persistence.EnumType;
 import javax.persistence.Enumerated;
-import javax.persistence.GeneratedValue;
-import javax.persistence.GenerationType;
 import javax.persistence.MappedSuperclass;
+import javax.persistence.PostLoad;
 import javax.persistence.Transient;
 import org.joda.money.Money;
 import org.joda.time.DateTime;

 /** A billable event in a domain's lifecycle. */
@MappedSuperclass
-@WithLongVKey
 public abstract class BillingEvent extends ImmutableObject
    implements Buildable, TransferServerApproveEntity {

@@ -108,10 +108,7 @@ public abstract class BillingEvent extends ImmutableObject
  }

  /** Entity id. */
-  @Id
-  @javax.persistence.Id
-  @GeneratedValue(strategy = GenerationType.IDENTITY)
-  Long id;
+  @Id @javax.persistence.Id Long id;

  @Parent @DoNotHydrate @Transient Key<HistoryEntry> parent;

@@ -149,6 +146,21 @@ public abstract class BillingEvent extends ImmutableObject
  @Nullable
  Set<Flag> flags;

+  @PostLoad
+  void postLoad() {
+    parent =
+        Key.create(
+            Key.create(DomainBase.class, domainRepoId),
+            HistoryEntry.class,
+            domainHistoryRevisionId);
+  }
+
+  @OnLoad
+  void onLoad() {
+    domainHistoryRevisionId = parent.getId();
+    domainRepoId = parent.getParent().getName();
+  }
+
  public String getClientId() {
    return clientId;
  }
@@ -245,7 +257,6 @@ public abstract class BillingEvent extends ImmutableObject
    }

    public B setParent(Key<HistoryEntry> parentKey) {
-      // TODO(shicong): Figure out how to set domainHistoryRevisionId and domainRepoId
      getInstance().parent = parentKey;
      return thisCastToDerived();
    }
@@ -258,6 +269,11 @@ public abstract class BillingEvent extends ImmutableObject
      checkNotNull(instance.eventTime, "Event time must be set");
      checkNotNull(instance.targetId, "Target ID must be set");
      checkNotNull(instance.parent, "Parent must be set");
+      checkNotNull(instance.parent.getParent(), "parent.getParent() must be set");
+      checkNotNull(
+          instance.parent.getParent().getName(), "parent.getParent().getName() must be set");
+      instance.domainHistoryRevisionId = instance.parent.getId();
+      instance.domainRepoId = instance.parent.getParent().getName();
      return super.build();
    }
  }
@@ -275,6 +291,7 @@ public abstract class BillingEvent extends ImmutableObject
        @javax.persistence.Index(columnList = "allocation_token_id")
      })
  @AttributeOverride(name = "id", column = @Column(name = "billing_event_id"))
+  @WithLongVKey
  public static class OneTime extends BillingEvent implements DatastoreAndSqlEntity {

    /** The billable value. */
@@ -310,7 +327,7 @@ public abstract class BillingEvent extends ImmutableObject
     * Cancellation}s.
     */
    @Column(name = "cancellation_matching_billing_recurrence_id")
-    VKey<? extends BillingEvent> cancellationMatchingBillingEvent;
+    VKey<Recurring> cancellationMatchingBillingEvent;

    /**
     * The {@link AllocationToken} used in the creation of this event, or null if one was not used.
@@ -392,7 +409,7 @@ public abstract class BillingEvent extends ImmutableObject
      }

      public Builder setCancellationMatchingBillingEvent(
-          VKey<? extends BillingEvent> cancellationMatchingBillingEvent) {
+          VKey<Recurring> cancellationMatchingBillingEvent) {
        getInstance().cancellationMatchingBillingEvent = cancellationMatchingBillingEvent;
        return this;
      }
@@ -451,6 +468,7 @@ public abstract class BillingEvent extends ImmutableObject
        @javax.persistence.Index(columnList = "recurrence_time_of_year")
      })
  @AttributeOverride(name = "id", column = @Column(name = "billing_recurrence_id"))
+  @WithLongVKey
  public static class Recurring extends BillingEvent implements DatastoreAndSqlEntity {

    /**
@@ -545,6 +563,7 @@ public abstract class BillingEvent extends ImmutableObject
        @javax.persistence.Index(columnList = "billingTime")
      })
  @AttributeOverride(name = "id", column = @Column(name = "billing_cancellation_id"))
+  @WithLongVKey
  public static class Cancellation extends BillingEvent implements DatastoreAndSqlEntity {

    /** The billing time of the charge that is being cancelled. */
@@ -665,6 +684,7 @@ public abstract class BillingEvent extends ImmutableObject
  /** An event representing a modification of an existing one-time billing event. */
  @ReportedOn
  @Entity
+  @WithLongVKey
  public static class Modification extends BillingEvent implements DatastoreAndSqlEntity {

    /** The change in cost that should be applied to the original billing event. */
@@ -14,18 +14,25 @@

 package google.registry.model.contact;

+import com.google.common.collect.ImmutableList;
 import com.googlecode.objectify.Key;
 import com.googlecode.objectify.annotation.EntitySubclass;
-import google.registry.model.EppResource;
+import google.registry.model.ImmutableObject;
+import google.registry.model.contact.ContactHistory.ContactHistoryId;
 import google.registry.model.reporting.HistoryEntry;
 import google.registry.persistence.VKey;
+import google.registry.schema.replay.DatastoreEntity;
+import google.registry.schema.replay.SqlEntity;
+import java.io.Serializable;
+import java.util.Optional;
+import javax.annotation.Nullable;
 import javax.persistence.Access;
 import javax.persistence.AccessType;
 import javax.persistence.Column;
 import javax.persistence.Entity;
-import javax.persistence.GeneratedValue;
-import javax.persistence.GenerationType;
 import javax.persistence.Id;
+import javax.persistence.IdClass;
+import javax.persistence.PostLoad;

 /**
 * A persisted history entry representing an EPP modification to a contact.
@@ -44,16 +51,25 @@ import javax.persistence.Id;
    })
@EntitySubclass
@Access(AccessType.FIELD)
-public class ContactHistory extends HistoryEntry {
+@IdClass(ContactHistoryId.class)
+public class ContactHistory extends HistoryEntry implements SqlEntity {

  // Store ContactBase instead of ContactResource so we don't pick up its @Id
-  ContactBase contactBase;
-
-  @Column(nullable = false)
-  VKey<ContactResource> contactRepoId;
+  @Nullable ContactBase contactBase;
+
+  @Id
+  @Access(AccessType.PROPERTY)
+  public String getContactRepoId() {
+    return parent.getName();
+  }
+
+  /** This method is private because it is only used by Hibernate. */
+  @SuppressWarnings("unused")
+  private void setContactRepoId(String contactRepoId) {
+    parent = Key.create(ContactResource.class, contactRepoId);
+  }

  @Id
-  @GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "HistorySequenceGenerator")
  @Column(name = "historyRevisionId")
  @Access(AccessType.PROPERTY)
  @Override
@@ -61,14 +77,98 @@ public class ContactHistory extends HistoryEntry {
    return super.getId();
  }

-  /** The state of the {@link ContactBase} object at this point in time. */
-  public ContactBase getContactBase() {
-    return contactBase;
+  /**
+   * The values of all the fields on the {@link ContactBase} object after the action represented by
+   * this history object was executed.
+   *
+   * <p>Will be absent for objects created prior to the Registry 3.0 SQL migration.
+   */
+  public Optional<ContactBase> getContactBase() {
+    return Optional.ofNullable(contactBase);
  }

  /** The key to the {@link ContactResource} this is based off of. */
-  public VKey<ContactResource> getContactRepoId() {
-    return contactRepoId;
+  public VKey<ContactResource> getParentVKey() {
+    return VKey.create(ContactResource.class, getContactRepoId());
+  }
+
+  /** Creates a {@link VKey} instance for this entity. */
+  public VKey<ContactHistory> createVKey() {
+    return VKey.create(
+        ContactHistory.class, new ContactHistoryId(getContactRepoId(), getId()), Key.create(this));
+  }
+
+  @PostLoad
+  void postLoad() {
+    // Normally Hibernate would see that the contact fields are all null and would fill contactBase
+    // with a null object. Unfortunately, the updateTimestamp is never null in SQL.
+    if (contactBase != null && contactBase.getContactId() == null) {
+      contactBase = null;
+    }
+  }
+
+  // In Datastore, save as a HistoryEntry object regardless of this object's type
+  @Override
+  public ImmutableList<DatastoreEntity> toDatastoreEntities() {
+    return ImmutableList.of(asHistoryEntry());
+  }
+
+  /** Class to represent the composite primary key of {@link ContactHistory} entity. */
+  static class ContactHistoryId extends ImmutableObject implements Serializable {
+
+    private String contactRepoId;
+
+    private Long id;
+
+    /** Hibernate requires this default constructor. */
+    private ContactHistoryId() {}
+
+    ContactHistoryId(String contactRepoId, long id) {
+      this.contactRepoId = contactRepoId;
+      this.id = id;
+    }
+
+    /**
+     * Returns the contact repository id.
+     *
+     * <p>This method is private because it is only used by Hibernate.
+     */
+    @SuppressWarnings("unused")
+    private String getContactRepoId() {
+      return contactRepoId;
+    }
+
+    /**
+     * Returns the history revision id.
+     *
+     * <p>This method is private because it is only used by Hibernate.
+     */
+    @SuppressWarnings("unused")
+    private long getId() {
+      return id;
+    }
+
+    /**
+     * Sets the contact repository id.
+     *
+     * <p>This method is private because it is only used by Hibernate and should not be used
+     * externally to keep immutability.
+     */
+    @SuppressWarnings("unused")
+    private void setContactRepoId(String contactRepoId) {
+      this.contactRepoId = contactRepoId;
+    }
+
+    /**
+     * Sets the history revision id.
+     *
+     * <p>This method is private because it is only used by Hibernate and should not be used
+     * externally to keep immutability.
+     */
+    @SuppressWarnings("unused")
+    private void setId(long id) {
+      this.id = id;
+    }
  }

  @Override
@@ -89,18 +189,8 @@ public class ContactHistory extends HistoryEntry {
      return this;
    }

-    public Builder setContactRepoId(VKey<ContactResource> contactRepoId) {
-      getInstance().contactRepoId = contactRepoId;
-      contactRepoId.maybeGetOfyKey().ifPresent(parent -> getInstance().parent = parent);
-      return this;
-    }
-
-    // We can remove this once all HistoryEntries are converted to History objects
-    @Override
-    public Builder setParent(Key<? extends EppResource> parent) {
-      super.setParent(parent);
-      getInstance().contactRepoId =
-          VKey.create(ContactResource.class, parent.getName(), (Key<ContactResource>) parent);
+    public Builder setContactRepoId(String contactRepoId) {
+      getInstance().parent = Key.create(ContactResource.class, contactRepoId);
      return this;
    }
  }
@@ -51,7 +51,6 @@ public class ContactResource extends ContactBase

  @Override
  public VKey<ContactResource> createVKey() {
-    // TODO(mmuller): create symmetric keys if we can ever reload both sides.
    return VKey.create(ContactResource.class, getRepoId(), Key.create(this));
  }

@@ -19,6 +19,7 @@ import google.registry.model.EppResource;
 import google.registry.model.EppResource.ForeignKeyedEppResource;
 import google.registry.model.annotations.ExternalMessagingName;
 import google.registry.model.annotations.ReportedOn;
+import google.registry.model.domain.secdns.DelegationSignerData;
 import google.registry.model.host.HostResource;
 import google.registry.persistence.VKey;
 import google.registry.persistence.WithStringVKey;
@@ -106,6 +107,32 @@ public class DomainBase extends DomainContent
    return gracePeriods;
  }

+  /**
+   * Returns the set of {@link DelegationSignerData} associated with the domain.
+   *
+   * <p>This is the getter method specific for Hibernate to access the field so it is set to
+   * private. The caller can use the public {@link #getDsData()} to get the DS data.
+   *
+   * <p>Note that we need to set `insertable = false, updatable = false` for @JoinColumn, otherwise
+   * Hibernate would try to set the foreign key to null(through an UPDATE TABLE sql) instead of
+   * deleting the whole entry from the table when the {@link DelegationSignerData} is removed from
+   * the set.
+   */
+  @Access(AccessType.PROPERTY)
+  @OneToMany(
+      cascade = {CascadeType.ALL},
+      fetch = FetchType.EAGER,
+      orphanRemoval = true)
+  @JoinColumn(
+      name = "domainRepoId",
+      referencedColumnName = "repoId",
+      insertable = false,
+      updatable = false)
+  @SuppressWarnings("UnusedMethod")
+  private Set<DelegationSignerData> getInternalDelegationSignerData() {
+    return dsData;
+  }
+
  @Override
  public VKey<DomainBase> createVKey() {
    return VKey.create(DomainBase.class, getRepoId(), Key.create(this));
@@ -35,11 +35,13 @@ import static google.registry.util.DomainNameUtils.canonicalizeDomainName;
 import static google.registry.util.DomainNameUtils.getTldFromDomainName;
 import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;

+import com.google.common.base.Predicate;
 import com.google.common.collect.ImmutableSet;
 import com.google.common.collect.ImmutableSortedSet;
 import com.google.common.collect.Ordering;
 import com.google.common.collect.Sets;
 import com.google.common.collect.Streams;
+import com.googlecode.objectify.Key;
 import com.googlecode.objectify.annotation.Ignore;
 import com.googlecode.objectify.annotation.IgnoreSave;
 import com.googlecode.objectify.annotation.Index;
@@ -49,6 +51,7 @@ import google.registry.flows.ResourceFlowUtils;
 import google.registry.model.EppResource;
 import google.registry.model.EppResource.ResourceWithTransferData;
 import google.registry.model.billing.BillingEvent;
+import google.registry.model.common.EntityGroupRoot;
 import google.registry.model.contact.ContactResource;
 import google.registry.model.domain.launch.LaunchNotice;
 import google.registry.model.domain.rgp.GracePeriodStatus;
@@ -57,6 +60,7 @@ import google.registry.model.eppcommon.StatusValue;
 import google.registry.model.host.HostResource;
 import google.registry.model.poll.PollMessage;
 import google.registry.model.registry.Registry;
+import google.registry.model.reporting.HistoryEntry;
 import google.registry.model.transfer.DomainTransferData;
 import google.registry.model.transfer.TransferStatus;
 import google.registry.persistence.VKey;
@@ -65,7 +69,6 @@ import java.util.HashSet;
 import java.util.Objects;
 import java.util.Optional;
 import java.util.Set;
-import java.util.function.Predicate;
 import javax.annotation.Nullable;
 import javax.persistence.Access;
 import javax.persistence.AccessType;
@@ -209,6 +212,15 @@ public class DomainContent extends EppResource
  @Column(name = "deletion_poll_message_id")
  VKey<PollMessage.OneTime> deletePollMessage;

+  /**
+   * History record for the delete poll message.
+   *
+   * <p>Here so we can restore the original ofy key from sql.
+   */
+  @Column(name = "deletion_poll_message_history_id")
+  @Ignore
+  Long deletePollMessageHistoryId;
+
  /**
   * The recurring billing event associated with this domain's autorenewals.
   *
@@ -220,6 +232,15 @@ public class DomainContent extends EppResource
  @Column(name = "billing_recurrence_id")
  VKey<BillingEvent.Recurring> autorenewBillingEvent;

+  /**
+   * History record for the autorenew billing event.
+   *
+   * <p>Here so we can restore the original ofy key from sql.
+   */
+  @Column(name = "billing_recurrence_history_id")
+  @Ignore
+  Long autorenewBillingEventHistoryId;
+
  /**
   * The recurring poll message associated with this domain's autorenewals.
   *
@@ -231,6 +252,13 @@ public class DomainContent extends EppResource
  @Column(name = "autorenew_poll_message_id")
  VKey<PollMessage.Autorenew> autorenewPollMessage;

+  /**
+   * History record for the autorenew poll message.
+   *
+   * <p>Here so we can restore the original ofy key from sql.
+   */
+  @Ignore Long autorenewPollMessageHistoryId;
+
  /** The unexpired grace periods for this domain (some of which may not be active yet). */
  @Transient Set<GracePeriod> gracePeriods;

@@ -284,10 +312,20 @@ public class DomainContent extends EppResource
        nullToEmptyImmutableCopy(gracePeriods).stream()
            .map(gracePeriod -> gracePeriod.cloneWithDomainRepoId(getRepoId()))
            .collect(toImmutableSet());
+
+    // Restore history record ids.
+    autorenewPollMessageHistoryId = getHistoryId(autorenewPollMessage);
+    autorenewBillingEventHistoryId = getHistoryId(autorenewBillingEvent);
+    deletePollMessageHistoryId = getHistoryId(deletePollMessage);
+    dsData =
+        nullToEmptyImmutableCopy(dsData).stream()
+            .map(dsData -> dsData.cloneWithDomainRepoId(getRepoId()))
+            .collect(toImmutableSet());
  }

  @PostLoad
-  void postLoad() {
+  @SuppressWarnings("UnusedMethod")
+  private final void postLoad() {
    // Reconstitute the contact list.
    ImmutableSet.Builder<DesignatedContact> contactsBuilder = new ImmutableSet.Builder<>();

@@ -306,6 +344,23 @@ public class DomainContent extends EppResource
    }

    allContacts = contactsBuilder.build();
+
+    // Reconstitute the composite ofy keys from the SQL data.
+    Key<DomainBase> myKey = Key.create(DomainBase.class, getRepoId());
+    deletePollMessage = restoreOfyFrom(myKey, deletePollMessage, deletePollMessageHistoryId);
+    autorenewBillingEvent =
+        restoreOfyFrom(myKey, autorenewBillingEvent, autorenewBillingEventHistoryId);
+    autorenewPollMessage =
+        restoreOfyFrom(myKey, autorenewPollMessage, autorenewPollMessageHistoryId);
+  }
+
+  private <T> VKey<T> restoreOfyFrom(Key<DomainBase> domainKey, VKey<T> key, Long historyId) {
+    if (historyId == null) {
+      // This is a legacy key (or a null key, in which case this works too)
+      return VKey.restoreOfyFrom(key, EntityGroupRoot.class, "per-tld");
+    } else {
+      return VKey.restoreOfyFrom(key, domainKey, HistoryEntry.class, historyId);
+    }
  }

  public ImmutableSet<String> getSubordinateHosts() {
@@ -387,6 +442,12 @@ public class DomainContent extends EppResource
    this.gracePeriods = gracePeriods;
  }

+  // Hibernate needs this in order to populate dsData but no one else should ever use it
+  @SuppressWarnings("UnusedMethod")
+  private void setInternalDelegationSignerData(Set<DelegationSignerData> dsData) {
+    this.dsData = dsData;
+  }
+
  public final String getCurrentSponsorClientId() {
    return getPersistedCurrentSponsorClientId();
  }
@@ -655,13 +716,24 @@ public class DomainContent extends EppResource
            + " use DomainBase instead");
  }

+  private static Long getHistoryId(VKey<?> key) {
+    if (key == null) {
+      return null;
+    }
+    Key<?> parent = key.getOfyKey().getParent();
+    if (parent == null || parent.getKind().equals("EntityGroupRoot")) {
+      return null;
+    }
+    return parent.getId();
+  }
+
  /** Predicate to determine if a given {@link DesignatedContact} is the registrant. */
  static final Predicate<DesignatedContact> IS_REGISTRANT =
      (DesignatedContact contact) -> DesignatedContact.Type.REGISTRANT.equals(contact.type);

  /** An override of {@link EppResource#asBuilder} with tighter typing. */
  @Override
-  public Builder asBuilder() {
+  public Builder<? extends DomainContent, ?> asBuilder() {
    return new Builder<>(clone(this));
  }

@@ -698,10 +770,16 @@ public class DomainContent extends EppResource
      instance.tld = getTldFromDomainName(instance.fullyQualifiedDomainName);

      T newDomain = super.build();
-      // Hibernate throws exception if gracePeriods is null because we enabled all cascadable
-      // operations and orphan removal.
+      // Hibernate throws exception if gracePeriods or dsData is null because we enabled all
+      // cascadable operations and orphan removal.
      newDomain.gracePeriods =
          newDomain.gracePeriods == null ? ImmutableSet.of() : newDomain.gracePeriods;
+      newDomain.dsData =
+          newDomain.dsData == null
+              ? ImmutableSet.of()
+              : newDomain.dsData.stream()
+                  .map(ds -> ds.cloneWithDomainRepoId(instance.getRepoId()))
+                  .collect(toImmutableSet());
      return newDomain;
    }

@@ -824,16 +902,19 @@ public class DomainContent extends EppResource

    public B setDeletePollMessage(VKey<PollMessage.OneTime> deletePollMessage) {
      getInstance().deletePollMessage = deletePollMessage;
+      getInstance().deletePollMessageHistoryId = getHistoryId(deletePollMessage);
      return thisCastToDerived();
    }

    public B setAutorenewBillingEvent(VKey<BillingEvent.Recurring> autorenewBillingEvent) {
      getInstance().autorenewBillingEvent = autorenewBillingEvent;
+      getInstance().autorenewBillingEventHistoryId = getHistoryId(autorenewBillingEvent);
      return thisCastToDerived();
    }

    public B setAutorenewPollMessage(VKey<PollMessage.Autorenew> autorenewPollMessage) {
      getInstance().autorenewPollMessage = autorenewPollMessage;
+      getInstance().autorenewPollMessageHistoryId = getHistoryId(autorenewPollMessage);
      return thisCastToDerived();
    }

@@ -16,28 +16,36 @@ package google.registry.model.domain;

 import static google.registry.util.CollectionUtils.nullToEmptyImmutableCopy;

+import com.google.common.collect.ImmutableList;
 import com.googlecode.objectify.Key;
 import com.googlecode.objectify.annotation.EntitySubclass;
 import com.googlecode.objectify.annotation.Ignore;
-import google.registry.model.EppResource;
 import google.registry.model.ImmutableObject;
 import google.registry.model.domain.DomainHistory.DomainHistoryId;
 import google.registry.model.host.HostResource;
+import google.registry.model.reporting.DomainTransactionRecord;
 import google.registry.model.reporting.HistoryEntry;
 import google.registry.persistence.VKey;
+import google.registry.schema.replay.DatastoreEntity;
+import google.registry.schema.replay.SqlEntity;
 import java.io.Serializable;
+import java.util.Optional;
 import java.util.Set;
+import javax.annotation.Nullable;
 import javax.persistence.Access;
 import javax.persistence.AccessType;
+import javax.persistence.AttributeOverride;
+import javax.persistence.AttributeOverrides;
+import javax.persistence.CascadeType;
 import javax.persistence.Column;
 import javax.persistence.ElementCollection;
 import javax.persistence.Entity;
-import javax.persistence.GeneratedValue;
-import javax.persistence.GenerationType;
 import javax.persistence.Id;
 import javax.persistence.IdClass;
 import javax.persistence.Index;
+import javax.persistence.JoinColumn;
 import javax.persistence.JoinTable;
+import javax.persistence.OneToMany;
 import javax.persistence.PostLoad;
 import javax.persistence.Table;

@@ -59,12 +67,22 @@ import javax.persistence.Table;
@EntitySubclass
@Access(AccessType.FIELD)
@IdClass(DomainHistoryId.class)
-public class DomainHistory extends HistoryEntry {
+public class DomainHistory extends HistoryEntry implements SqlEntity {

  // Store DomainContent instead of DomainBase so we don't pick up its @Id
-  DomainContent domainContent;
+  @Nullable DomainContent domainContent;

-  @Id String domainRepoId;
+  @Id
+  @Access(AccessType.PROPERTY)
+  public String getDomainRepoId() {
+    return parent.getName();
+  }
+
+  /** This method is private because it is only used by Hibernate. */
+  @SuppressWarnings("unused")
+  private void setDomainRepoId(String domainRepoId) {
+    parent = Key.create(DomainBase.class, domainRepoId);
+  }

  // We could have reused domainContent.nsHosts here, but Hibernate throws a weird exception after
  // we change to use a composite primary key.
@@ -75,8 +93,51 @@ public class DomainHistory extends HistoryEntry {
  @Column(name = "host_repo_id")
  Set<VKey<HostResource>> nsHosts;

+  @Override
+  @Nullable
+  @Access(AccessType.PROPERTY)
+  @AttributeOverrides({
+      @AttributeOverride(
+          name = "unit",
+          column = @Column(name = "historyPeriodUnit")),
+      @AttributeOverride(
+          name = "value",
+          column = @Column(name = "historyPeriodValue"))
+  })
+  public Period getPeriod() {
+    return super.getPeriod();
+  }
+
+  /**
+   * For transfers, the id of the other registrar.
+   *
+   * <p>For requests and cancels, the other registrar is the losing party (because the registrar
+   * sending the EPP transfer command is the gaining party). For approves and rejects, the other
+   * registrar is the gaining party.
+   */
+  @Nullable
+  @Access(AccessType.PROPERTY)
+  @Column(name = "historyOtherRegistrarId")
+  public String getOtherRegistrarId() {
+    return super.getOtherClientId();
+  }
+
+  /**
+   * Logging field for transaction reporting.
+   *
+   * <p>This will be empty for any DomainHistory/HistoryEntry generated before this field was added,
+   * mid-2017, as well as any action that does not generate billable events (e.g. updates).
+   */
+  @Access(AccessType.PROPERTY)
+  @OneToMany(cascade = {CascadeType.ALL})
+  @JoinColumn(name = "historyRevisionId", referencedColumnName = "historyRevisionId")
+  @JoinColumn(name = "domainRepoId", referencedColumnName = "domainRepoId")
+  @Override
+  public Set<DomainTransactionRecord> getDomainTransactionRecords() {
+    return super.getDomainTransactionRecords();
+  }
+
  @Id
-  @GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "HistorySequenceGenerator")
  @Column(name = "historyRevisionId")
  @Access(AccessType.PROPERTY)
  @Override
@@ -89,27 +150,49 @@ public class DomainHistory extends HistoryEntry {
    return nsHosts;
  }

-  /** The state of the {@link DomainContent} object at this point in time. */
-  public DomainContent getDomainContent() {
-    return domainContent;
+  /**
+   * The values of all the fields on the {@link DomainContent} object after the action represented
+   * by this history object was executed.
+   *
+   * <p>Will be absent for objects created prior to the Registry 3.0 SQL migration.
+   */
+  public Optional<DomainContent> getDomainContent() {
+    return Optional.ofNullable(domainContent);
  }

  /** The key to the {@link DomainBase} this is based off of. */
-  public VKey<DomainBase> getDomainRepoId() {
-    return VKey.create(DomainBase.class, domainRepoId, Key.create(DomainBase.class, domainRepoId));
+  public VKey<DomainBase> getParentVKey() {
+    return VKey.create(DomainBase.class, getDomainRepoId());
  }

+  /** Creates a {@link VKey} instance for this entity. */
  public VKey<DomainHistory> createVKey() {
-    return VKey.createSql(DomainHistory.class, new DomainHistoryId(domainRepoId, getId()));
+    return VKey.create(
+        DomainHistory.class, new DomainHistoryId(getDomainRepoId(), getId()), Key.create(this));
  }

  @PostLoad
  void postLoad() {
    if (domainContent != null) {
      domainContent.nsHosts = nullToEmptyImmutableCopy(nsHosts);
+      // Normally Hibernate would see that the domain fields are all null and would fill
+      // domainContent with a null object. Unfortunately, the updateTimestamp is never null in SQL.
+      if (domainContent.getDomainName() == null) {
+        domainContent = null;
+      } else {
+        if (domainContent.getRepoId() == null) {
+          domainContent = domainContent.asBuilder().setRepoId(parent.getName()).build();
+        }
+      }
    }
  }

+  // In Datastore, save as a HistoryEntry object regardless of this object's type
+  @Override
+  public ImmutableList<DatastoreEntity> toDatastoreEntities() {
+    return ImmutableList.of(asHistoryEntry());
+  }
+
  /** Class to represent the composite primary key of {@link DomainHistory} entity. */
  static class DomainHistoryId extends ImmutableObject implements Serializable {

@@ -125,19 +208,45 @@ public class DomainHistory extends HistoryEntry {
      this.id = id;
    }

-    String getDomainRepoId() {
+    /**
+     * Returns the domain repository id.
+     *
+     * <p>This method is private because it is only used by Hibernate.
+     */
+    @SuppressWarnings("unused")
+    private String getDomainRepoId() {
      return domainRepoId;
    }

-    void setDomainRepoId(String domainRepoId) {
-      this.domainRepoId = domainRepoId;
-    }
-
-    long getId() {
+    /**
+     * Returns the history revision id.
+     *
+     * <p>This method is private because it is only used by Hibernate.
+     */
+    @SuppressWarnings("unused")
+    private long getId() {
      return id;
    }

-    void setId(long id) {
+    /**
+     * Sets the domain repository id.
+     *
+     * <p>This method is private because it is only used by Hibernate and should not be used
+     * externally to keep immutability.
+     */
+    @SuppressWarnings("unused")
+    private void setDomainRepoId(String domainRepoId) {
+      this.domainRepoId = domainRepoId;
+    }
+
+    /**
+     * Sets the history revision id.
+     *
+     * <p>This method is private because it is only used by Hibernate and should not be used
+     * externally to keep immutability.
+     */
+    @SuppressWarnings("unused")
+    private void setId(long id) {
      this.id = id;
    }
  }
@@ -164,17 +273,8 @@ public class DomainHistory extends HistoryEntry {
    }

    public Builder setDomainRepoId(String domainRepoId) {
-      getInstance().domainRepoId = domainRepoId;
      getInstance().parent = Key.create(DomainBase.class, domainRepoId);
      return this;
    }
-
-    // We can remove this once all HistoryEntries are converted to History objects
-    @Override
-    public Builder setParent(Key<? extends EppResource> parent) {
-      super.setParent(parent);
-      getInstance().domainRepoId = parent.getName();
-      return this;
-    }
  }
 }
@@ -22,6 +22,7 @@ import google.registry.model.billing.BillingEvent;
 import google.registry.model.billing.BillingEvent.Recurring;
 import google.registry.model.domain.rgp.GracePeriodStatus;
 import google.registry.persistence.VKey;
+import google.registry.schema.replay.DatastoreAndSqlEntity;
 import javax.annotation.Nullable;
 import javax.persistence.Entity;
 import javax.persistence.Index;
@@ -37,7 +38,7 @@ import org.joda.time.DateTime;
@Embed
@Entity
@Table(indexes = @Index(columnList = "domainRepoId"))
-public class GracePeriod extends GracePeriodBase {
+public class GracePeriod extends GracePeriodBase implements DatastoreAndSqlEntity {

  private static GracePeriod createInternal(
      GracePeriodStatus type,
@@ -31,9 +31,9 @@ public class Period extends ImmutableObject {
  @XmlAttribute
  Unit unit;

-  @XmlValue
-  Integer value;
+  @XmlValue Integer value;

+  @Enumerated(EnumType.STRING)
  public Unit getUnit() {
    return unit;
  }
@@ -42,6 +42,18 @@ public class Period extends ImmutableObject {
    return value;
  }

+  /** This method exists solely to satisfy Hibernate. Use {@link #create(int, Unit)} instead. */
+  @SuppressWarnings("UnusedMethod")
+  private void setUnit(Unit unit) {
+    this.unit = unit;
+  }
+
+  /** This method exists solely to satisfy Hibernate. Use {@link #create(int, Unit)} instead. */
+  @SuppressWarnings("UnusedMethod")
+  private void setValue(Integer value) {
+    this.value = value;
+  }
+
  /** The unit enum. */
  public enum Unit {
    @XmlEnumValue("y")
@@ -14,10 +14,22 @@

 package google.registry.model.domain.secdns;

+import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
+
 import com.googlecode.objectify.annotation.Embed;
+import com.googlecode.objectify.annotation.Ignore;
 import google.registry.model.ImmutableObject;
+import google.registry.model.domain.secdns.DelegationSignerData.DelegationSignerDataId;
+import google.registry.schema.replay.DatastoreAndSqlEntity;
+import java.io.Serializable;
+import javax.persistence.Column;
+import javax.persistence.Entity;
+import javax.persistence.IdClass;
+import javax.persistence.Index;
+import javax.persistence.Table;
 import javax.xml.bind.DatatypeConverter;
 import javax.xml.bind.annotation.XmlElement;
+import javax.xml.bind.annotation.XmlTransient;
 import javax.xml.bind.annotation.XmlType;
 import javax.xml.bind.annotation.adapters.HexBinaryAdapter;
 import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;
@@ -30,19 +42,26 @@ import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;
 */
@Embed
@XmlType(name = "dsData")
-@javax.persistence.Entity
-public class DelegationSignerData extends ImmutableObject {
+@Entity
+@Table(indexes = @Index(columnList = "domainRepoId"))
+@IdClass(DelegationSignerDataId.class)
+public class DelegationSignerData extends ImmutableObject implements DatastoreAndSqlEntity {

  private DelegationSignerData() {}

+  @Ignore @XmlTransient @javax.persistence.Id String domainRepoId;
+
  /** The identifier for this particular key in the domain. */
-  @javax.persistence.Id int keyTag;
+  @javax.persistence.Id
+  @Column(nullable = false)
+  int keyTag;

  /**
   * The algorithm used by this key.
   *
   * @see <a href="http://tools.ietf.org/html/rfc4034#appendix-A.1">RFC 4034 Appendix A.1</a>
   */
+  @Column(nullable = false)
  @XmlElement(name = "alg")
  int algorithm;

@@ -51,6 +70,7 @@ public class DelegationSignerData extends ImmutableObject {
   *
   * @see <a href="http://tools.ietf.org/html/rfc4034#appendix-A.2">RFC 4034 Appendix A.2</a>
   */
+  @Column(nullable = false)
  int digestType;

  /**
@@ -58,6 +78,7 @@ public class DelegationSignerData extends ImmutableObject {
   *
   * @see <a href="http://tools.ietf.org/html/rfc4034#section-5.1.4">RFC 4034 Section 5.1.4</a>
   */
+  @Column(nullable = false)
  @XmlJavaTypeAdapter(HexBinaryAdapter.class)
  byte[] digest;

@@ -81,16 +102,34 @@ public class DelegationSignerData extends ImmutableObject {
    return digest == null ? "" : DatatypeConverter.printHexBinary(digest);
  }

+  public DelegationSignerData cloneWithDomainRepoId(String domainRepoId) {
+    DelegationSignerData clone = clone(this);
+    clone.domainRepoId = checkArgumentNotNull(domainRepoId);
+    return clone;
+  }
+
+  public DelegationSignerData cloneWithoutDomainRepoId() {
+    DelegationSignerData clone = clone(this);
+    clone.domainRepoId = null;
+    return clone;
+  }
+
  public static DelegationSignerData create(
-      int keyTag, int algorithm, int digestType, byte[] digest) {
+      int keyTag, int algorithm, int digestType, byte[] digest, String domainRepoId) {
    DelegationSignerData instance = new DelegationSignerData();
    instance.keyTag = keyTag;
    instance.algorithm = algorithm;
    instance.digestType = digestType;
    instance.digest = digest;
+    instance.domainRepoId = domainRepoId;
    return instance;
  }

+  public static DelegationSignerData create(
+      int keyTag, int algorithm, int digestType, byte[] digest) {
+    return create(keyTag, algorithm, digestType, digest, null);
+  }
+
  public static DelegationSignerData create(
      int keyTag, int algorithm, int digestType, String digestAsHex) {
    return create(keyTag, algorithm, digestType, DatatypeConverter.parseHexBinary(digestAsHex));
@@ -106,4 +145,20 @@ public class DelegationSignerData extends ImmutableObject {
        "%d %d %d %s",
        this.keyTag, this.algorithm, this.digestType, DatatypeConverter.printHexBinary(digest));
  }
+
+  static class DelegationSignerDataId extends ImmutableObject implements Serializable {
+    String domainRepoId;
+    int keyTag;
+
+    private DelegationSignerDataId() {}
+
+    private DelegationSignerDataId(String domainRepoId, int keyTag) {
+      this.domainRepoId = domainRepoId;
+      this.keyTag = keyTag;
+    }
+
+    public static DelegationSignerDataId create(String domainRepoId, int keyTag) {
+      return new DelegationSignerDataId(checkArgumentNotNull(domainRepoId), keyTag);
+    }
+  }
 }
@@ -20,8 +20,10 @@ import static com.google.common.base.Strings.nullToEmpty;

 import com.google.common.collect.ImmutableSet;
 import google.registry.model.EppResource;
+import google.registry.model.contact.ContactBase;
 import google.registry.model.contact.ContactResource;
 import google.registry.model.domain.DomainBase;
+import google.registry.model.domain.DomainContent;
 import google.registry.model.host.HostBase;
 import google.registry.model.host.HostResource;
 import google.registry.model.translators.EnumToAttributeAdapter.EppEnum;
@@ -128,9 +130,15 @@ public enum StatusValue implements EppEnum {

  /** Enum to help clearly list which resource types a status value is allowed to be present on. */
  private enum AllowedOn {
-    ALL(ContactResource.class, DomainBase.class, HostBase.class, HostResource.class),
+    ALL(
+        ContactBase.class,
+        ContactResource.class,
+        DomainContent.class,
+        DomainBase.class,
+        HostBase.class,
+        HostResource.class),
    NONE,
-    DOMAINS(DomainBase.class);
+    DOMAINS(DomainContent.class, DomainBase.class);

    private final ImmutableSet<Class<? extends EppResource>> classes;

@@ -14,18 +14,25 @@

 package google.registry.model.host;

+import com.google.common.collect.ImmutableList;
 import com.googlecode.objectify.Key;
 import com.googlecode.objectify.annotation.EntitySubclass;
-import google.registry.model.EppResource;
+import google.registry.model.ImmutableObject;
+import google.registry.model.host.HostHistory.HostHistoryId;
 import google.registry.model.reporting.HistoryEntry;
 import google.registry.persistence.VKey;
+import google.registry.schema.replay.DatastoreEntity;
+import google.registry.schema.replay.SqlEntity;
+import java.io.Serializable;
+import java.util.Optional;
+import javax.annotation.Nullable;
 import javax.persistence.Access;
 import javax.persistence.AccessType;
 import javax.persistence.Column;
 import javax.persistence.Entity;
-import javax.persistence.GeneratedValue;
-import javax.persistence.GenerationType;
 import javax.persistence.Id;
+import javax.persistence.IdClass;
+import javax.persistence.PostLoad;

 /**
 * A persisted history entry representing an EPP modification to a host.
@@ -45,16 +52,25 @@ import javax.persistence.Id;
    })
@EntitySubclass
@Access(AccessType.FIELD)
-public class HostHistory extends HistoryEntry {
+@IdClass(HostHistoryId.class)
+public class HostHistory extends HistoryEntry implements SqlEntity {

  // Store HostBase instead of HostResource so we don't pick up its @Id
-  HostBase hostBase;
-
-  @Column(nullable = false)
-  VKey<HostResource> hostRepoId;
+  @Nullable HostBase hostBase;
+
+  @Id
+  @Access(AccessType.PROPERTY)
+  public String getHostRepoId() {
+    return parent.getName();
+  }
+
+  /** This method is private because it is only used by Hibernate. */
+  @SuppressWarnings("unused")
+  private void setHostRepoId(String hostRepoId) {
+    parent = Key.create(HostResource.class, hostRepoId);
+  }

  @Id
-  @GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "HistorySequenceGenerator")
  @Column(name = "historyRevisionId")
  @Access(AccessType.PROPERTY)
  @Override
@@ -62,14 +78,98 @@ public class HostHistory extends HistoryEntry {
    return super.getId();
  }

-  /** The state of the {@link HostBase} object at this point in time. */
-  public HostBase getHostBase() {
-    return hostBase;
+  /**
+   * The values of all the fields on the {@link HostBase} object after the action represented by
+   * this history object was executed.
+   *
+   * <p>Will be absent for objects created prior to the Registry 3.0 SQL migration.
+   */
+  public Optional<HostBase> getHostBase() {
+    return Optional.ofNullable(hostBase);
  }

  /** The key to the {@link google.registry.model.host.HostResource} this is based off of. */
-  public VKey<HostResource> getHostRepoId() {
-    return hostRepoId;
+  public VKey<HostResource> getParentVKey() {
+    return VKey.create(HostResource.class, getHostRepoId());
+  }
+
+  /** Creates a {@link VKey} instance for this entity. */
+  public VKey<HostHistory> createVKey() {
+    return VKey.create(
+        HostHistory.class, new HostHistoryId(getHostRepoId(), getId()), Key.create(this));
+  }
+
+  @PostLoad
+  void postLoad() {
+    // Normally Hibernate would see that the host fields are all null and would fill hostBase
+    // with a null object. Unfortunately, the updateTimestamp is never null in SQL.
+    if (hostBase != null && hostBase.getHostName() == null) {
+      hostBase = null;
+    }
+  }
+
+  // In Datastore, save as a HistoryEntry object regardless of this object's type
+  @Override
+  public ImmutableList<DatastoreEntity> toDatastoreEntities() {
+    return ImmutableList.of(asHistoryEntry());
+  }
+
+  /** Class to represent the composite primary key of {@link HostHistory} entity. */
+  static class HostHistoryId extends ImmutableObject implements Serializable {
+
+    private String hostRepoId;
+
+    private Long id;
+
+    /** Hibernate requires this default constructor. */
+    private HostHistoryId() {}
+
+    HostHistoryId(String hostRepoId, long id) {
+      this.hostRepoId = hostRepoId;
+      this.id = id;
+    }
+
+    /**
+     * Returns the host repository id.
+     *
+     * <p>This method is private because it is only used by Hibernate.
+     */
+    @SuppressWarnings("unused")
+    private String getHostRepoId() {
+      return hostRepoId;
+    }
+
+    /**
+     * Returns the history revision id.
+     *
+     * <p>This method is private because it is only used by Hibernate.
+     */
+    @SuppressWarnings("unused")
+    private long getId() {
+      return id;
+    }
+
+    /**
+     * Sets the host repository id.
+     *
+     * <p>This method is private because it is only used by Hibernate and should not be used
+     * externally to keep immutability.
+     */
+    @SuppressWarnings("unused")
+    private void setHostRepoId(String hostRepoId) {
+      this.hostRepoId = hostRepoId;
+    }
+
+    /**
+     * Sets the history revision id.
+     *
+     * <p>This method is private because it is only used by Hibernate and should not be used
+     * externally to keep immutability.
+     */
+    @SuppressWarnings("unused")
+    private void setId(long id) {
+      this.id = id;
+    }
  }

  @Override
@@ -90,18 +190,8 @@ public class HostHistory extends HistoryEntry {
      return this;
    }

-    public Builder setHostRepoId(VKey<HostResource> hostRepoId) {
-      getInstance().hostRepoId = hostRepoId;
-      hostRepoId.maybeGetOfyKey().ifPresent(parent -> getInstance().parent = parent);
-      return this;
-    }
-
-    // We can remove this once all HistoryEntries are converted to History objects
-    @Override
-    public Builder setParent(Key<? extends EppResource> parent) {
-      super.setParent(parent);
-      getInstance().hostRepoId =
-          VKey.create(HostResource.class, parent.getName(), (Key<HostResource>) parent);
+    public Builder setHostRepoId(String hostRepoId) {
+      getInstance().parent = Key.create(HostResource.class, hostRepoId);
      return this;
    }
  }
@@ -32,7 +32,7 @@ import javax.persistence.AccessType;
 */
@ReportedOn
@Entity
-@javax.persistence.Entity
+@javax.persistence.Entity(name = "Host")
@ExternalMessagingName("host")
@WithStringVKey
@Access(AccessType.FIELD) // otherwise it'll use the default if the repoId (property)
@@ -101,22 +101,22 @@ public class DatastoreTransactionManager implements TransactionManager {
  }

  @Override
-  public void saveNew(Object entity) {
+  public void insert(Object entity) {
    saveEntity(entity);
  }

  @Override
-  public void saveAllNew(ImmutableCollection<?> entities) {
+  public void insertAll(ImmutableCollection<?> entities) {
    getOfy().save().entities(entities);
  }

  @Override
-  public void saveNewOrUpdate(Object entity) {
+  public void put(Object entity) {
    saveEntity(entity);
  }

  @Override
-  public void saveNewOrUpdateAll(ImmutableCollection<?> entities) {
+  public void putAll(ImmutableCollection<?> entities) {
    getOfy().save().entities(entities);
  }

@@ -131,12 +131,12 @@ public class DatastoreTransactionManager implements TransactionManager {
  }

  @Override
-  public boolean checkExists(Object entity) {
+  public boolean exists(Object entity) {
    return getOfy().load().key(Key.create(entity)).now() != null;
  }

  @Override
-  public <T> boolean checkExists(VKey<T> key) {
+  public <T> boolean exists(VKey<T> key) {
    return loadNullable(key) != null;
  }

@@ -27,14 +27,18 @@ import com.googlecode.objectify.annotation.EntitySubclass;
 import com.googlecode.objectify.annotation.Id;
 import com.googlecode.objectify.annotation.Ignore;
 import com.googlecode.objectify.annotation.Index;
+import com.googlecode.objectify.annotation.OnLoad;
 import com.googlecode.objectify.annotation.Parent;
 import google.registry.model.Buildable;
 import google.registry.model.EppResource;
 import google.registry.model.ImmutableObject;
 import google.registry.model.annotations.ExternalMessagingName;
 import google.registry.model.annotations.ReportedOn;
+import google.registry.model.contact.ContactResource;
+import google.registry.model.domain.DomainBase;
 import google.registry.model.domain.DomainRenewData;
 import google.registry.model.eppoutput.EppResponse.ResponseData;
+import google.registry.model.host.HostResource;
 import google.registry.model.poll.PendingActionNotificationResponse.ContactPendingActionNotificationResponse;
 import google.registry.model.poll.PendingActionNotificationResponse.DomainPendingActionNotificationResponse;
 import google.registry.model.poll.PendingActionNotificationResponse.HostPendingActionNotificationResponse;
@@ -54,10 +58,9 @@ import javax.persistence.Column;
 import javax.persistence.DiscriminatorColumn;
 import javax.persistence.DiscriminatorValue;
 import javax.persistence.Embedded;
-import javax.persistence.GeneratedValue;
-import javax.persistence.GenerationType;
 import javax.persistence.Inheritance;
 import javax.persistence.InheritanceType;
+import javax.persistence.PostLoad;
 import javax.persistence.Transient;
 import org.joda.time.DateTime;

@@ -98,7 +101,6 @@ public abstract class PollMessage extends ImmutableObject
  /** Entity id. */
  @Id
  @javax.persistence.Id
-  @GeneratedValue(strategy = GenerationType.IDENTITY)
  @Column(name = "poll_message_id")
  Long id;

@@ -124,11 +126,11 @@ public abstract class PollMessage extends ImmutableObject

  @Ignore String hostRepoId;

-  @Ignore Long domainRevisionId;
+  @Ignore Long domainHistoryRevisionId;

-  @Ignore Long contactRevisionId;
+  @Ignore Long contactHistoryRevisionId;

-  @Ignore Long hostRevisionId;
+  @Ignore Long hostHistoryRevisionId;

  public Key<HistoryEntry> getParentKey() {
    return parent;
@@ -152,6 +154,34 @@ public abstract class PollMessage extends ImmutableObject

  public abstract ImmutableList<ResponseData> getResponseData();

+  @PostLoad
+  void postLoad() {
+    if (domainRepoId != null) {
+      parent =
+          Key.create(
+              Key.create(DomainBase.class, domainRepoId),
+              HistoryEntry.class,
+              domainHistoryRevisionId);
+    } else if (contactRepoId != null) {
+      parent =
+          Key.create(
+              Key.create(ContactResource.class, contactRepoId),
+              HistoryEntry.class,
+              contactHistoryRevisionId);
+    } else if (hostHistoryRevisionId != null) {
+      parent =
+          Key.create(
+              Key.create(HostResource.class, hostRepoId),
+              HistoryEntry.class,
+              hostHistoryRevisionId);
+    }
+  }
+
+  @OnLoad
+  void onLoad() {
+    setSqlForeignKeys(this);
+  }
+
  @Override
  public abstract VKey<? extends PollMessage> createVKey();

@@ -217,10 +247,30 @@ public abstract class PollMessage extends ImmutableObject
      checkArgumentNotNull(instance.clientId, "clientId must be specified");
      checkArgumentNotNull(instance.eventTime, "eventTime must be specified");
      checkArgumentNotNull(instance.parent, "parent must be specified");
+      checkArgumentNotNull(instance.parent.getParent(), "parent.getParent() must be specified");
+      setSqlForeignKeys(instance);
      return super.build();
    }
  }

+  private static void setSqlForeignKeys(PollMessage pollMessage) {
+    String grandparentKind = pollMessage.parent.getParent().getKind();
+    String repoId = pollMessage.parent.getParent().getName();
+    long historyRevisionId = pollMessage.parent.getId();
+    if (Key.getKind(DomainBase.class).equals(grandparentKind)) {
+      pollMessage.domainRepoId = repoId;
+      pollMessage.domainHistoryRevisionId = historyRevisionId;
+    } else if (Key.getKind(ContactResource.class).equals(grandparentKind)) {
+      pollMessage.contactRepoId = repoId;
+      pollMessage.contactHistoryRevisionId = historyRevisionId;
+    } else if (Key.getKind(HostResource.class).equals(grandparentKind)) {
+      pollMessage.hostRepoId = repoId;
+      pollMessage.hostHistoryRevisionId = historyRevisionId;
+    } else {
+      throw new IllegalArgumentException("Unknown grandparent kind: " + grandparentKind);
+    }
+  }
+
  /**
   * A one-time poll message.
   *
@@ -62,6 +62,7 @@ import google.registry.model.domain.fee.BaseFee.FeeType;
 import google.registry.model.domain.fee.Fee;
 import google.registry.model.registry.label.PremiumList;
 import google.registry.model.registry.label.ReservedList;
+import google.registry.schema.replay.DatastoreAndSqlEntity;
 import google.registry.util.Idn;
 import java.util.Map;
 import java.util.Optional;
@@ -70,6 +71,13 @@ import java.util.concurrent.ExecutionException;
 import java.util.function.Predicate;
 import java.util.regex.Pattern;
 import javax.annotation.Nullable;
+import javax.persistence.AttributeOverride;
+import javax.persistence.AttributeOverrides;
+import javax.persistence.Column;
+import javax.persistence.EnumType;
+import javax.persistence.Enumerated;
+import javax.persistence.PostLoad;
+import javax.persistence.Transient;
 import org.joda.money.CurrencyUnit;
 import org.joda.money.Money;
 import org.joda.time.DateTime;
@@ -78,21 +86,31 @@ import org.joda.time.Duration;
 /** Persisted per-TLD configuration data. */
@ReportedOn
@Entity
-public class Registry extends ImmutableObject implements Buildable {
+@javax.persistence.Entity(name = "Tld")
+public class Registry extends ImmutableObject implements Buildable, DatastoreAndSqlEntity {

-  @Parent Key<EntityGroupRoot> parent = getCrossTldKey();
+  @Parent @Transient Key<EntityGroupRoot> parent = getCrossTldKey();

  /**
   * The canonical string representation of the TLD associated with this {@link Registry}, which is
   * the standard ASCII for regular TLDs and punycoded ASCII for IDN TLDs.
   */
-  @Id String tldStrId;
+  @Id
+  @javax.persistence.Id
+  @Column(name = "tld_name", nullable = false)
+  String tldStrId;

  /**
   * A duplicate of {@link #tldStrId}, to simplify BigQuery reporting since the id field becomes
   * {@code __key__.name} rather than being exported as a named field.
   */
-  String tldStr;
+  @Transient String tldStr;
+
+  /** Sets the Datastore specific field, tldStr, when the entity is loaded from Cloud SQL */
+  @PostLoad
+  void postLoad() {
+    tldStr = tldStrId;
+  }

  /** The suffix that identifies roids as belonging to this specific tld, e.g. -HOW for .how. */
  String roidSuffix;
@@ -116,6 +134,7 @@ public class Registry extends ImmutableObject implements Buildable {
  public static final Money DEFAULT_RENEW_BILLING_COST = Money.of(USD, 8);
  public static final Money DEFAULT_RESTORE_BILLING_COST = Money.of(USD, 100);
  public static final Money DEFAULT_SERVER_STATUS_CHANGE_BILLING_COST = Money.of(USD, 20);
+  public static final Money DEFAULT_REGISTRY_LOCK_OR_UNLOCK_BILLING_COST = Money.of(USD, 0);

  /** The type of TLD, which determines things like backups and escrow policy. */
  public enum TldType {
@@ -289,6 +308,7 @@ public class Registry extends ImmutableObject implements Buildable {
   * <p>All entries of this list must be valid keys for the map of {@code DnsWriter}s injected by
   * <code>@Inject Map<String, DnsWriter></code>
   */
+  @Column(nullable = false)
  Set<String> dnsWriters;

  /**
@@ -312,6 +332,7 @@ public class Registry extends ImmutableObject implements Buildable {
   * <p>Failure to do so can result in parallel writes to the {@link
   * google.registry.dns.writer.DnsWriter}, which may be dangerous depending on your implementation.
   */
+  @Column(nullable = false)
  int numDnsPublishLocks;

  /** Updates an unset numDnsPublishLocks (0) to the standard default of 1. */
@@ -327,6 +348,7 @@ public class Registry extends ImmutableObject implements Buildable {
   * <p>This will be equal to {@link #tldStr} for ASCII TLDs, but will be non-ASCII for IDN TLDs. We
   * store this in a field so that it will be retained upon import into BigQuery.
   */
+  @Column(nullable = false)
  String tldUnicode;

  /**
@@ -334,9 +356,11 @@ public class Registry extends ImmutableObject implements Buildable {
   *
   * <p>This is optional; if not configured, then information won't be exported for this TLD.
   */
-  String driveFolderId;
+  @Nullable String driveFolderId;

  /** The type of the TLD, whether it's real or for testing. */
+  @Column(nullable = false)
+  @Enumerated(EnumType.STRING)
  TldType tldType = TldType.REAL;

  /**
@@ -345,20 +369,24 @@ public class Registry extends ImmutableObject implements Buildable {
   * <p>Note that this boolean is the sole determiner on whether invoices should be generated for a
   * TLD. This applies to {@link TldType#TEST} TLDs as well.
   */
+  @Column(nullable = false)
  boolean invoicingEnabled = false;

  /**
   * A property that transitions to different TldStates at different times. Stored as a list of
   * TldStateTransition embedded objects using the @Mapify annotation.
   */
+  @Column(nullable = false)
  @Mapify(TimedTransitionProperty.TimeMapper.class)
  TimedTransitionProperty<TldState, TldStateTransition> tldStateTransitions =
      TimedTransitionProperty.forMapify(DEFAULT_TLD_STATE, TldStateTransition.class);

  /** An automatically managed creation timestamp. */
+  @Column(nullable = false)
  CreateAutoTimestamp creationTime = CreateAutoTimestamp.create(null);

  /** The set of reserved lists that are applicable to this registry. */
+  @Column(name = "reserved_list_names", nullable = false)
  Set<Key<ReservedList>> reservedLists;

  /** Retrieves an ImmutableSet of all ReservedLists associated with this tld. */
@@ -367,12 +395,15 @@ public class Registry extends ImmutableObject implements Buildable {
  }

  /** The static {@link PremiumList} for this TLD, if there is one. */
+  @Column(name = "premium_list_name", nullable = true)
  Key<PremiumList> premiumList;

  /** Should RDE upload a nightly escrow deposit for this TLD? */
+  @Column(nullable = false)
  boolean escrowEnabled = DEFAULT_ESCROW_ENABLED;

  /** Whether the pull queue that writes to authoritative DNS is paused for this TLD. */
+  @Column(nullable = false)
  boolean dnsPaused = DEFAULT_DNS_PAUSED;

  /**
@@ -381,41 +412,85 @@ public class Registry extends ImmutableObject implements Buildable {
   * <p>Domain deletes are free and effective immediately so long as they take place within this
   * amount of time following creation.
   */
+  @Column(nullable = false)
  Duration addGracePeriodLength = DEFAULT_ADD_GRACE_PERIOD;

  /** The length of the anchor tenant add grace period for this TLD. */
+  @Column(nullable = false)
  Duration anchorTenantAddGracePeriodLength = DEFAULT_ANCHOR_TENANT_ADD_GRACE_PERIOD;

  /** The length of the auto renew grace period for this TLD. */
+  @Column(nullable = false)
  Duration autoRenewGracePeriodLength = DEFAULT_AUTO_RENEW_GRACE_PERIOD;

  /** The length of the redemption grace period for this TLD. */
+  @Column(nullable = false)
  Duration redemptionGracePeriodLength = DEFAULT_REDEMPTION_GRACE_PERIOD;

  /** The length of the renew grace period for this TLD. */
+  @Column(nullable = false)
  Duration renewGracePeriodLength = DEFAULT_RENEW_GRACE_PERIOD;

  /** The length of the transfer grace period for this TLD. */
+  @Column(nullable = false)
  Duration transferGracePeriodLength = DEFAULT_TRANSFER_GRACE_PERIOD;

  /** The length of time before a transfer is automatically approved for this TLD. */
+  @Column(nullable = false)
  Duration automaticTransferLength = DEFAULT_AUTOMATIC_TRANSFER_LENGTH;

  /** The length of time a domain spends in the non-redeemable pending delete phase for this TLD. */
+  @Column(nullable = false)
  Duration pendingDeleteLength = DEFAULT_PENDING_DELETE_LENGTH;

  /** The currency unit for all costs associated with this TLD. */
+  @Column(nullable = false)
  CurrencyUnit currency = DEFAULT_CURRENCY;

  /** The per-year billing cost for registering a new domain name. */
+  @AttributeOverrides({
+    @AttributeOverride(
+        name = "money.amount",
+        column = @Column(name = "create_billing_cost_amount")),
+    @AttributeOverride(
+        name = "money.currency",
+        column = @Column(name = "create_billing_cost_currency"))
+  })
  Money createBillingCost = DEFAULT_CREATE_BILLING_COST;

  /** The one-time billing cost for restoring a domain name from the redemption grace period. */
+  @AttributeOverrides({
+    @AttributeOverride(
+        name = "money.amount",
+        column = @Column(name = "restore_billing_cost_amount")),
+    @AttributeOverride(
+        name = "money.currency",
+        column = @Column(name = "restore_billing_cost_currency"))
+  })
  Money restoreBillingCost = DEFAULT_RESTORE_BILLING_COST;

  /** The one-time billing cost for changing the server status (i.e. lock). */
+  @AttributeOverrides({
+    @AttributeOverride(
+        name = "money.amount",
+        column = @Column(name = "server_status_change_billing_cost_amount")),
+    @AttributeOverride(
+        name = "money.currency",
+        column = @Column(name = "server_status_change_billing_cost_currency"))
+  })
  Money serverStatusChangeBillingCost = DEFAULT_SERVER_STATUS_CHANGE_BILLING_COST;

+  /** The one-time billing cost for a registry lock/unlock action initiated by a registrar. */
+  @AttributeOverrides({
+      @AttributeOverride(
+          name = "money.amount",
+          column = @Column(name = "registry_lock_or_unlock_cost_amount")),
+      @AttributeOverride(
+          name = "money.currency",
+          column = @Column(name = "registry_lock_or_unlock_cost_currency"))
+  })
+  Money registryLockOrUnlockBillingCost = DEFAULT_REGISTRY_LOCK_OR_UNLOCK_BILLING_COST;
+
  /**
   * A property that transitions to different renew billing costs at different times. Stored as a
   * list of BillingCostTransition embedded objects using the @Mapify annotation.
@@ -424,11 +499,13 @@ public class Registry extends ImmutableObject implements Buildable {
   * name. This cost is also used to compute costs for transfers, since each transfer includes a
   * renewal to ensure transfers have a cost.
   */
+  @Column(nullable = false)
  @Mapify(TimedTransitionProperty.TimeMapper.class)
  TimedTransitionProperty<Money, BillingCostTransition> renewBillingCostTransitions =
      TimedTransitionProperty.forMapify(DEFAULT_RENEW_BILLING_COST, BillingCostTransition.class);

  /** A property that tracks the EAP fee schedule (if any) for the TLD. */
+  @Column(nullable = false)
  @Mapify(TimedTransitionProperty.TimeMapper.class)
  TimedTransitionProperty<Money, BillingCostTransition> eapFeeSchedule =
      TimedTransitionProperty.forMapify(DEFAULT_EAP_BILLING_COST, BillingCostTransition.class);
@@ -437,13 +514,14 @@ public class Registry extends ImmutableObject implements Buildable {
  String lordnUsername;

  /** The end of the claims period (at or after this time, claims no longer applies). */
+  @Column(nullable = false)
  DateTime claimsPeriodEnd = END_OF_TIME;

  /** An allow list of clients allowed to be used on domains on this TLD (ignored if empty). */
-  Set<String> allowedRegistrantContactIds;
+  @Nullable Set<String> allowedRegistrantContactIds;

  /** An allow list of hosts allowed to be used on domains on this TLD (ignored if empty). */
-  Set<String> allowedFullyQualifiedHostNames;
+  @Nullable Set<String> allowedFullyQualifiedHostNames;

  public String getTldStr() {
    return tldStr;
@@ -566,6 +644,11 @@ public class Registry extends ImmutableObject implements Buildable {
    return serverStatusChangeBillingCost;
  }

+  /** Returns the cost of a registry lock/unlock. */
+  public Money getRegistryLockOrUnlockBillingCost() {
+    return registryLockOrUnlockBillingCost;
+  }
+
  public ImmutableSortedMap<DateTime, TldState> getTldStateTransitions() {
    return tldStateTransitions.toValueMap();
  }
@@ -867,6 +950,12 @@ public class Registry extends ImmutableObject implements Buildable {
      return this;
    }

+    public Builder setRegistryLockOrUnlockBillingCost(Money amount) {
+      checkArgument(amount.isPositiveOrZero(), "Registry lock/unlock cost cannot be negative");
+      getInstance().registryLockOrUnlockBillingCost = amount;
+      return this;
+    }
+
    public Builder setLordnUsername(String username) {
      getInstance().lordnUsername = username;
      return this;
@@ -918,6 +1007,9 @@ public class Registry extends ImmutableObject implements Buildable {
      checkArgument(
          instance.getServerStatusChangeCost().getCurrencyUnit().equals(instance.currency),
          "Server status change cost must be in the registry's currency");
+      checkArgument(
+          instance.getRegistryLockOrUnlockBillingCost().getCurrencyUnit().equals(instance.currency),
+          "Registry lock/unlock cost must be in the registry's currency");
      Predicate<Money> currencyCheck =
          (Money money) -> money.getCurrencyUnit().equals(instance.currency);
      checkArgument(
@@ -43,7 +43,7 @@ public class ReservedListDualWriteDao {

  /** Persist a new reserved list to Cloud SQL. */
  public static void save(ReservedList reservedList) {
-    ofyTm().transact(() -> ofyTm().saveNewOrUpdate(reservedList));
+    ofyTm().transact(() -> ofyTm().put(reservedList));
    try {
      logger.atInfo().log("Saving reserved list %s to Cloud SQL", reservedList.getName());
      ReservedListSqlDao.save(reservedList);
@@ -31,7 +31,7 @@ public class ReservedListSqlDao {
  /** Persist a new reserved list to Cloud SQL. */
  public static void save(ReservedList reservedList) {
    checkArgumentNotNull(reservedList, "Must specify reservedList");
-    jpaTm().transact(() -> jpaTm().saveNew(reservedList));
+    jpaTm().transact(() -> jpaTm().insert(reservedList));
  }

  /**
@@ -19,8 +19,17 @@ import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;

 import com.google.common.collect.ImmutableSet;
 import com.googlecode.objectify.annotation.Embed;
+import com.googlecode.objectify.annotation.Ignore;
 import google.registry.model.Buildable;
 import google.registry.model.ImmutableObject;
+import google.registry.schema.replay.DatastoreAndSqlEntity;
+import javax.persistence.Column;
+import javax.persistence.Entity;
+import javax.persistence.EnumType;
+import javax.persistence.Enumerated;
+import javax.persistence.GeneratedValue;
+import javax.persistence.GenerationType;
+import javax.persistence.Id;
 import org.joda.time.DateTime;

 /**
@@ -34,9 +43,17 @@ import org.joda.time.DateTime;
 * uses HistoryEntry.otherClientId because the losing party in a transfer is always the otherClient.
 */
@Embed
-public class DomainTransactionRecord extends ImmutableObject implements Buildable {
+@Entity
+public class DomainTransactionRecord extends ImmutableObject
+    implements Buildable, DatastoreAndSqlEntity {
+
+  @Id
+  @Ignore
+  @GeneratedValue(strategy = GenerationType.IDENTITY)
+  Long id;

  /** The TLD this record operates on. */
+  @Column(nullable = false)
  String tld;

  /**
@@ -50,9 +67,12 @@ public class DomainTransactionRecord extends ImmutableObject implements Buildabl
   *     href="https://www.icann.org/resources/unthemed-pages/registry-agmt-appc-10-2001-05-11-en">
   *     Grace period spec</a>
   */
+  @Column(nullable = false)
  DateTime reportingTime;

  /** The transaction report field we add reportAmount to for this registrar. */
+  @Column(nullable = false)
+  @Enumerated(value = EnumType.STRING)
  TransactionReportField reportField;

  /**
@@ -67,6 +87,7 @@ public class DomainTransactionRecord extends ImmutableObject implements Buildabl
   * original SUCCESSFUL transfer counters. Finally, if we explicitly allow a transfer, the report
   * amount is 0, as we've already counted the transfer in the original request.
   */
+  @Column(nullable = false)
  Integer reportAmount;

  /**
@@ -62,7 +62,7 @@ import org.joda.time.DateTime;
@MappedSuperclass
@WithStringVKey // TODO(b/162229294): This should be resolved during the course of that bug
@Access(AccessType.FIELD)
-public class HistoryEntry extends ImmutableObject implements Buildable, DatastoreEntity, SqlEntity {
+public class HistoryEntry extends ImmutableObject implements Buildable, DatastoreEntity {

  /** Represents the type of history entry. */
  public enum Type {
@@ -178,7 +178,7 @@ public class HistoryEntry extends ImmutableObject implements Buildable, Datastor
  boolean bySuperuser;

  /** Reason for the change. */
-  @Column(nullable = false, name = "historyReason")
+  @Column(name = "historyReason")
  String reason;

  /** Whether this change was requested by a registrar. */
@@ -202,7 +202,7 @@ public class HistoryEntry extends ImmutableObject implements Buildable, Datastor
    return id == null ? 0L : id;
  }

-  // This method is required by Hibernate.
+  /** This method exists solely to satisfy Hibernate. Use the {@link Builder} instead. */
  @SuppressWarnings("UnusedMethod")
  private void setId(long id) {
    this.id = id;
@@ -254,10 +254,28 @@ public class HistoryEntry extends ImmutableObject implements Buildable, Datastor
    return requestedByRegistrar;
  }

-  public ImmutableSet<DomainTransactionRecord> getDomainTransactionRecords() {
+  public Set<DomainTransactionRecord> getDomainTransactionRecords() {
    return nullToEmptyImmutableCopy(domainTransactionRecords);
  }

+  /** This method exists solely to satisfy Hibernate. Use the {@link Builder} instead. */
+  @SuppressWarnings("UnusedMethod")
+  private void setPeriod(Period period) {
+    this.period = period;
+  }
+
+  /** This method exists solely to satisfy Hibernate. Use the {@link Builder} instead. */
+  @SuppressWarnings("UnusedMethod")
+  private void setOtherRegistrarId(String otherRegistrarId) {
+    this.otherClientId = otherRegistrarId;
+  }
+
+  /** This method exists solely to satisfy Hibernate. Use the {@link Builder} instead. */
+  @SuppressWarnings("UnusedMethod")
+  private void setDomainTransactionRecords(Set<DomainTransactionRecord> domainTransactionRecords) {
+    this.domainTransactionRecords = ImmutableSet.copyOf(domainTransactionRecords);
+  }
+
  public static VKey<HistoryEntry> createVKey(Key<HistoryEntry> key) {
    // TODO(b/159207551): This will likely need some revision.  As it stands, this method was
    // introduced purely to facilitate testing of VKey specialization in VKeyTranslatorFactory.
@@ -290,19 +308,10 @@ public class HistoryEntry extends ImmutableObject implements Buildable, Datastor
          new DomainHistory.Builder().copyFrom(this).setDomainRepoId(parent.getName()).build();
    } else if (parentKind.equals(getKind(HostResource.class))) {
      resultEntity =
-          new HostHistory.Builder()
-              .copyFrom(this)
-              .setHostRepoId(
-                  VKey.create(HostResource.class, parent.getName(), (Key<HostResource>) parent))
-              .build();
+          new HostHistory.Builder().copyFrom(this).setHostRepoId(parent.getName()).build();
    } else if (parentKind.equals(getKind(ContactResource.class))) {
      resultEntity =
-          new ContactHistory.Builder()
-              .copyFrom(this)
-              .setContactRepoId(
-                  VKey.create(
-                      ContactResource.class, parent.getName(), (Key<ContactResource>) parent))
-              .build();
+          new ContactHistory.Builder().copyFrom(this).setContactRepoId(parent.getName()).build();
    } else {
      throw new IllegalStateException(
          String.format("Unknown kind of HistoryEntry parent %s", parentKind));
@@ -313,13 +322,7 @@ public class HistoryEntry extends ImmutableObject implements Buildable, Datastor
  // In SQL, save the child type
  @Override
  public ImmutableList<SqlEntity> toSqlEntities() {
-    return ImmutableList.of(toChildHistoryEntity());
-  }
-
-  // In Datastore, save as a HistoryEntry object regardless of this object's type
-  @Override
-  public ImmutableList<DatastoreEntity> toDatastoreEntities() {
-    return ImmutableList.of(asHistoryEntry());
+    return ImmutableList.of((SqlEntity) toChildHistoryEntity());
  }

  /** A builder for {@link HistoryEntry} since it is immutable */
@@ -346,7 +349,10 @@ public class HistoryEntry extends ImmutableObject implements Buildable, Datastor
      setBySuperuser(historyEntry.bySuperuser);
      setReason(historyEntry.reason);
      setRequestedByRegistrar(historyEntry.requestedByRegistrar);
-      setDomainTransactionRecords(nullToEmptyImmutableCopy(historyEntry.domainTransactionRecords));
+      setDomainTransactionRecords(
+          historyEntry.domainTransactionRecords == null
+              ? null
+              : ImmutableSet.copyOf(historyEntry.domainTransactionRecords));
      return thisCastToDerived();
    }

@@ -46,6 +46,7 @@ import google.registry.model.annotations.NotBackedUp;
 import google.registry.model.annotations.NotBackedUp.Reason;
 import google.registry.model.annotations.VirtualEntity;
 import google.registry.model.common.CrossTldSingleton;
+import google.registry.schema.replay.DatastoreAndSqlEntity;
 import google.registry.schema.replay.DatastoreEntity;
 import google.registry.schema.replay.SqlEntity;
 import google.registry.util.CollectionUtils;
@@ -97,7 +98,7 @@ import org.joda.time.DateTime;
@NotBackedUp(reason = Reason.EXTERNALLY_SOURCED)
@javax.persistence.Entity(name = "ClaimsList")
@Table
-public class ClaimsListShard extends ImmutableObject implements DatastoreEntity {
+public class ClaimsListShard extends ImmutableObject implements DatastoreAndSqlEntity {

  private static final FluentLogger logger = FluentLogger.forEnclosingClass();

@@ -354,11 +355,6 @@ public class ClaimsListShard extends ImmutableObject implements DatastoreEntity
    }
  }

-  @Override
-  public ImmutableList<SqlEntity> toSqlEntities() {
-    return ImmutableList.of(); // ClaimsLists are dually written
-  }
-
  /** Virtual parent entity for claims list shards of a specific revision. */
  @Entity
  @VirtualEntity
@@ -88,7 +88,7 @@ public class VKey<T> extends ImmutableObject implements Serializable {
   */
  public static <T> VKey<T> create(Class<T> kind, long id) {
    checkArgument(
-        kind.isAssignableFrom(BackupGroupRoot.class),
+        BackupGroupRoot.class.isAssignableFrom(kind),
        "The kind %s is not a BackupGroupRoot and thus needs its entire entity group chain"
            + " specified in a parent",
        kind.getCanonicalName());
@@ -106,13 +106,90 @@ public class VKey<T> extends ImmutableObject implements Serializable {
   */
  public static <T> VKey<T> create(Class<T> kind, String name) {
    checkArgument(
-        kind.isAssignableFrom(BackupGroupRoot.class),
+        BackupGroupRoot.class.isAssignableFrom(kind),
        "The kind %s is not a BackupGroupRoot and thus needs its entire entity group chain"
            + " specified in a parent",
        kind.getCanonicalName());
    return new VKey<T>(kind, Key.create(kind, name), name);
  }

+  /**
+   * Returns a clone with an ofy key restored from {@code ancestors}.
+   *
+   * <p>The arguments should generally consist of pairs of Class and value, where the Class is the
+   * kind of the ancestor key and the value is either a String or a Long.
+   *
+   * <p>For example, to restore the objectify key for
+   * DomainBase("COM-1234")/HistoryEntry(123)/PollEvent(567), one might use:
+   *
+   * <pre>{@code
+   * pollEvent.restoreOfy(DomainBase.class, "COM-1234", HistoryEntry.class, 567)
+   * }</pre>
+   *
+   * <p>The final key id or name is obtained from the SQL key. It is assumed that this value must be
+   * either a long integer or a {@code String} and that this proper identifier for the objectify
+   * key.
+   *
+   * <p>As a special case, an objectify Key may be used as the first ancestor instead of a Class,
+   * value pair.
+   */
+  public VKey<T> restoreOfy(Object... ancestors) {
+    Class lastClass = null;
+    Key<?> lastKey = null;
+    for (Object ancestor : ancestors) {
+      if (ancestor instanceof Class) {
+        if (lastClass != null) {
+          throw new IllegalArgumentException(ancestor + " used as a key value.");
+        }
+        lastClass = (Class) ancestor;
+        continue;
+      } else if (ancestor instanceof Key) {
+        if (lastKey != null) {
+          throw new IllegalArgumentException(
+              "Objectify keys may only be used for the first argument");
+        }
+        lastKey = (Key) ancestor;
+        continue;
+      }
+
+      // The argument should be a value.
+      if (lastClass == null) {
+        throw new IllegalArgumentException("Argument " + ancestor + " should be a class.");
+      }
+      if (ancestor instanceof Long) {
+        lastKey = Key.create(lastKey, lastClass, (Long) ancestor);
+      } else if (ancestor instanceof String) {
+        lastKey = Key.create(lastKey, lastClass, (String) ancestor);
+      } else {
+        throw new IllegalArgumentException("Key value " + ancestor + " must be a string or long.");
+      }
+      lastClass = null;
+    }
+
+    // Make sure we didn't end up with a dangling class with no value.
+    if (lastClass != null) {
+      throw new IllegalArgumentException("Missing value for last key of type " + lastClass);
+    }
+
+    Object sqlKey = getSqlKey();
+    Key<T> ofyKey =
+        sqlKey instanceof Long
+            ? Key.create(lastKey, getKind(), (Long) sqlKey)
+            : Key.create(lastKey, getKind(), (String) sqlKey);
+
+    return VKey.create((Class<T>) getKind(), sqlKey, ofyKey);
+  }
+
+  /**
+   * Returns a clone of {@code key} with an ofy key restored from {@code ancestors}.
+   *
+   * <p>This is the static form of the method restoreOfy() above. If {@code key} is null, it returns
+   * null.
+   */
+  public static <T> VKey<T> restoreOfyFrom(@Nullable VKey<T> key, Object... ancestors) {
+    return key == null ? null : key.restoreOfy(ancestors);
+  }
+
  /** Returns the type of the entity. */
  public Class<? extends T> getKind() {
    return this.kind;
@@ -32,7 +32,11 @@ public class CreateAutoTimestampConverter
    implements AttributeConverter<CreateAutoTimestamp, Timestamp> {

  @Override
-  public Timestamp convertToDatabaseColumn(CreateAutoTimestamp entity) {
+  @Nullable
+  public Timestamp convertToDatabaseColumn(@Nullable CreateAutoTimestamp entity) {
+    if (entity == null) {
+      return null;
+    }
    DateTime dateTime = firstNonNull(entity.getTimestamp(), jpaTm().getTransactionTime());
    return Timestamp.from(DateTimeUtils.toZonedDateTime(dateTime).toInstant());
  }
@@ -18,20 +18,19 @@ import static google.registry.model.common.EntityGroupRoot.getCrossTldKey;

 import com.googlecode.objectify.Key;
 import google.registry.model.registry.label.ReservedList;
-import javax.persistence.AttributeConverter;
 import javax.persistence.Converter;

-/** JPA converter for a {@link Key} containing a {@link ReservedList} */
+/** JPA converter for a set of {@link Key} containing a {@link ReservedList} */
@Converter(autoApply = true)
-public class ReservedListKeyConverter implements AttributeConverter<Key<ReservedList>, String> {
+public class ReservedListKeySetConverter extends StringSetConverterBase<Key<ReservedList>> {

  @Override
-  public String convertToDatabaseColumn(Key<ReservedList> attribute) {
-    return (attribute == null) ? null : attribute.getName();
+  String toString(Key<ReservedList> key) {
+    return key.getName();
  }

  @Override
-  public Key<ReservedList> convertToEntityAttribute(String dbData) {
-    return (dbData == null) ? null : Key.create(getCrossTldKey(), ReservedList.class, dbData);
+  Key<ReservedList> fromString(String value) {
+    return Key.create(getCrossTldKey(), ReservedList.class, value);
  }
 }
@@ -224,7 +224,7 @@ public class JpaTransactionManagerImpl implements JpaTransactionManager {
  }

  @Override
-  public void saveNew(Object entity) {
+  public void insert(Object entity) {
    checkArgumentNotNull(entity, "entity must be specified");
    assertInTransaction();
    getEntityManager().persist(entity);
@@ -232,14 +232,14 @@ public class JpaTransactionManagerImpl implements JpaTransactionManager {
  }

  @Override
-  public void saveAllNew(ImmutableCollection<?> entities) {
+  public void insertAll(ImmutableCollection<?> entities) {
    checkArgumentNotNull(entities, "entities must be specified");
    assertInTransaction();
-    entities.forEach(this::saveNew);
+    entities.forEach(this::insert);
  }

  @Override
-  public void saveNewOrUpdate(Object entity) {
+  public void put(Object entity) {
    checkArgumentNotNull(entity, "entity must be specified");
    assertInTransaction();
    getEntityManager().merge(entity);
@@ -247,17 +247,17 @@ public class JpaTransactionManagerImpl implements JpaTransactionManager {
  }

  @Override
-  public void saveNewOrUpdateAll(ImmutableCollection<?> entities) {
+  public void putAll(ImmutableCollection<?> entities) {
    checkArgumentNotNull(entities, "entities must be specified");
    assertInTransaction();
-    entities.forEach(this::saveNewOrUpdate);
+    entities.forEach(this::put);
  }

  @Override
  public void update(Object entity) {
    checkArgumentNotNull(entity, "entity must be specified");
    assertInTransaction();
-    checkArgument(checkExists(entity), "Given entity does not exist");
+    checkArgument(exists(entity), "Given entity does not exist");
    getEntityManager().merge(entity);
    transactionInfo.get().addUpdate(entity);
  }
@@ -270,22 +270,22 @@ public class JpaTransactionManagerImpl implements JpaTransactionManager {
  }

  @Override
-  public <T> boolean checkExists(VKey<T> key) {
+  public <T> boolean exists(VKey<T> key) {
    checkArgumentNotNull(key, "key must be specified");
    EntityType<?> entityType = getEntityType(key.getKind());
    ImmutableSet<EntityId> entityIds = getEntityIdsFromSqlKey(entityType, key.getSqlKey());
-    return checkExists(entityType.getName(), entityIds);
+    return exists(entityType.getName(), entityIds);
  }

  @Override
-  public boolean checkExists(Object entity) {
+  public boolean exists(Object entity) {
    checkArgumentNotNull(entity, "entity must be specified");
    EntityType<?> entityType = getEntityType(entity.getClass());
    ImmutableSet<EntityId> entityIds = getEntityIdsFromEntity(entityType, entity);
-    return checkExists(entityType.getName(), entityIds);
+    return exists(entityType.getName(), entityIds);
  }

-  private boolean checkExists(String entityName, ImmutableSet<EntityId> entityIds) {
+  private boolean exists(String entityName, ImmutableSet<EntityId> entityIds) {
    assertInTransaction();
    TypedQuery<Integer> query =
        getEntityManager()
@@ -391,7 +391,7 @@ public class JpaTransactionManagerImpl implements JpaTransactionManager {
  private static ImmutableSet<EntityId> getEntityIdsFromEntity(
      EntityType<?> entityType, Object entity) {
    if (entityType.hasSingleIdAttribute()) {
-      String idName = entityType.getDeclaredId(entityType.getIdType().getJavaType()).getName();
+      String idName = entityType.getId(entityType.getIdType().getJavaType()).getName();
      Object idValue = getFieldValue(entity, idName);
      return ImmutableSet.of(new EntityId(idName, idValue));
    } else {
@@ -402,7 +402,7 @@ public class JpaTransactionManagerImpl implements JpaTransactionManager {
  private static ImmutableSet<EntityId> getEntityIdsFromSqlKey(
      EntityType<?> entityType, Object sqlKey) {
    if (entityType.hasSingleIdAttribute()) {
-      String idName = entityType.getDeclaredId(entityType.getIdType().getJavaType()).getName();
+      String idName = entityType.getId(entityType.getIdType().getJavaType()).getName();
      return ImmutableSet.of(new EntityId(idName, sqlKey));
    } else {
      return getEntityIdsFromIdContainer(entityType, sqlKey);
@@ -429,7 +429,7 @@ public class JpaTransactionManagerImpl implements JpaTransactionManager {

  private static Object getFieldValue(Object object, String fieldName) {
    try {
-      Field field = object.getClass().getDeclaredField(fieldName);
+      Field field = getField(object.getClass(), fieldName);
      field.setAccessible(true);
      return field.get(object);
    } catch (NoSuchFieldException | IllegalAccessException e) {
@@ -437,6 +437,21 @@ public class JpaTransactionManagerImpl implements JpaTransactionManager {
    }
  }

+  /** Gets the field definition from clazz or any superclass. */
+  private static Field getField(Class clazz, String fieldName) throws NoSuchFieldException {
+    try {
+      // Note that we have to use getDeclaredField() for this, getField() just finds public fields.
+      return clazz.getDeclaredField(fieldName);
+    } catch (NoSuchFieldException e) {
+      Class base = clazz.getSuperclass();
+      if (base != null) {
+        return getField(base, fieldName);
+      } else {
+        throw e;
+      }
+    }
+  }
+
  private static class TransactionInfo {
    EntityManager entityManager;
    boolean inTransaction = false;
@@ -208,7 +208,7 @@ public class Transaction extends ImmutableObject implements Buildable {

    @Override
    public void writeToDatastore() {
-      ofyTm().saveNewOrUpdate(entity);
+      ofyTm().put(entity);
    }

    @Override
@@ -86,16 +86,16 @@ public interface TransactionManager {
  DateTime getTransactionTime();

  /** Persists a new entity in the database, throws exception if the entity already exists. */
-  void saveNew(Object entity);
+  void insert(Object entity);

  /** Persists all new entities in the database, throws exception if any entity already exists. */
-  void saveAllNew(ImmutableCollection<?> entities);
+  void insertAll(ImmutableCollection<?> entities);

  /** Persists a new entity or update the existing entity in the database. */
-  void saveNewOrUpdate(Object entity);
+  void put(Object entity);

  /** Persists all new entities or update the existing entities in the database. */
-  void saveNewOrUpdateAll(ImmutableCollection<?> entities);
+  void putAll(ImmutableCollection<?> entities);

  /** Updates an entity in the database, throws exception if the entity does not exist. */
  void update(Object entity);
@@ -104,10 +104,10 @@ public interface TransactionManager {
  void updateAll(ImmutableCollection<?> entities);

  /** Returns whether the given entity with same ID exists. */
-  boolean checkExists(Object entity);
+  boolean exists(Object entity);

  /** Returns whether the entity of given key exists. */
-  <T> boolean checkExists(VKey<T> key);
+  <T> boolean exists(VKey<T> key);

  /** Loads the entity by its id, returns empty if the entity doesn't exist. */
  <T> Optional<T> maybeLoad(VKey<T> key);
@@ -116,14 +116,10 @@ public class IcannHttpReporter {
      // 1000 (i. e. success), there is no need to parse it.
      if (response.getStatusCode() == HttpStatusCodes.STATUS_CODE_BAD_REQUEST) {
        success = false;
-        // To debug if there is a problem with our parsing, we wrap the response and print the stack
-        // trace of it. As far as we can tell, the stack trace for such an exception contains the
-        // response content that is decoded correctly using the expected charset.
-        new HttpResponseException(response).printStackTrace();
        XjcIirdeaResult result = parseResult(content);
        logger.atWarning().log(
            "PUT rejected, status code %s:\n%s\n%s",
-            result.getCode(), result.getMsg(), result.getDescription());
+            result.getCode().getValue(), result.getMsg(), result.getDescription());
      }
    } finally {
      if (response != null) {
@@ -110,6 +110,12 @@ abstract class CreateOrUpdateTldCommand extends MutatingCommand {
      description = "One-time billing cost for a server status change")
  private Money serverStatusChangeCost;

+  @Nullable
+  @Parameter(
+      names = "--registry_lock_or_unlock_cost",
+      description = "One-time billing cost for a registry lock or unlock")
+  private Money registryLockOrUnlockCost;
+
  @Nullable
  @Parameter(
      names = "--tld_type",
@@ -326,6 +332,8 @@ abstract class CreateOrUpdateTldCommand extends MutatingCommand {
      Optional.ofNullable(roidSuffix).ifPresent(builder::setRoidSuffix);
      Optional.ofNullable(serverStatusChangeCost)
          .ifPresent(builder::setServerStatusChangeBillingCost);
+      Optional.ofNullable(registryLockOrUnlockCost)
+          .ifPresent(builder::setRegistryLockOrUnlockBillingCost);
      Optional.ofNullable(tldType).ifPresent(builder::setTldType);
      Optional.ofNullable(invoicingEnabled).ifPresent(builder::setInvoicingEnabled);
      Optional.ofNullable(lordnUsername).ifPresent(u -> builder.setLordnUsername(u.orElse(null)));
@@ -72,7 +72,7 @@ final class CreateRegistrarCommand extends CreateOrUpdateRegistrarCommand

  @Override
  void saveToCloudSql(Registrar registrar) {
-    jpaTm().saveNew(registrar);
+    jpaTm().insert(registrar);
  }

  @Nullable
@@ -380,7 +380,7 @@ public final class DomainLockUtils {
              .setReason(Reason.SERVER_STATUS)
              .setTargetId(domain.getForeignKey())
              .setClientId(domain.getCurrentSponsorClientId())
-              .setCost(Registry.get(domain.getTld()).getServerStatusChangeCost())
+              .setCost(Registry.get(domain.getTld()).getRegistryLockOrUnlockBillingCost())
              .setEventTime(now)
              .setBillingTime(now)
              .setParent(historyEntry)
@@ -11,8 +11,6 @@
    </attributes>
  </embeddable>

-  <sequence-generator name="HistorySequenceGenerator" sequence-name="history_id_sequence" />
-
  <persistence-unit-metadata>
    <persistence-unit-defaults>
      <entity-listeners>
@@ -45,25 +45,27 @@
    <class>google.registry.model.contact.ContactResource</class>
    <class>google.registry.model.domain.DomainBase</class>
    <class>google.registry.model.domain.DomainHistory</class>
+    <class>google.registry.model.domain.GracePeriod</class>
+    <class>google.registry.model.domain.secdns.DelegationSignerData</class>
    <class>google.registry.model.domain.token.AllocationToken</class>
    <class>google.registry.model.host.HostHistory</class>
    <class>google.registry.model.host.HostResource</class>
-    <class>google.registry.model.registrar.Registrar</class>
-    <class>google.registry.model.registrar.RegistrarContact</class>
-    <class>google.registry.model.registry.label.PremiumList</class>
-    <class>google.registry.model.reporting.Spec11ThreatMatch</class>
-    <class>google.registry.persistence.transaction.TransactionEntity</class>
-    <class>google.registry.model.tmch.ClaimsListShard</class>
-    <class>google.registry.schema.domain.RegistryLock</class>
-    <class>google.registry.schema.cursor.Cursor</class>
-    <class>google.registry.schema.server.Lock</class>
-    <class>google.registry.schema.tld.PremiumEntry</class>
-    <class>google.registry.model.domain.secdns.DelegationSignerData</class>
-    <class>google.registry.model.domain.GracePeriod</class>
    <class>google.registry.model.poll.PollMessage</class>
    <class>google.registry.model.poll.PollMessage$OneTime</class>
    <class>google.registry.model.poll.PollMessage$Autorenew</class>
+    <class>google.registry.model.registrar.Registrar</class>
+    <class>google.registry.model.registrar.RegistrarContact</class>
+    <class>google.registry.model.registry.label.PremiumList</class>
    <class>google.registry.model.registry.label.ReservedList</class>
+    <class>google.registry.model.registry.Registry</class>
+    <class>google.registry.model.reporting.DomainTransactionRecord</class>
+    <class>google.registry.model.reporting.Spec11ThreatMatch</class>
+    <class>google.registry.model.tmch.ClaimsListShard</class>
+    <class>google.registry.persistence.transaction.TransactionEntity</class>
+    <class>google.registry.schema.cursor.Cursor</class>
+    <class>google.registry.schema.domain.RegistryLock</class>
+    <class>google.registry.schema.server.Lock</class>
+    <class>google.registry.schema.tld.PremiumEntry</class>

    <!-- Customized type converters -->
    <class>google.registry.persistence.converter.AllocationTokenStatusTransitionConverter</class>
@@ -81,7 +83,7 @@
    <class>google.registry.persistence.converter.PostalInfoChoiceListConverter</class>
    <class>google.registry.persistence.converter.PremiumListKeyConverter</class>
    <class>google.registry.persistence.converter.RegistrarPocSetConverter</class>
-    <class>google.registry.persistence.converter.ReservedListKeyConverter</class>
+    <class>google.registry.persistence.converter.ReservedListKeySetConverter</class>
    <class>google.registry.persistence.converter.Spec11ThreatMatchThreatTypeSetConverter</class>
    <class>google.registry.persistence.converter.StatusValueSetConverter</class>
    <class>google.registry.persistence.converter.StringListConverter</class>
@@ -92,7 +94,10 @@
    <class>google.registry.persistence.converter.ZonedDateTimeConverter</class>

    <!-- Generated converters for VKey -->
-    <class>google.registry.model.billing.VKeyConverter_BillingEvent</class>
+    <class>google.registry.model.billing.VKeyConverter_Cancellation</class>
+    <class>google.registry.model.billing.VKeyConverter_Modification</class>
+    <class>google.registry.model.billing.VKeyConverter_OneTime</class>
+    <class>google.registry.model.billing.VKeyConverter_Recurring</class>
    <class>google.registry.model.contact.VKeyConverter_ContactResource</class>
    <class>google.registry.model.domain.VKeyConverter_DomainBase</class>
    <class>google.registry.model.domain.token.VKeyConverter_AllocationToken</class>
@@ -273,8 +273,11 @@ class InitSqlPipelineTest {
            "revisions",
            "updateTimestamp",
            "autorenewBillingEvent",
+            "autorenewBillingEventHistoryId",
            "autorenewPollMessage",
+            "autorenewPollMessageHistoryId",
            "deletePollMessage",
+            "deletePollMessageHistoryId",
            "nsHosts",
            "transferData");
    assertThat(actual.getAdminContact().getSqlKey())
@@ -85,7 +85,7 @@ class WriteToSqlTest implements Serializable {
      // Required for contacts created below.
      Registrar ofyRegistrar = AppEngineExtension.makeRegistrar2();
      store.insertOrUpdate(ofyRegistrar);
-      jpaTm().transact(() -> jpaTm().saveNewOrUpdate(store.loadAsOfyEntity(ofyRegistrar)));
+      jpaTm().transact(() -> jpaTm().put(store.loadAsOfyEntity(ofyRegistrar)));

      ImmutableList.Builder<Entity> builder = new ImmutableList.Builder<>();

@@ -285,7 +285,7 @@ class Spec11PipelineTest {
            .build();

    verify(mockJpaTm).transact(any(Runnable.class));
-    verify(mockJpaTm).saveNew(expected);
+    verify(mockJpaTm).insert(expected);
    verifyNoMoreInteractions(mockJpaTm);
  }

@@ -821,10 +821,12 @@ class DomainCreateFlowTest extends ResourceFlowTestCase<DomainCreateFlow, Domain
    setEppInput("domain_create_dsdata_no_maxsiglife.xml");
    persistContactsAndHosts("tld"); // For some reason this sample uses "tld".
    doSuccessfulTest("tld");
+    DomainBase domain = reloadResourceByForeignKey();
    assertAboutDomains()
-        .that(reloadResourceByForeignKey())
+        .that(domain)
        .hasExactlyDsData(
-            DelegationSignerData.create(12345, 3, 1, base16().decode("49FD46E6C4B45C55D4AC")));
+            DelegationSignerData.create(12345, 3, 1, base16().decode("49FD46E6C4B45C55D4AC"))
+                .cloneWithDomainRepoId(domain.getRepoId()));
  }

  @Test
@@ -597,6 +597,7 @@ class DomainRenewFlowTest extends ResourceFlowTestCase<DomainRenewFlow, DomainBa
            .setRestoreBillingCost(Money.of(EUR, 11))
            .setRenewBillingCostTransitions(ImmutableSortedMap.of(START_OF_TIME, Money.of(EUR, 7)))
            .setEapFeeSchedule(ImmutableSortedMap.of(START_OF_TIME, Money.zero(EUR)))
+            .setRegistryLockOrUnlockBillingCost(Money.of(EUR, 20))
            .setServerStatusChangeBillingCost(Money.of(EUR, 19))
            .build());
    persistDomain();
@@ -615,6 +616,7 @@ class DomainRenewFlowTest extends ResourceFlowTestCase<DomainRenewFlow, DomainBa
            .setRestoreBillingCost(Money.of(EUR, 11))
            .setRenewBillingCostTransitions(ImmutableSortedMap.of(START_OF_TIME, Money.of(EUR, 7)))
            .setEapFeeSchedule(ImmutableSortedMap.of(START_OF_TIME, Money.zero(EUR)))
+            .setRegistryLockOrUnlockBillingCost(Money.of(EUR, 20))
            .setServerStatusChangeBillingCost(Money.of(EUR, 19))
            .build());
    persistDomain();
@@ -633,6 +635,7 @@ class DomainRenewFlowTest extends ResourceFlowTestCase<DomainRenewFlow, DomainBa
            .setRestoreBillingCost(Money.of(EUR, 11))
            .setRenewBillingCostTransitions(ImmutableSortedMap.of(START_OF_TIME, Money.of(EUR, 7)))
            .setEapFeeSchedule(ImmutableSortedMap.of(START_OF_TIME, Money.zero(EUR)))
+            .setRegistryLockOrUnlockBillingCost(Money.of(EUR, 20))
            .setServerStatusChangeBillingCost(Money.of(EUR, 19))
            .build());
    persistDomain();
@@ -533,6 +533,7 @@ class DomainRestoreRequestFlowTest
            .setRenewBillingCostTransitions(ImmutableSortedMap.of(START_OF_TIME, Money.of(EUR, 7)))
            .setEapFeeSchedule(ImmutableSortedMap.of(START_OF_TIME, Money.zero(EUR)))
            .setServerStatusChangeBillingCost(Money.of(EUR, 19))
+            .setRegistryLockOrUnlockBillingCost(Money.of(EUR, 0))
            .build());
    EppException thrown = assertThrows(CurrencyUnitMismatchException.class, this::runFlow);
    assertAboutEppExceptions().that(thrown).marshalsToXml();
@@ -14,6 +14,7 @@

 package google.registry.flows.domain;

+import static com.google.common.collect.ImmutableSet.toImmutableSet;
 import static com.google.common.collect.Sets.union;
 import static com.google.common.io.BaseEncoding.base16;
 import static com.google.common.truth.Truth.assertThat;
@@ -470,7 +471,11 @@ class DomainUpdateFlowTest extends ResourceFlowTestCase<DomainUpdateFlow, Domain
        .that(resource)
        .hasOnlyOneHistoryEntryWhich()
        .hasType(HistoryEntry.Type.DOMAIN_UPDATE);
-    assertThat(resource.getDsData()).isEqualTo(expectedDsData);
+    assertThat(resource.getDsData())
+        .isEqualTo(
+            expectedDsData.stream()
+                .map(ds -> ds.cloneWithDomainRepoId(resource.getRepoId()))
+                .collect(toImmutableSet()));
    assertDnsTasksEnqueued("example.tld");
  }

@@ -32,17 +32,25 @@ import javax.annotation.Nullable;
 /** Truth subject for asserting things about ImmutableObjects that are not built in. */
 public final class ImmutableObjectSubject extends Subject {

-  private final ImmutableObject actual;
+  @Nullable private final ImmutableObject actual;

-  protected ImmutableObjectSubject(FailureMetadata failureMetadata, ImmutableObject actual) {
+  protected ImmutableObjectSubject(
+      FailureMetadata failureMetadata, @Nullable ImmutableObject actual) {
    super(failureMetadata, actual);
    this.actual = actual;
  }

-  public void isEqualExceptFields(ImmutableObject expected, String... ignoredFields) {
-    Map<Field, Object> actualFields = filterFields(actual, ignoredFields);
-    Map<Field, Object> expectedFields = filterFields(expected, ignoredFields);
-    assertThat(actualFields).containsExactlyEntriesIn(expectedFields);
+  public void isEqualExceptFields(@Nullable ImmutableObject expected, String... ignoredFields) {
+    if (actual == null) {
+      assertThat(expected).isNull();
+    } else {
+      assertThat(expected).isNotNull();
+    }
+    if (actual != null) {
+      Map<Field, Object> actualFields = filterFields(actual, ignoredFields);
+      Map<Field, Object> expectedFields = filterFields(expected, ignoredFields);
+      assertThat(actualFields).containsExactlyEntriesIn(expectedFields);
+    }
  }

  public static Correspondence<ImmutableObject, ImmutableObject> immutableObjectCorrespondence(
@@ -39,7 +39,6 @@ import google.registry.model.domain.rgp.GracePeriodStatus;
 import google.registry.model.domain.token.AllocationToken;
 import google.registry.model.domain.token.AllocationToken.TokenStatus;
 import google.registry.model.reporting.HistoryEntry;
-import google.registry.persistence.VKey;
 import google.registry.util.DateTimeUtils;
 import org.joda.money.Money;
 import org.joda.time.DateTime;
@@ -57,7 +56,6 @@ public class BillingEventTest extends EntityTestCase {
  private HistoryEntry historyEntry;
  private HistoryEntry historyEntry2;
  private DomainBase domain;
-  private BillingEvent.OneTime sqlOneTime;
  private BillingEvent.OneTime oneTime;
  private BillingEvent.OneTime oneTimeSynthetic;
  private BillingEvent.Recurring recurring;
@@ -107,16 +105,6 @@ public class BillingEventTest extends EntityTestCase {
                    .setBillingTime(now.plusDays(5))
                    .setAllocationToken(allocationToken.createVKey())));

-    sqlOneTime =
-        oneTime
-            .asBuilder()
-            .setDomainRepoId(domain.getRepoId())
-            .setDomainHistoryRevisionId(1L)
-            .setAllocationToken(
-                VKey.create(
-                    AllocationToken.class, allocationToken.getToken(), Key.create(allocationToken)))
-            .build();
-
    recurring =
        persistResource(
            commonInit(
@@ -179,80 +167,43 @@ public class BillingEventTest extends EntityTestCase {
  }

  private void saveNewBillingEvent(BillingEvent billingEvent) {
-    billingEvent.id = null;
-    jpaTm().transact(() -> jpaTm().saveNew(billingEvent));
+    jpaTm().transact(() -> jpaTm().insert(billingEvent));
  }

  @Test
  void testCloudSqlPersistence_OneTime() {
    saveRegistrar("a registrar");
-    saveNewBillingEvent(sqlOneTime);
+    saveNewBillingEvent(oneTime);

-    BillingEvent.OneTime persisted =
-        jpaTm()
-            .transact(
-                () -> jpaTm().load(VKey.createSql(BillingEvent.OneTime.class, sqlOneTime.id)));
-    // TODO(shicong): Remove these fixes after the entities are fully compatible
+    BillingEvent.OneTime persisted = jpaTm().transact(() -> jpaTm().load(oneTime.createVKey()));
+    // TODO(b/168325240): Remove this fix after VKeyConverter generates symmetric key for
+    // AllocationToken.
    BillingEvent.OneTime fixed =
-        persisted
-            .asBuilder()
-            .setParent(sqlOneTime.getParentKey())
-            .setAllocationToken(sqlOneTime.getAllocationToken().get())
-            .build();
-    assertThat(fixed).isEqualTo(sqlOneTime);
+        persisted.asBuilder().setAllocationToken(oneTime.getAllocationToken().get()).build();
+    assertThat(fixed).isEqualTo(oneTime);
  }

  @Test
  void testCloudSqlPersistence_Cancellation() {
    saveRegistrar("a registrar");
-    saveNewBillingEvent(sqlOneTime);
-    VKey<BillingEvent.OneTime> sqlVKey = VKey.createSql(BillingEvent.OneTime.class, sqlOneTime.id);
-    BillingEvent sqlCancellationOneTime =
-        cancellationOneTime
-            .asBuilder()
-            .setOneTimeEventKey(sqlVKey)
-            .setDomainRepoId(domain.getRepoId())
-            .setDomainHistoryRevisionId(1L)
-            .build();
-    saveNewBillingEvent(sqlCancellationOneTime);
+    saveNewBillingEvent(oneTime);
+    saveNewBillingEvent(cancellationOneTime);

    BillingEvent.Cancellation persisted =
-        jpaTm()
-            .transact(
-                () ->
-                    jpaTm()
-                        .load(
-                            VKey.createSql(
-                                BillingEvent.Cancellation.class, sqlCancellationOneTime.id)));
-    // TODO(shicong): Remove these fixes after the entities are fully compatible
+        jpaTm().transact(() -> jpaTm().load(cancellationOneTime.createVKey()));
+    // TODO(b/168537779): Remove this fix after VKey<OneTime> can be reconstructed correctly.
    BillingEvent.Cancellation fixed =
-        persisted
-            .asBuilder()
-            .setParent(sqlCancellationOneTime.getParentKey())
-            .setOneTimeEventKey(sqlVKey)
-            .build();
-    assertThat(fixed).isEqualTo(sqlCancellationOneTime);
+        persisted.asBuilder().setOneTimeEventKey(oneTime.createVKey()).build();
+    assertThat(fixed).isEqualTo(cancellationOneTime);
  }

  @Test
  void testCloudSqlPersistence_Recurring() {
    saveRegistrar("a registrar");
-    BillingEvent.Recurring sqlRecurring =
-        recurring
-            .asBuilder()
-            .setDomainRepoId(domain.getRepoId())
-            .setDomainHistoryRevisionId(1L)
-            .build();
-    saveNewBillingEvent(sqlRecurring);
+    saveNewBillingEvent(recurring);

-    BillingEvent.Recurring persisted =
-        jpaTm()
-            .transact(
-                () -> jpaTm().load(VKey.createSql(BillingEvent.Recurring.class, sqlRecurring.id)));
-    // TODO(shicong): Remove these fixes after the entities are fully compatible
-    BillingEvent.Recurring fixed =
-        persisted.asBuilder().setParent(sqlRecurring.getParentKey()).build();
-    assertThat(fixed).isEqualTo(sqlRecurring);
+    BillingEvent.Recurring persisted = jpaTm().transact(() -> jpaTm().load(recurring.createVKey()));
+    assertThat(persisted).isEqualTo(recurring);
  }

  @Test
@@ -124,7 +124,7 @@ public class ContactResourceTest extends EntityTestCase {

  @Test
  void testCloudSqlPersistence_failWhenViolateForeignKeyConstraint() {
-    assertThrowForeignKeyViolation(() -> jpaTm().transact(() -> jpaTm().saveNew(originalContact)));
+    assertThrowForeignKeyViolation(() -> jpaTm().transact(() -> jpaTm().insert(originalContact)));
  }

  @Test
@@ -134,7 +134,7 @@ public class ContactResourceTest extends EntityTestCase {
    saveRegistrar("registrar3");
    saveRegistrar("gaining");
    saveRegistrar("losing");
-    jpaTm().transact(() -> jpaTm().saveNew(originalContact));
+    jpaTm().transact(() -> jpaTm().insert(originalContact));
    ContactResource persisted =
        jpaTm()
            .transact(
@@ -25,6 +25,12 @@ import static org.joda.time.DateTimeZone.UTC;
 import static org.junit.jupiter.api.Assertions.fail;

 import com.google.common.collect.ImmutableSet;
+import com.google.common.collect.Sets;
+import com.googlecode.objectify.Key;
+import google.registry.model.billing.BillingEvent;
+import google.registry.model.billing.BillingEvent.Flag;
+import google.registry.model.billing.BillingEvent.Reason;
+import google.registry.model.common.EntityGroupRoot;
 import google.registry.model.contact.ContactResource;
 import google.registry.model.domain.DesignatedContact.Type;
 import google.registry.model.domain.launch.LaunchNotice;
@@ -33,6 +39,8 @@ import google.registry.model.domain.secdns.DelegationSignerData;
 import google.registry.model.eppcommon.AuthInfo.PasswordAuth;
 import google.registry.model.eppcommon.StatusValue;
 import google.registry.model.host.HostResource;
+import google.registry.model.poll.PollMessage;
+import google.registry.model.reporting.HistoryEntry;
 import google.registry.model.transfer.ContactTransferData;
 import google.registry.persistence.VKey;
 import google.registry.persistence.transaction.JpaTestRules;
@@ -60,6 +68,7 @@ public class DomainBaseSqlTest {
      new JpaTestRules.Builder().withClock(fakeClock).buildIntegrationWithCoverageExtension();

  private DomainBase domain;
+  private DomainHistory historyEntry;
  private VKey<ContactResource> contactKey;
  private VKey<ContactResource> contact2Key;
  private VKey<HostResource> host1VKey;
@@ -139,9 +148,9 @@ public class DomainBaseSqlTest {
                .transact(
                    () -> {
                      // Persist the domain without the associated host object.
-                      jpaTm().saveNew(contact);
-                      jpaTm().saveNew(contact2);
-                      jpaTm().saveNew(domain);
+                      jpaTm().insert(contact);
+                      jpaTm().insert(contact2);
+                      jpaTm().insert(domain);
                    }));
  }

@@ -153,8 +162,8 @@ public class DomainBaseSqlTest {
                .transact(
                    () -> {
                      // Persist the domain without the associated contact objects.
-                      jpaTm().saveNew(domain);
-                      jpaTm().saveNew(host);
+                      jpaTm().insert(domain);
+                      jpaTm().insert(host);
                    }));
  }

@@ -165,7 +174,7 @@ public class DomainBaseSqlTest {
        .transact(
            () -> {
              DomainBase persisted = jpaTm().load(domain.createVKey());
-              jpaTm().saveNewOrUpdate(persisted.asBuilder().build());
+              jpaTm().put(persisted.asBuilder().build());
            });
    jpaTm()
        .transact(
@@ -185,7 +194,7 @@ public class DomainBaseSqlTest {
              DomainBase persisted = jpaTm().load(domain.createVKey());
              DomainBase modified =
                  persisted.asBuilder().setGracePeriods(ImmutableSet.of()).build();
-              jpaTm().saveNewOrUpdate(modified);
+              jpaTm().put(modified);
            });

    jpaTm()
@@ -204,7 +213,7 @@ public class DomainBaseSqlTest {
            () -> {
              DomainBase persisted = jpaTm().load(domain.createVKey());
              DomainBase modified = persisted.asBuilder().setGracePeriods(null).build();
-              jpaTm().saveNewOrUpdate(modified);
+              jpaTm().put(modified);
            });

    jpaTm()
@@ -229,7 +238,7 @@ public class DomainBaseSqlTest {
                          GracePeriod.create(
                              GracePeriodStatus.RENEW, "4-COM", END_OF_TIME, "registrar1", null))
                      .build();
-              jpaTm().saveNewOrUpdate(modified);
+              jpaTm().put(modified);
            });

    jpaTm()
@@ -281,7 +290,7 @@ public class DomainBaseSqlTest {
                  builder.removeGracePeriod(gracePeriod);
                }
              }
-              jpaTm().saveNewOrUpdate(builder.build());
+              jpaTm().put(builder.build());
            });

    jpaTm()
@@ -301,7 +310,7 @@ public class DomainBaseSqlTest {
              DomainBase persisted = jpaTm().load(domain.createVKey());
              DomainBase modified =
                  persisted.asBuilder().setGracePeriods(ImmutableSet.of()).build();
-              jpaTm().saveNewOrUpdate(modified);
+              jpaTm().put(modified);
            });

    jpaTm()
@@ -316,7 +325,7 @@ public class DomainBaseSqlTest {
                          GracePeriod.create(
                              GracePeriodStatus.ADD, "4-COM", END_OF_TIME, "registrar1", null))
                      .build();
-              jpaTm().saveNewOrUpdate(modified);
+              jpaTm().put(modified);
            });

    jpaTm()
@@ -334,32 +343,66 @@ public class DomainBaseSqlTest {
            });
  }

+  @Test
+  void testModifyDsData_addThenRemoveSuccessfully() {
+    persistDomain();
+    DelegationSignerData extraDsData =
+        DelegationSignerData.create(2, 2, 3, new byte[] {0, 1, 2}, "4-COM");
+    ImmutableSet<DelegationSignerData> unionDsData =
+        Sets.union(domain.getDsData(), ImmutableSet.of(extraDsData)).immutableCopy();
+
+    // Add an extra DelegationSignerData to dsData set.
+    jpaTm()
+        .transact(
+            () -> {
+              DomainBase persisted = jpaTm().load(domain.createVKey());
+              assertThat(persisted.getDsData()).containsExactlyElementsIn(domain.getDsData());
+              DomainBase modified = persisted.asBuilder().setDsData(unionDsData).build();
+              jpaTm().put(modified);
+            });
+
+    // Verify that the persisted domain entity contains both DelegationSignerData records.
+    jpaTm()
+        .transact(
+            () -> {
+              DomainBase persisted = jpaTm().load(domain.createVKey());
+              assertThat(persisted.getDsData()).containsExactlyElementsIn(unionDsData);
+              assertEqualDomainExcept(persisted, "dsData");
+            });
+
+    // Remove the extra DelegationSignerData record from dsData set.
+    jpaTm()
+        .transact(
+            () -> {
+              DomainBase persisted = jpaTm().load(domain.createVKey());
+              jpaTm().put(persisted.asBuilder().setDsData(domain.getDsData()).build());
+            });
+
+    // Verify that the persisted domain is equal to the original domain.
+    jpaTm()
+        .transact(
+            () -> {
+              DomainBase persisted = jpaTm().load(domain.createVKey());
+              assertEqualDomainExcept(persisted);
+            });
+  }
+
  @Test
  void testUpdates() {
    jpaTm()
        .transact(
            () -> {
-              jpaTm().saveNew(contact);
-              jpaTm().saveNew(contact2);
-              jpaTm().saveNew(domain);
-              jpaTm().saveNew(host);
+              jpaTm().insert(contact);
+              jpaTm().insert(contact2);
+              jpaTm().insert(domain);
+              jpaTm().insert(host);
            });
    domain = domain.asBuilder().setNameservers(ImmutableSet.of()).build();
-    jpaTm().transact(() -> jpaTm().saveNewOrUpdate(domain));
+    jpaTm().transact(() -> jpaTm().put(domain));
    jpaTm()
        .transact(
            () -> {
              DomainBase result = jpaTm().load(domain.createVKey());
-
-              // Fix DS data, since we can't persist that yet.
-              result =
-                  result
-                      .asBuilder()
-                      .setDsData(
-                          ImmutableSet.of(
-                              DelegationSignerData.create(1, 2, 3, new byte[] {0, 1, 2})))
-                      .build();
-
              assertAboutImmutableObjects()
                  .that(result)
                  .isEqualExceptFields(domain, "updateTimestamp", "creationTime");
@@ -382,27 +425,188 @@ public class DomainBaseSqlTest {
              // Persist the contacts.  Note that these need to be persisted before the domain
              // otherwise we get a foreign key constraint error.  If we ever decide to defer the
              // relevant foreign key checks to commit time, then the order would not matter.
-              jpaTm().saveNew(contact);
-              jpaTm().saveNew(contact2);
+              jpaTm().insert(contact);
+              jpaTm().insert(contact2);

              // Persist the domain.
-              jpaTm().saveNew(domain);
+              jpaTm().insert(domain);

              // Persist the host.  This does _not_ need to be persisted before the domain,
              // because only the row in the join table (DomainHost) is subject to foreign key
              // constraints, and Hibernate knows to insert it after domain and host.
-              jpaTm().saveNew(host);
+              jpaTm().insert(host);
            });
  }

-  private void assertEqualDomainExcept(DomainBase thatDomain, String... excepts) {
-    // Fix DS data, since we can't persist it yet.
-    thatDomain =
-        thatDomain
-            .asBuilder()
-            .setDsData(ImmutableSet.of(DelegationSignerData.create(1, 2, 3, new byte[] {0, 1, 2})))
-            .build();
+  @Test
+  void persistDomainWithCompositeVKeys() {
+    jpaTm()
+        .transact(
+            () -> {
+              historyEntry =
+                  new DomainHistory.Builder()
+                      .setType(HistoryEntry.Type.DOMAIN_CREATE)
+                      .setPeriod(Period.create(1, Period.Unit.YEARS))
+                      .setModificationTime(DateTime.now(UTC))
+                      .setParent(Key.create(DomainBase.class, "4-COM"))
+                      .setDomainRepoId("4-COM")

+                      // These are non-null, but I don't think some tests set them.
+                      .setReason("felt like it")
+                      .setRequestedByRegistrar(false)
+                      .setXmlBytes(new byte[0])
+                      .build();
+              BillingEvent.Recurring billEvent =
+                  new BillingEvent.Recurring.Builder()
+                      .setReason(Reason.RENEW)
+                      .setFlags(ImmutableSet.of(Flag.AUTO_RENEW))
+                      .setTargetId("example.com")
+                      .setClientId("registrar1")
+                      .setDomainRepoId("4-COM")
+                      .setDomainHistoryRevisionId(1L)
+                      .setEventTime(DateTime.now(UTC).plusYears(1))
+                      .setRecurrenceEndTime(END_OF_TIME)
+                      .setParent(historyEntry)
+                      .build();
+              PollMessage.Autorenew autorenewPollMessage =
+                  new PollMessage.Autorenew.Builder()
+                      .setClientId("registrar1")
+                      .setEventTime(DateTime.now(UTC).plusYears(1))
+                      .setParent(historyEntry)
+                      .build();
+              PollMessage.OneTime deletePollMessage =
+                  new PollMessage.OneTime.Builder()
+                      .setClientId("registrar1")
+                      .setEventTime(DateTime.now(UTC).plusYears(1))
+                      .setParent(historyEntry)
+                      .build();
+
+              jpaTm().insert(contact);
+              jpaTm().insert(contact2);
+              jpaTm().insert(host);
+              domain =
+                  domain
+                      .asBuilder()
+                      .setAutorenewBillingEvent(billEvent.createVKey())
+                      .setAutorenewPollMessage(autorenewPollMessage.createVKey())
+                      .setDeletePollMessage(deletePollMessage.createVKey())
+                      .build();
+              historyEntry = historyEntry.asBuilder().setDomainContent(domain).build();
+              jpaTm().insert(historyEntry);
+              jpaTm().insert(autorenewPollMessage);
+              jpaTm().insert(billEvent);
+              jpaTm().insert(deletePollMessage);
+              jpaTm().insert(domain);
+            });
+
+    // Store the existing BillingRecurrence VKey.  This happens after the event has been persisted.
+    DomainBase persisted = jpaTm().transact(() -> jpaTm().load(domain.createVKey()));
+
+    // Verify that the domain data has been persisted.
+    // dsData still isn't persisted.  gracePeriods appears to have the same values but for some
+    // reason is showing up as different.
+    assertEqualDomainExcept(persisted, "creationTime", "dsData", "gracePeriods");
+
+    // Verify that the DomainContent object from the history record sets the fields correctly.
+    DomainHistory persistedHistoryEntry =
+        jpaTm().transact(() -> jpaTm().load(historyEntry.createVKey()));
+    assertThat(persistedHistoryEntry.getDomainContent().get().getAutorenewPollMessage())
+        .isEqualTo(domain.getAutorenewPollMessage());
+    assertThat(persistedHistoryEntry.getDomainContent().get().getAutorenewBillingEvent())
+        .isEqualTo(domain.getAutorenewBillingEvent());
+    assertThat(persistedHistoryEntry.getDomainContent().get().getDeletePollMessage())
+        .isEqualTo(domain.getDeletePollMessage());
+  }
+
+  @Test
+  void persistDomainWithLegacyVKeys() {
+    jpaTm()
+        .transact(
+            () -> {
+              historyEntry =
+                  new DomainHistory.Builder()
+                      .setType(HistoryEntry.Type.DOMAIN_CREATE)
+                      .setPeriod(Period.create(1, Period.Unit.YEARS))
+                      .setModificationTime(DateTime.now(UTC))
+                      .setParent(Key.create(DomainBase.class, "4-COM"))
+                      .setDomainRepoId("4-COM")
+
+                      // These are non-null, but I don't think some tests set them.
+                      .setReason("felt like it")
+                      .setRequestedByRegistrar(false)
+                      .setXmlBytes(new byte[0])
+                      .build();
+              BillingEvent.Recurring billEvent =
+                  new BillingEvent.Recurring.Builder()
+                      .setReason(Reason.RENEW)
+                      .setFlags(ImmutableSet.of(Flag.AUTO_RENEW))
+                      .setTargetId("example.com")
+                      .setClientId("registrar1")
+                      .setDomainRepoId("4-COM")
+                      .setDomainHistoryRevisionId(1L)
+                      .setEventTime(DateTime.now(UTC).plusYears(1))
+                      .setRecurrenceEndTime(END_OF_TIME)
+                      .setParent(historyEntry)
+                      .build();
+              PollMessage.Autorenew autorenewPollMessage =
+                  new PollMessage.Autorenew.Builder()
+                      .setClientId("registrar1")
+                      .setEventTime(DateTime.now(UTC).plusYears(1))
+                      .setParent(historyEntry)
+                      .build();
+              PollMessage.OneTime deletePollMessage =
+                  new PollMessage.OneTime.Builder()
+                      .setClientId("registrar1")
+                      .setEventTime(DateTime.now(UTC).plusYears(1))
+                      .setParent(historyEntry)
+                      .build();
+
+              jpaTm().insert(contact);
+              jpaTm().insert(contact2);
+              jpaTm().insert(host);
+              domain =
+                  domain
+                      .asBuilder()
+                      .setAutorenewBillingEvent(
+                          createLegacyVKey(BillingEvent.Recurring.class, billEvent.getId()))
+                      .setAutorenewPollMessage(
+                          createLegacyVKey(PollMessage.Autorenew.class, deletePollMessage.getId()))
+                      .setDeletePollMessage(
+                          createLegacyVKey(PollMessage.OneTime.class, autorenewPollMessage.getId()))
+                      .build();
+              historyEntry = historyEntry.asBuilder().setDomainContent(domain).build();
+              jpaTm().insert(historyEntry);
+              jpaTm().insert(autorenewPollMessage);
+              jpaTm().insert(billEvent);
+              jpaTm().insert(deletePollMessage);
+              jpaTm().insert(domain);
+            });
+
+    // Store the existing BillingRecurrence VKey.  This happens after the event has been persisted.
+    DomainBase persisted = jpaTm().transact(() -> jpaTm().load(domain.createVKey()));
+
+    // Verify that the domain data has been persisted.
+    // dsData still isn't persisted.  gracePeriods appears to have the same values but for some
+    // reason is showing up as different.
+    assertEqualDomainExcept(persisted, "creationTime", "dsData", "gracePeriods");
+
+    // Verify that the DomainContent object from the history record sets the fields correctly.
+    DomainHistory persistedHistoryEntry =
+        jpaTm().transact(() -> jpaTm().load(historyEntry.createVKey()));
+    assertThat(persistedHistoryEntry.getDomainContent().get().getAutorenewPollMessage())
+        .isEqualTo(domain.getAutorenewPollMessage());
+    assertThat(persistedHistoryEntry.getDomainContent().get().getAutorenewBillingEvent())
+        .isEqualTo(domain.getAutorenewBillingEvent());
+    assertThat(persistedHistoryEntry.getDomainContent().get().getDeletePollMessage())
+        .isEqualTo(domain.getDeletePollMessage());
+  }
+
+  private <T> VKey<T> createLegacyVKey(Class<T> clazz, long id) {
+    return VKey.create(
+        clazz, id, Key.create(Key.create(EntityGroupRoot.class, "per-tld"), clazz, id));
+  }
+
+  private void assertEqualDomainExcept(DomainBase thatDomain, String... excepts) {
    // Fix the original creation timestamp (this gets initialized on first write)
    DomainBase org = domain.asBuilder().setCreationTime(thatDomain.getCreationTime()).build();

@@ -95,8 +95,8 @@ public class AllocationTokenTest extends EntityTestCase {
    jpaTm()
        .transact(
            () -> {
-              jpaTm().saveNew(unlimitedUseToken);
-              jpaTm().saveNew(singleUseToken);
+              jpaTm().insert(unlimitedUseToken);
+              jpaTm().insert(singleUseToken);
            });
    jpaTm()
        .transact(
@@ -44,19 +44,41 @@ public class ContactHistoryTest extends EntityTestCase {
    saveRegistrar("TheRegistrar");

    ContactResource contact = newContactResourceWithRoid("contactId", "contact1");
-    jpaTm().transact(() -> jpaTm().saveNew(contact));
+    jpaTm().transact(() -> jpaTm().insert(contact));
    VKey<ContactResource> contactVKey = contact.createVKey();
    ContactResource contactFromDb = jpaTm().transact(() -> jpaTm().load(contactVKey));
-    ContactHistory contactHistory = createContactHistory(contactFromDb, contactVKey);
-    contactHistory.id = null;
-    jpaTm().transact(() -> jpaTm().saveNew(contactHistory));
+    ContactHistory contactHistory = createContactHistory(contactFromDb, contact.getRepoId());
+    jpaTm().transact(() -> jpaTm().insert(contactHistory));
    jpaTm()
        .transact(
            () -> {
-              ContactHistory fromDatabase = jpaTm().load(VKey.createSql(ContactHistory.class, 1L));
+              ContactHistory fromDatabase = jpaTm().load(contactHistory.createVKey());
              assertContactHistoriesEqual(fromDatabase, contactHistory);
-              assertThat(fromDatabase.getContactRepoId().getSqlKey())
-                  .isEqualTo(contactHistory.getContactRepoId().getSqlKey());
+              assertThat(fromDatabase.getParentVKey()).isEqualTo(contactHistory.getParentVKey());
+            });
+  }
+
+  @Test
+  void testLegacyPersistence_nullContactBase() {
+    saveRegistrar("TheRegistrar");
+
+    ContactResource contact = newContactResourceWithRoid("contactId", "contact1");
+    jpaTm().transact(() -> jpaTm().insert(contact));
+    VKey<ContactResource> contactVKey = contact.createVKey();
+    ContactResource contactFromDb = jpaTm().transact(() -> jpaTm().load(contactVKey));
+    ContactHistory contactHistory =
+        createContactHistory(contactFromDb, contact.getRepoId())
+            .asBuilder()
+            .setContactBase(null)
+            .build();
+    jpaTm().transact(() -> jpaTm().insert(contactHistory));
+
+    jpaTm()
+        .transact(
+            () -> {
+              ContactHistory fromDatabase = jpaTm().load(contactHistory.createVKey());
+              assertContactHistoriesEqual(fromDatabase, contactHistory);
+              assertThat(fromDatabase.getParentVKey()).isEqualTo(contactHistory.getParentVKey());
            });
  }

@@ -65,18 +87,17 @@ public class ContactHistoryTest extends EntityTestCase {
    saveRegistrar("TheRegistrar");

    ContactResource contact = newContactResourceWithRoid("contactId", "contact1");
-    tm().transact(() -> tm().saveNew(contact));
+    tm().transact(() -> tm().insert(contact));
    VKey<ContactResource> contactVKey = contact.createVKey();
    ContactResource contactFromDb = tm().transact(() -> tm().load(contactVKey));
    fakeClock.advanceOneMilli();
-    ContactHistory contactHistory = createContactHistory(contactFromDb, contactVKey);
-    tm().transact(() -> tm().saveNew(contactHistory));
+    ContactHistory contactHistory = createContactHistory(contactFromDb, contact.getRepoId());
+    tm().transact(() -> tm().insert(contactHistory));

    // retrieving a HistoryEntry or a ContactHistory with the same key should return the same object
    // note: due to the @EntitySubclass annotation. all Keys for ContactHistory objects will have
    // type HistoryEntry
-    VKey<ContactHistory> contactHistoryVKey =
-        VKey.createOfy(ContactHistory.class, Key.create(contactHistory));
+    VKey<ContactHistory> contactHistoryVKey = contactHistory.createVKey();
    VKey<HistoryEntry> historyEntryVKey =
        VKey.createOfy(HistoryEntry.class, Key.create(contactHistory.asHistoryEntry()));
    ContactHistory hostHistoryFromDb = tm().transact(() -> tm().load(contactHistoryVKey));
@@ -85,8 +106,7 @@ public class ContactHistoryTest extends EntityTestCase {
    assertThat(hostHistoryFromDb).isEqualTo(historyEntryFromDb);
  }

-  private ContactHistory createContactHistory(
-      ContactBase contact, VKey<ContactResource> contactVKey) {
+  private ContactHistory createContactHistory(ContactBase contact, String contactRepoId) {
    return new ContactHistory.Builder()
        .setType(HistoryEntry.Type.HOST_CREATE)
        .setXmlBytes("<xml></xml>".getBytes(UTF_8))
@@ -97,16 +117,16 @@ public class ContactHistoryTest extends EntityTestCase {
        .setReason("reason")
        .setRequestedByRegistrar(true)
        .setContactBase(contact)
-        .setContactRepoId(contactVKey)
+        .setContactRepoId(contactRepoId)
        .build();
  }

  static void assertContactHistoriesEqual(ContactHistory one, ContactHistory two) {
    assertAboutImmutableObjects()
        .that(one)
-        .isEqualExceptFields(two, "contactBase", "contactRepoId", "parent");
+        .isEqualExceptFields(two, "contactBase", "contactRepoId");
    assertAboutImmutableObjects()
-        .that(one.getContactBase())
-        .isEqualExceptFields(two.getContactBase(), "repoId");
+        .that(one.getContactBase().orElse(null))
+        .isEqualExceptFields(two.getContactBase().orElse(null), "repoId");
  }
 }
@@ -17,6 +17,7 @@ package google.registry.model.history;
 import static com.google.common.collect.ImmutableSet.toImmutableSet;
 import static com.google.common.truth.Truth.assertThat;
 import static google.registry.model.ImmutableObjectSubject.assertAboutImmutableObjects;
+import static google.registry.model.ImmutableObjectSubject.immutableObjectCorrespondence;
 import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
 import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
 import static google.registry.testing.DatastoreHelper.newContactResourceWithRoid;
@@ -25,16 +26,21 @@ import static google.registry.testing.DatastoreHelper.newHostResourceWithRoid;
 import static google.registry.testing.SqlHelper.saveRegistrar;
 import static java.nio.charset.StandardCharsets.UTF_8;

+import com.google.common.collect.ImmutableSet;
 import com.googlecode.objectify.Key;
 import google.registry.model.EntityTestCase;
 import google.registry.model.contact.ContactResource;
 import google.registry.model.domain.DomainBase;
 import google.registry.model.domain.DomainContent;
 import google.registry.model.domain.DomainHistory;
+import google.registry.model.domain.Period;
 import google.registry.model.eppcommon.Trid;
 import google.registry.model.host.HostResource;
+import google.registry.model.reporting.DomainTransactionRecord;
+import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
 import google.registry.model.reporting.HistoryEntry;
 import google.registry.persistence.VKey;
+import org.junit.jupiter.api.BeforeEach;
 import org.junit.jupiter.api.Test;

 /** Tests for {@link DomainHistory}. */
@@ -44,38 +50,39 @@ public class DomainHistoryTest extends EntityTestCase {
    super(JpaEntityCoverageCheck.ENABLED);
  }

+  @BeforeEach
+  void beforeEach() {
+    saveRegistrar("TheRegistrar");
+  }
+
  @Test
  void testPersistence() {
-    saveRegistrar("TheRegistrar");
-
-    HostResource host = newHostResourceWithRoid("ns1.example.com", "host1");
-    ContactResource contact = newContactResourceWithRoid("contactId", "contact1");
-
-    jpaTm()
-        .transact(
-            () -> {
-              jpaTm().saveNew(host);
-              jpaTm().saveNew(contact);
-            });
-
-    DomainBase domain =
-        newDomainBase("example.tld", "domainRepoId", contact)
-            .asBuilder()
-            .setNameservers(host.createVKey())
-            .build();
-    jpaTm().transact(() -> jpaTm().saveNew(domain));
-
+    DomainBase domain = createDomainWithContactsAndHosts();
    DomainHistory domainHistory = createDomainHistory(domain);
-    domainHistory.id = null;
-    jpaTm().transact(() -> jpaTm().saveNew(domainHistory));
+    jpaTm().transact(() -> jpaTm().insert(domainHistory));

    jpaTm()
        .transact(
            () -> {
              DomainHistory fromDatabase = jpaTm().load(domainHistory.createVKey());
              assertDomainHistoriesEqual(fromDatabase, domainHistory);
-              assertThat(fromDatabase.getDomainRepoId().getSqlKey())
-                  .isEqualTo(domainHistory.getDomainRepoId().getSqlKey());
+              assertThat(fromDatabase.getParentVKey()).isEqualTo(domainHistory.getParentVKey());
+            });
+  }
+
+  @Test
+  void testLegacyPersistence_nullResource() {
+    DomainBase domain = createDomainWithContactsAndHosts();
+    DomainHistory domainHistory =
+        createDomainHistory(domain).asBuilder().setDomainContent(null).build();
+    jpaTm().transact(() -> jpaTm().insert(domainHistory));
+
+    jpaTm()
+        .transact(
+            () -> {
+              DomainHistory fromDatabase = jpaTm().load(domainHistory.createVKey());
+              assertDomainHistoriesEqual(fromDatabase, domainHistory);
+              assertThat(fromDatabase.getParentVKey()).isEqualTo(domainHistory.getParentVKey());
              assertThat(fromDatabase.getNsHosts())
                  .containsExactlyElementsIn(
                      domainHistory.getNsHosts().stream()
@@ -86,15 +93,13 @@ public class DomainHistoryTest extends EntityTestCase {

  @Test
  void testOfyPersistence() {
-    saveRegistrar("TheRegistrar");
-
    HostResource host = newHostResourceWithRoid("ns1.example.com", "host1");
    ContactResource contact = newContactResourceWithRoid("contactId", "contact1");

    tm().transact(
            () -> {
-              tm().saveNew(host);
-              tm().saveNew(contact);
+              tm().insert(host);
+              tm().insert(contact);
            });
    fakeClock.advanceOneMilli();

@@ -103,17 +108,16 @@ public class DomainHistoryTest extends EntityTestCase {
            .asBuilder()
            .setNameservers(host.createVKey())
            .build();
-    tm().transact(() -> tm().saveNew(domain));
+    tm().transact(() -> tm().insert(domain));

    fakeClock.advanceOneMilli();
    DomainHistory domainHistory = createDomainHistory(domain);
-    tm().transact(() -> tm().saveNew(domainHistory));
+    tm().transact(() -> tm().insert(domainHistory));

    // retrieving a HistoryEntry or a DomainHistory with the same key should return the same object
-    // note: due to the @EntitySubclass annotation. all Keys for ContactHistory objects will have
+    // note: due to the @EntitySubclass annotation. all Keys for DomainHistory objects will have
    // type HistoryEntry
-    VKey<DomainHistory> domainHistoryVKey =
-        VKey.createOfy(DomainHistory.class, Key.create(domainHistory));
+    VKey<DomainHistory> domainHistoryVKey = domainHistory.createVKey();
    VKey<HistoryEntry> historyEntryVKey =
        VKey.createOfy(HistoryEntry.class, Key.create(domainHistory.asHistoryEntry()));
    DomainHistory domainHistoryFromDb = tm().transact(() -> tm().load(domainHistoryVKey));
@@ -122,13 +126,49 @@ public class DomainHistoryTest extends EntityTestCase {
    assertThat(domainHistoryFromDb).isEqualTo(historyEntryFromDb);
  }

+  static DomainBase createDomainWithContactsAndHosts() {
+    HostResource host = newHostResourceWithRoid("ns1.example.com", "host1");
+    ContactResource contact = newContactResourceWithRoid("contactId", "contact1");
+
+    jpaTm()
+        .transact(
+            () -> {
+              jpaTm().insert(host);
+              jpaTm().insert(contact);
+            });
+
+    DomainBase domain =
+        newDomainBase("example.tld", "domainRepoId", contact)
+            .asBuilder()
+            .setNameservers(host.createVKey())
+            .build();
+    jpaTm().transact(() -> jpaTm().insert(domain));
+    return domain;
+  }
+
  static void assertDomainHistoriesEqual(DomainHistory one, DomainHistory two) {
    assertAboutImmutableObjects()
        .that(one)
-        .isEqualExceptFields(two, "domainContent", "domainRepoId", "parent", "nsHosts");
+        .isEqualExceptFields(
+            two, "domainContent", "domainRepoId", "nsHosts", "domainTransactionRecords");
+    assertThat(one.getDomainContent().map(DomainContent::getDomainName))
+        .isEqualTo(two.getDomainContent().map(DomainContent::getDomainName));
+    // NB: the record's ID gets reset by Hibernate, causing the hash code to differ so we have to
+    // compare it separately
+    assertThat(one.getDomainTransactionRecords())
+        .comparingElementsUsing(immutableObjectCorrespondence())
+        .containsExactlyElementsIn(two.getDomainTransactionRecords());
  }

  private DomainHistory createDomainHistory(DomainContent domain) {
+    DomainTransactionRecord transactionRecord =
+        new DomainTransactionRecord.Builder()
+            .setTld("tld")
+            .setReportingTime(fakeClock.nowUtc())
+            .setReportField(TransactionReportField.NET_ADDS_1_YR)
+            .setReportAmount(1)
+            .build();
+
    return new DomainHistory.Builder()
        .setType(HistoryEntry.Type.DOMAIN_CREATE)
        .setXmlBytes("<xml></xml>".getBytes(UTF_8))
@@ -140,6 +180,9 @@ public class DomainHistoryTest extends EntityTestCase {
        .setRequestedByRegistrar(true)
        .setDomainContent(domain)
        .setDomainRepoId(domain.getRepoId())
+        .setDomainTransactionRecords(ImmutableSet.of(transactionRecord))
+        .setOtherClientId("otherClient")
+        .setPeriod(Period.create(1, Period.Unit.YEARS))
        .build();
  }
 }
@@ -1,4 +1,4 @@
-// Copyright 2017 The Nomulus Authors. All Rights Reserved.
+// Copyright 2020 The Nomulus Authors. All Rights Reserved.
 //
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
@@ -44,39 +44,58 @@ public class HostHistoryTest extends EntityTestCase {
    saveRegistrar("TheRegistrar");

    HostResource host = newHostResourceWithRoid("ns1.example.com", "host1");
-    jpaTm().transact(() -> jpaTm().saveNew(host));
-    VKey<HostResource> hostVKey = VKey.createSql(HostResource.class, "host1");
+    jpaTm().transact(() -> jpaTm().insert(host));
+    VKey<HostResource> hostVKey =
+        VKey.create(HostResource.class, "host1", Key.create(HostResource.class, "host1"));
    HostResource hostFromDb = jpaTm().transact(() -> jpaTm().load(hostVKey));
-    HostHistory hostHistory = createHostHistory(hostFromDb, hostVKey);
-    hostHistory.id = null;
-    jpaTm().transact(() -> jpaTm().saveNew(hostHistory));
+    HostHistory hostHistory = createHostHistory(hostFromDb, host.getRepoId());
+    jpaTm().transact(() -> jpaTm().insert(hostHistory));
    jpaTm()
        .transact(
            () -> {
-              HostHistory fromDatabase =
-                  jpaTm().load(VKey.createSql(HostHistory.class, hostHistory.getId()));
+              HostHistory fromDatabase = jpaTm().load(hostHistory.createVKey());
              assertHostHistoriesEqual(fromDatabase, hostHistory);
-              assertThat(fromDatabase.getHostRepoId().getSqlKey())
-                  .isEqualTo(hostHistory.getHostRepoId().getSqlKey());
+              assertThat(fromDatabase.getParentVKey()).isEqualTo(hostHistory.getParentVKey());
            });
  }

  @Test
-  public void testOfySave() {
+  void testLegacyPersistence_nullHostBase() {
+    saveRegistrar("TheRegistrar");
+    HostResource host = newHostResourceWithRoid("ns1.example.com", "host1");
+    jpaTm().transact(() -> jpaTm().insert(host));
+
+    HostResource hostFromDb = jpaTm().transact(() -> jpaTm().load(host.createVKey()));
+    HostHistory hostHistory =
+        createHostHistory(hostFromDb, host.getRepoId()).asBuilder().setHostBase(null).build();
+    jpaTm().transact(() -> jpaTm().insert(hostHistory));
+
+    jpaTm()
+        .transact(
+            () -> {
+              HostHistory fromDatabase = jpaTm().load(hostHistory.createVKey());
+              assertHostHistoriesEqual(fromDatabase, hostHistory);
+              assertThat(fromDatabase.getParentVKey()).isEqualTo(hostHistory.getParentVKey());
+            });
+  }
+
+  @Test
+  void testOfySave() {
    saveRegistrar("registrar1");

    HostResource host = newHostResourceWithRoid("ns1.example.com", "host1");
-    tm().transact(() -> tm().saveNew(host));
-    VKey<HostResource> hostVKey = VKey.create(HostResource.class, "host1", Key.create(host));
+    tm().transact(() -> tm().insert(host));
+    VKey<HostResource> hostVKey =
+        VKey.create(HostResource.class, "host1", Key.create(HostResource.class, "host1"));
    HostResource hostFromDb = tm().transact(() -> tm().load(hostVKey));
-    HostHistory hostHistory = createHostHistory(hostFromDb, hostVKey);
+    HostHistory hostHistory = createHostHistory(hostFromDb, host.getRepoId());
    fakeClock.advanceOneMilli();
-    tm().transact(() -> tm().saveNew(hostHistory));
+    tm().transact(() -> tm().insert(hostHistory));

    // retrieving a HistoryEntry or a HostHistory with the same key should return the same object
-    // note: due to the @EntitySubclass annotation. all Keys for ContactHistory objects will have
+    // note: due to the @EntitySubclass annotation. all Keys for HostHistory objects will have
    // type HistoryEntry
-    VKey<HostHistory> hostHistoryVKey = VKey.createOfy(HostHistory.class, Key.create(hostHistory));
+    VKey<HostHistory> hostHistoryVKey = hostHistory.createVKey();
    VKey<HistoryEntry> historyEntryVKey =
        VKey.createOfy(HistoryEntry.class, Key.create(hostHistory.asHistoryEntry()));
    HostHistory hostHistoryFromDb = tm().transact(() -> tm().load(hostHistoryVKey));
@@ -88,11 +107,11 @@ public class HostHistoryTest extends EntityTestCase {
  private void assertHostHistoriesEqual(HostHistory one, HostHistory two) {
    assertAboutImmutableObjects().that(one).isEqualExceptFields(two, "hostBase");
    assertAboutImmutableObjects()
-        .that(one.getHostBase())
-        .isEqualExceptFields(two.getHostBase(), "repoId");
+        .that(one.getHostBase().orElse(null))
+        .isEqualExceptFields(two.getHostBase().orElse(null), "repoId");
  }

-  private HostHistory createHostHistory(HostBase hostBase, VKey<HostResource> hostVKey) {
+  private HostHistory createHostHistory(HostBase hostBase, String hostRepoId) {
    return new HostHistory.Builder()
        .setType(HistoryEntry.Type.HOST_CREATE)
        .setXmlBytes("<xml></xml>".getBytes(UTF_8))
@@ -103,7 +122,7 @@ public class HostHistoryTest extends EntityTestCase {
        .setReason("reason")
        .setRequestedByRegistrar(true)
        .setHostBase(hostBase)
-        .setHostRepoId(hostVKey)
+        .setHostRepoId(hostRepoId)
        .build();
  }
 }
@@ -0,0 +1,203 @@
+// Copyright 2020 The Nomulus Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package google.registry.model.history;
+
+import static com.google.common.truth.Truth.assertThat;
+import static google.registry.model.ImmutableObjectSubject.assertAboutImmutableObjects;
+import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
+import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
+import static google.registry.testing.DatastoreHelper.newContactResourceWithRoid;
+import static google.registry.testing.DatastoreHelper.newHostResourceWithRoid;
+import static google.registry.testing.SqlHelper.saveRegistrar;
+import static google.registry.util.CollectionUtils.nullToEmpty;
+import static java.nio.charset.StandardCharsets.UTF_8;
+
+import com.google.common.collect.ImmutableSet;
+import com.googlecode.objectify.Key;
+import google.registry.model.EntityTestCase;
+import google.registry.model.EppResource;
+import google.registry.model.contact.ContactHistory;
+import google.registry.model.contact.ContactResource;
+import google.registry.model.domain.DomainBase;
+import google.registry.model.domain.DomainHistory;
+import google.registry.model.domain.Period;
+import google.registry.model.eppcommon.Trid;
+import google.registry.model.host.HostHistory;
+import google.registry.model.host.HostResource;
+import google.registry.model.reporting.DomainTransactionRecord;
+import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
+import google.registry.model.reporting.HistoryEntry;
+import google.registry.persistence.VKey;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+/** Tests to check {@link HistoryEntry} + its subclasses' transitions to/from Datastore/SQL. */
+public class LegacyHistoryObjectTest extends EntityTestCase {
+
+  public LegacyHistoryObjectTest() {
+    super(JpaEntityCoverageCheck.ENABLED);
+  }
+
+  @BeforeEach
+  void beforeEach() {
+    saveRegistrar("TheRegistrar");
+  }
+
+  @Test
+  void testFullConversion_contact() {
+    // Create+save an old contact HistoryEntry, reload it, and verify it's a proper ContactHistory
+    ContactResource contact = newContactResourceWithRoid("contactId", "contact1");
+    HistoryEntry legacyHistoryEntry = historyEntryBuilderFor(contact).build();
+    tm().transact(() -> tm().insert(legacyHistoryEntry));
+
+    // In Datastore, we will save it as HistoryEntry but retrieve it as ContactHistory
+    long historyEntryId = legacyHistoryEntry.getId();
+    HistoryEntry fromObjectify =
+        tm().transact(
+                () ->
+                    tm().load(
+                            VKey.create(
+                                HistoryEntry.class,
+                                historyEntryId,
+                                Key.create(legacyHistoryEntry))));
+    // The objects will be mostly the same, but the ContactHistory object has a couple extra fields
+    assertAboutImmutableObjects()
+        .that(legacyHistoryEntry)
+        .isEqualExceptFields(fromObjectify, "contactBase", "contactRepoId");
+    assertThat(fromObjectify instanceof ContactHistory).isTrue();
+    ContactHistory legacyContactHistory = (ContactHistory) fromObjectify;
+
+    // Next, save that from-Datastore object in SQL and verify we can load it back in
+    jpaTm()
+        .transact(
+            () -> {
+              jpaTm().insert(contact);
+              jpaTm().insert(legacyContactHistory);
+            });
+    ContactHistory legacyHistoryFromSql =
+        jpaTm().transact(() -> jpaTm().load(legacyContactHistory.createVKey()));
+    assertAboutImmutableObjects()
+        .that(legacyContactHistory)
+        .isEqualExceptFields(legacyHistoryFromSql);
+    // can't compare contactRepoId directly since it doesn't save the ofy key
+    assertThat(legacyContactHistory.getParentVKey().getSqlKey())
+        .isEqualTo(legacyHistoryFromSql.getParentVKey().getSqlKey());
+  }
+
+  @Test
+  void testFullConversion_domain() {
+    // Create+save an old domain HistoryEntry, reload it, and verify it's a proper DomainHistory
+    DomainBase domain = DomainHistoryTest.createDomainWithContactsAndHosts();
+    HistoryEntry legacyHistoryEntry = historyEntryForDomain(domain);
+    tm().transact(() -> tm().insert(legacyHistoryEntry));
+
+    // In Datastore, we will save it as HistoryEntry but retrieve it as DomainHistory
+    long historyEntryId = legacyHistoryEntry.getId();
+    HistoryEntry fromObjectify =
+        tm().transact(
+                () ->
+                    tm().load(
+                            VKey.create(
+                                HistoryEntry.class,
+                                historyEntryId,
+                                Key.create(legacyHistoryEntry))));
+    // The objects will be mostly the same, but the DomainHistory object has a couple extra fields
+    assertAboutImmutableObjects()
+        .that(legacyHistoryEntry)
+        .isEqualExceptFields(fromObjectify, "domainContent", "domainRepoId", "nsHosts");
+    assertThat(fromObjectify instanceof DomainHistory).isTrue();
+    DomainHistory legacyDomainHistory = (DomainHistory) fromObjectify;
+
+    // Next, save that from-Datastore object in SQL and verify we can load it back in
+    jpaTm().transact(() -> jpaTm().insert(legacyDomainHistory));
+    DomainHistory legacyHistoryFromSql =
+        jpaTm().transact(() -> jpaTm().load(legacyDomainHistory.createVKey()));
+    // Don't compare nsHosts directly because one is null and the other is empty
+    assertAboutImmutableObjects()
+        .that(legacyDomainHistory)
+        .isEqualExceptFields(
+            // NB: period, transaction records, and other client ID are added in #794
+            legacyHistoryFromSql, "period", "domainTransactionRecords", "otherClientId", "nsHosts");
+    assertThat(nullToEmpty(legacyDomainHistory.getNsHosts()))
+        .isEqualTo(nullToEmpty(legacyHistoryFromSql.getNsHosts()));
+  }
+
+  @Test
+  void testFullConversion_host() {
+    // Create+save an old host HistoryEntry, reload it, and verify it's a proper HostHistory
+    HostResource host = newHostResourceWithRoid("hs1.example.com", "host1");
+    HistoryEntry legacyHistoryEntry = historyEntryBuilderFor(host).build();
+    tm().transact(() -> tm().insert(legacyHistoryEntry));
+
+    // In Datastore, we will save it as HistoryEntry but retrieve it as HostHistory
+    long historyEntryId = legacyHistoryEntry.getId();
+    HistoryEntry fromObjectify =
+        tm().transact(
+                () ->
+                    tm().load(
+                            VKey.create(
+                                HistoryEntry.class,
+                                historyEntryId,
+                                Key.create(legacyHistoryEntry))));
+    // The objects will be mostly the same, but the HostHistory object has a couple extra fields
+    assertAboutImmutableObjects()
+        .that(legacyHistoryEntry)
+        .isEqualExceptFields(fromObjectify, "hostBase", "hostRepoId");
+    assertThat(fromObjectify instanceof HostHistory).isTrue();
+    HostHistory legacyHostHistory = (HostHistory) fromObjectify;
+
+    // Next, save that from-Datastore object in SQL and verify we can load it back in
+    jpaTm()
+        .transact(
+            () -> {
+              jpaTm().insert(host);
+              jpaTm().insert(legacyHostHistory);
+            });
+    HostHistory legacyHistoryFromSql =
+        jpaTm().transact(() -> jpaTm().load(legacyHostHistory.createVKey()));
+    assertAboutImmutableObjects().that(legacyHostHistory).isEqualExceptFields(legacyHistoryFromSql);
+    // can't compare hostRepoId directly since it doesn't save the ofy key in SQL
+    assertThat(legacyHostHistory.getParentVKey().getSqlKey())
+        .isEqualTo(legacyHistoryFromSql.getParentVKey().getSqlKey());
+  }
+
+  private HistoryEntry historyEntryForDomain(DomainBase domain) {
+    DomainTransactionRecord transactionRecord =
+        new DomainTransactionRecord.Builder()
+            .setTld("foobar")
+            .setReportingTime(fakeClock.nowUtc())
+            .setReportField(TransactionReportField.NET_ADDS_1_YR)
+            .setReportAmount(1)
+            .build();
+    return historyEntryBuilderFor(domain)
+        .setPeriod(Period.create(1, Period.Unit.YEARS))
+        .setDomainTransactionRecords(ImmutableSet.of(transactionRecord))
+        .setOtherClientId("TheRegistrar")
+        .build();
+  }
+
+  private HistoryEntry.Builder historyEntryBuilderFor(EppResource parent) {
+    return new HistoryEntry.Builder()
+        .setParent(parent)
+        .setType(HistoryEntry.Type.DOMAIN_CREATE)
+        .setXmlBytes("<xml></xml>".getBytes(UTF_8))
+        .setModificationTime(fakeClock.nowUtc())
+        .setClientId("TheRegistrar")
+        .setTrid(Trid.create("ABC-123", "server-trid"))
+        .setBySuperuser(false)
+        .setReason("reason")
+        .setRequestedByRegistrar(false);
+  }
+}
@@ -18,12 +18,15 @@ import static com.google.common.truth.Truth.assertThat;
 import static google.registry.model.ofy.ObjectifyService.ofy;
 import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
 import static google.registry.testing.DatastoreHelper.createTld;
-import static google.registry.testing.DatastoreHelper.persistActiveDomain;
+import static google.registry.testing.DatastoreHelper.newDomainBase;
+import static google.registry.testing.DatastoreHelper.persistActiveContact;
 import static google.registry.testing.DatastoreHelper.persistResource;
 import static google.registry.testing.SqlHelper.saveRegistrar;
 import static java.nio.charset.StandardCharsets.UTF_8;

 import google.registry.model.EntityTestCase;
+import google.registry.model.contact.ContactResource;
+import google.registry.model.domain.DomainBase;
 import google.registry.model.domain.Period;
 import google.registry.model.eppcommon.Trid;
 import google.registry.model.reporting.HistoryEntry;
@@ -34,6 +37,7 @@ import org.junit.jupiter.api.Test;
 /** Unit tests for {@link PollMessage}. */
 public class PollMessageTest extends EntityTestCase {

+  private DomainBase domain;
  private HistoryEntry historyEntry;
  private PollMessage.OneTime oneTime;
  private PollMessage.Autorenew autoRenew;
@@ -45,15 +49,17 @@ public class PollMessageTest extends EntityTestCase {
  @BeforeEach
  void setUp() {
    createTld("foobar");
+    ContactResource contact = persistActiveContact("contact1234");
+    domain = persistResource(newDomainBase("foo.foobar", contact));
    historyEntry =
        persistResource(
            new HistoryEntry.Builder()
-                .setParent(persistActiveDomain("foo.foobar"))
+                .setParent(domain)
                .setType(HistoryEntry.Type.DOMAIN_CREATE)
                .setPeriod(Period.create(1, Period.Unit.YEARS))
                .setXmlBytes("<xml></xml>".getBytes(UTF_8))
                .setModificationTime(fakeClock.nowUtc())
-                .setClientId("foo")
+                .setClientId("TheRegistrar")
                .setTrid(Trid.create("ABC-123", "server-trid"))
                .setBySuperuser(false)
                .setReason("reason")
@@ -61,6 +67,7 @@ public class PollMessageTest extends EntityTestCase {
                .build());
    oneTime =
        new PollMessage.OneTime.Builder()
+            .setId(100L)
            .setClientId("TheRegistrar")
            .setEventTime(fakeClock.nowUtc())
            .setMsg("Test poll message")
@@ -68,6 +75,7 @@ public class PollMessageTest extends EntityTestCase {
            .build();
    autoRenew =
        new PollMessage.Autorenew.Builder()
+            .setId(200L)
            .setClientId("TheRegistrar")
            .setEventTime(fakeClock.nowUtc())
            .setMsg("Test poll message")
@@ -75,50 +83,46 @@ public class PollMessageTest extends EntityTestCase {
            .setAutorenewEndTime(fakeClock.nowUtc().plusDays(365))
            .setTargetId("foobar.foo")
            .build();
-    // TODO(shicong): Remove these two lines after we use symmetric vkey and change the cloud sql
-    //  schema
-    oneTime.id = null;
-    autoRenew.id = null;
+    jpaTm()
+        .transact(
+            () -> {
+              saveRegistrar("TheRegistrar");
+              jpaTm().insert(contact);
+              jpaTm().insert(domain);
+              jpaTm().insert(historyEntry.toChildHistoryEntity());
+            });
  }

  @Test
  void testCloudSqlPersistenceOneTime() {
-    saveRegistrar("TheRegistrar");
-    jpaTm().transact(() -> jpaTm().saveNew(oneTime));
+    jpaTm().transact(() -> jpaTm().insert(oneTime));
    PollMessage.OneTime persisted =
        jpaTm().transact(() -> jpaTm().load(VKey.createSql(PollMessage.OneTime.class, oneTime.id)));
-    persisted.parent = oneTime.parent;
    assertThat(persisted).isEqualTo(oneTime);
  }

  @Test
  void testCloudSqlPersistenceAutorenew() {
-    saveRegistrar("TheRegistrar");
-    jpaTm().transact(() -> jpaTm().saveNew(autoRenew));
+    jpaTm().transact(() -> jpaTm().insert(autoRenew));
    PollMessage.Autorenew persisted =
        jpaTm()
            .transact(
                () -> jpaTm().load(VKey.createSql(PollMessage.Autorenew.class, autoRenew.id)));
-    persisted.parent = autoRenew.parent;
    assertThat(persisted).isEqualTo(autoRenew);
  }

  @Test
  void testCloudSqlSupportForPolymorphicVKey() {
-    saveRegistrar("TheRegistrar");
-
-    jpaTm().transact(() -> jpaTm().saveNew(oneTime));
+    jpaTm().transact(() -> jpaTm().insert(oneTime));
    PollMessage persistedOneTime =
        jpaTm().transact(() -> jpaTm().load(VKey.createSql(PollMessage.class, oneTime.getId())));
    assertThat(persistedOneTime).isInstanceOf(PollMessage.OneTime.class);
-    persistedOneTime.parent = oneTime.parent;
    assertThat(persistedOneTime).isEqualTo(oneTime);

-    jpaTm().transact(() -> jpaTm().saveNew(autoRenew));
+    jpaTm().transact(() -> jpaTm().insert(autoRenew));
    PollMessage persistedAutoRenew =
        jpaTm().transact(() -> jpaTm().load(VKey.createSql(PollMessage.class, autoRenew.getId())));
    assertThat(persistedAutoRenew).isInstanceOf(PollMessage.Autorenew.class);
-    persistedAutoRenew.parent = oneTime.parent;
    assertThat(persistedAutoRenew).isEqualTo(autoRenew);
  }

--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
gbrodman	4ec7f23e84	Use the parent to store the history repo ID and fill in the base object (#830 ) * Use the parent to store the history repo ID and fill in the base object Storing the repo ID in the parent and in the base object has two primary benefits. First, it unifies the parent information in the HistoryEntry's `parent` object. This simplifies the builders and the data flow. Second, when possible (which should be always, post-migration) we fill out the DomainContent's repo ID (similarly for the other EPP resources) which means that when reconstituting the ofy keys we don't need to pass the repo ID in from a separate object. This way, all the data are encapsulated where they should be. The primary downside here is that it further reduces the "immutability" of the history objects (since we're using the Hibernate setter for the parent repo ID) but we weren't immutable anyway. * Respond to CR - compare the entire vkeys in tests - always return the parent for repo ID * Simplify creation of parent VKeys * Fix flipped isAssignableFrom check in VKey * Merge remote-tracking branch 'origin/master' into historyRepoId	2020-10-09 16:01:51 -04:00
Shicong Huang	7a68b1b6f0	Revert package-lock.json to version from #676 (#834 ) Co-authored-by: gbrodman <gbrodman@google.com>	2020-10-09 15:57:23 -04:00
Shicong Huang	14e593d9e1	Add SchemaCrawler as dependency (#833 )	2020-10-09 15:02:11 -04:00
Weimin Yu	2d5de96fbd	Minor python changes (#832 ) * Minor python changes Use dataclasses instead of attrs. The former is part of the standard lib while the latter may need to be installed separately. Also added python3 to the list of prerequisites.	2020-10-09 14:50:21 -04:00
Weimin Yu	13d30b0bfb	Maintain a release-to-Version map in deployment (#831 ) * Maintain a release-to-Version map in deployment Keep track of the mapping between Nomulus release tags and AppEngine version ids with a mapping file. This is necessary because AppEngine does not support custom versioning. With this mapping, rollbacks could be automated. Automation of rollbacks is important since there are test-supporting metadata to be updated, but are easily forgotten. During the last stage of deployment, current per-service version ids are fetched using gcloud and are appended to a file on GCS. Each line is of the format "{RELEASE_TAG},{APPENGINE_SERVICE},{APPENGINE_VERSION}. This change has been tested in crash. The rollback script is still a work in progress.	2020-10-09 13:32:52 -04:00
Shicong Huang	b05f6b4ba3	Add SQL schema for DelegationSignerData (#713 ) * Add SQL schema for DelegationSignerData * Remove join table * Rebased on HEAD * Rebase on head	2020-10-09 10:22:31 -04:00
Shicong Huang	17a1387184	Disable auto-generation on id for HostHistory and ContactHistory (#827 )	2020-10-08 12:30:54 -04:00
Ben McIlwain	2e230664fd	Convert CertificateViolation into an enum (#829 ) * Convert CertificateViolation into an enum This ends up being nicer to deal with from callsites than class instances, while still permitting full configurability of all parameters. There are various other changes/fixes as well.	2020-10-07 22:19:36 -04:00
Michael Muller	299b093f78	Correctly restore composite VKeys in DomainContent (#825 ) * Restore composite vkeys in DomainContent PollMessage/BillingEvent vkeys in DomainContent must have their ofy keys restored from other fields in DomainContent (namely the repo id and their specific history event ids). Add PostLoad methods to DomainContent and DomainHistory to do the restoration. * Fixes for review. * Deal with foreign-key cycles	2020-10-07 12:42:01 -04:00
Ben McIlwain	61e7fa89f7	Fix incorrect repackaged App Engine import (#828 ) * Fix incorrect repackaged App Engine import	2020-10-07 11:52:17 -04:00
sarahcaseybot	6ab69d4226	Add a CertificateChecker class (#793 ) * CertificateChecker with checks for expiration and key length * Add validity length check * Get rid of hard-coded constants and DSA checks * add files that for some reason weren't included in last commit * Rename violations and other fixes * Add displayMessage to CertificateViolation enum * Switch violations from an enum to a class * small changes * Get rid of ECDSA checks * add checks for old validity length * Change error message for validity length	2020-10-06 15:47:42 -04:00
gbrodman	0f09a4a0ab	Add more Datastore/Sql Entity annotations (#826 ) * Add more Datastore/Sql Entity annotations * Move comments up a line	2020-10-05 13:07:53 -04:00
Shicong Huang	95f6ccc657	Fix vkey reconstruction for PollMessage (#823 ) * Fix vkey reconstruction for PollMessage * Add foreign key * Rebase on HEAD	2020-10-05 10:35:40 -04:00
Michael Muller	77fabe4dc4	Move "WithLongVKey" to BillingEvent subclasses (#821 ) When loading the VKeys for the BillingEvents hierarchy, it is necessary to restore the original concrete class for the type, otherwise we end up with a different (and incompatible) VKey. As part of this, convert the cancellation matching billing event to VKey<Recurring>, which seems like the only thing it actually can be.	2020-10-02 15:20:23 -04:00
Lai Jiang	71fa12f773	Fix invoicing SQL (#824 )	2020-10-01 14:29:49 -04:00
Shicong Huang	fd40a6a2b9	Use composite primary key for HostHistory and ContactHistory (#809 ) * Use composite primary key for HostHistory and ContactHistory * Update flyway file version * Make getters private * Add javadoc * Rebase on HEAD	2020-10-01 11:01:57 -04:00
Michael Muller	71f86c9970	Add VKey.restoreOfy() method for fixing ofy keys (#820 ) Add a restoreOfy() instance method and a restoreOfyFrom() static method to assist in restoring the objectify key for classes that have composite keys that do not restore automatically.	2020-09-30 11:15:58 -04:00
Michael Muller	6f75dfd116	Create a flyway index file and verify correctness (#819 ) * Create a flyway index file and verify correctness Create an index file (flyway.txt) containing the names of all of the flyway files and verify that it is ordered and in sync with the actual contents of the flyway directory. Also provide a target (generateFlywayIndex) to automatically generate it. The purpose of flyway.txt is to cause a merge conflict in the event that two different developers add a flyway file with the same sequence number, an event which has occurred multiple times.	2020-09-29 11:26:05 -04:00
Lai Jiang	ad5a74fee9	Revert "Request 101m CPU in sandbox proxy (#813 )" (#818 ) This reverts commit `e30c0f9a11`. The proposed solution didn't work.	2020-09-25 11:55:46 -04:00
Lai Jiang	29b1ec4211	Add log4j-core as a runtime dependency (#817 ) Without it we kept getting the following warning: ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console...	2020-09-24 19:59:39 -04:00
Weimin Yu	553d5717cb	Enhance the test for forbidden Schema changes (#815 ) * Enhance the test for forbidden Schema changes Current test is git-based. It is difficult to maintain and does not catch out-of-order version numbers. It is also more aggressive than necessary, failing on changes to submitted scripts that have not been deployed yet. The new test starts a database, deploys the current schema to it, then deploys the set of Flyway scripts in this repository to the database.	2020-09-24 12:31:08 -04:00
Shicong Huang	1056fdbb64	Fix VKey reconstruction issue in BillingEvent (#805 ) * Fix VKey reconstruction issue in BillingEvent * Rebase on head	2020-09-23 19:04:58 -04:00
Lai Jiang	4aaf31be9f	Update IDN tables per ICANN's request (#812 ) See b/168508962 for the request.	2020-09-21 23:08:10 -04:00
Lai Jiang	e30c0f9a11	Request 101m CPU in sandbox proxy (#813 ) This is suggested as a mitigation to allow us to deploy to sandbox. The default value is 100m. See: https://b.corp.google.com/issues/167295064#comment36.	2020-09-21 21:22:41 -04:00
gbrodman	2a5d9c8ef5	Allow explicitly for null EPP resources in History objects (#790 ) * Allow explicitly for null EPP resources in History objects * Repo IDs should always be nonnull * Add a test to verify loading / comparison of legacy HistoryEntry objects * Format javadoc + annotations * More javadoc changes * V52 -> V56 * V56 -> V57 * saveNew -> insert in new tests	2020-09-21 15:50:15 -04:00
gbrodman	597f5746a4	Rename V54 -> V56 on host table rename (#811 )	2020-09-21 13:34:37 -04:00
Michael Muller	5bff53a711	Rename HostResource table to Host (#804 ) * Rename HostResource table to Host * Convert drop/creates to renames	2020-09-21 11:43:25 -04:00
Michael Muller	933394e8c3	Improve naming of TransactionManager methods (#802 ) * Improve naming of TransactionManager methods Per internal discussion, convert names of methods as follows: saveNew -> insert saveNewOrUpdate -> put checkExists -> exists Likewise, convert derived names to their corresponding forms, e.g. saveNewOrUpdateAll -> putAll.	2020-09-21 09:10:01 -04:00
gbrodman	3b841bbb5b	Add domain-specific history fields to DomainHistory objects (#794 ) * Add domain-specific history fields to DomainHistory objects * Add javadoc for Hibernate-only methods * V52 -> V54 * Use only a single DomainTransactionRecord table * Add nullables and fix up a comment * V54 -> V55 * Regenerate db schema * Regen SQL file	2020-09-18 15:55:17 -04:00
gbrodman	798879d031	Fix semantic merge conflict in Registry (#810 ) * Fix semantic merge conflict in Registry	2020-09-18 14:40:11 -04:00
gbrodman	1a63d50b82	Create a separate per-tld registry lock/unlock cost (#800 ) * Create a separate per-tld registry lock/unlock cost Currently we use the standard server status change cost for this, but this might not be ideal at some point in the future if we wish to allow manual forced updates outside of the standard registry lock system (we would charge for these manual forced updates, even if we don't charge for registry locks). * Remove period	2020-09-18 13:22:29 -04:00
Lai Jiang	054571a625	Update ICANN activity reporting logging (#808 ) 1. It appears that when we have a 200 response, the response content is garbled, but we don't care since we know the request is successful. When we have a 400 response, the response is indeed UTF-8 encoded. Print the stack trace of the HTTP exception doesn't help anymore. 2. The result code is a complex type which includes the xml element with all its attributes, which don't care. We only want to print the value.	2020-09-18 11:37:23 -04:00
sarahcaseybot	7468a9915b	Migrate Registry objects to a TLD table in Cloud SQL (#803 ) * Add TLD table * Change reservedLists to array * Change ReservedLists back to a set * Rename reservedListKeyConverter to ReservedListKeySetConverter * Add a postload method	2020-09-17 12:47:50 -04:00
Lai Jiang	157d9f75c1	Use the save API version for the HPA controller and the deployment (#807 ) This supposedly can fix our deployment problem. Tested on alpha. Also updated the deployment script to replace the service object as well.	2020-09-16 09:15:35 -04:00
Lai Jiang	b2e4f07bb9	Update IDN tables (#806 ) See b/168508962 for context. 1. Changed the contact to iana-contact@google.com 2. Changed the header from "Script" to "Language" for zh-Hans and zh-Hant. 3. Commented out the references in zh-Hans and Zh-Hant	2020-09-16 09:15:27 -04:00
Michael Muller	5488e1b323	Fix accessing superclass fields in checkExists() (#799 ) * Fix accessing superclass fields in checkExists() JpaTransactionManagerImpl doesn't respect @Id fields in mapped superclasses. Replace calls to getDeclaredId() and getDeclaredField() with superclass friendly counterparts.	2020-09-11 13:45:51 -04:00
Shicong Huang	5ab0f97351	Add and use temp_history_id_sequence to avoid release error (#795 )	2020-09-11 12:25:08 -04:00