We aren't turning on the cron job yet - we want to make sure it works correctly first.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=252726789
To do this we add a field of "rdapServers" in the Registrar object. Currently, we can only set this field manually, but a subsequent CL will add a cron-job to read these values from the ICANN servers.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=252438618
They are useful for viewing and diagnosing problems, and are not that
expensive.
We aren't enabling all the metrics though - the whole "per batch / per domain"
performance indicators are still disabled
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250876554
Although the GSON version in repositories.bzl was 2.8.5, the same name
(com_google_code_gson) was apparently used internally for version 2.7 for some
dependency of com.google.closure (soy?)
Changed the name, and it solved the problem. The new name is also "more
correct" as the package is com.google.gson, not com.google.code.gson.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250313017
Our database is big enough now that it takes > 4 hours to run the RDE Staging mapreduce.
Retrying after 4 hours just causes a new mapreduce to run, interfering with the old one.
8 hours still means 3 attempts per 24 hour period (00:07, 08:07, 16:07) so we can catch up with the cursor if needed.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=249829914
Also removed the rdapWhoisServer value, as it's just null and will likely stay that way (it isn't mentioned in the RDAP response profile)
If it'll ever become required, we can add it back.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=247643981
We currently create it from injected arguments, and pass it to every function.
Instead, we just create a provider for it and inject it where needed.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=247072517
Currently we try to reimplemnet the same behavior of the existing code as much
as possible.
We only fix issues that go against the RFC7483, but we don't yet update the
code to follow the latest (15feb19) RDAP Response Profile. That will require a
much bigger change especially for the test files, so it'll wait for a followup
CL.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246948018
Overriding getter methods to change values is a bit overkill when these values
are static (don't change based on internal state).
Just setting them in the base class' constructor is simpler.
Also, we can read the PATH of an Action based on the Annotation instead
returning it manually for each Action.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246135754
From to the RDAP response profile:
2.4.5. Abuse Contact (email, phone) - an RDAP server MUST include an *entity*
with the *abuse* role within the registrar *entity* which MUST include *tel*
and *email*, and MAY include other members
Even though this is a MUST - this field will only be shown if the registrar has a *visible* abuse contact.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242684303
We also ignore events that happened before the domain was created (for example, in a previous incarnation of the same domain name) and we set the last changed event to be the later of the last EPP change and any other event that happened before "now".
From RDAP response profile
2.3.2 The domain object in the RDAP response MAY contain the following events:
2.3.2.3 An event of *eventAction* type *transfer*, with the last date and time that the
domain was transferred. The event of *eventAction* type *transfer* MUST be omitted if
the domain name has not been transferred since it was created.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242461310
We're changing rdapConformance from rdap_level_0 to icann_rdap_response_profile_0, and changing the EPP status notices title from "EPP Status Codes" to "Status Codes" to conform with the changing spec.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241527211
This removes a LOT of "almost duplicate" test files, and moves some of the info into the java test itself making it clearer what's actually different between tests.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=240230982
Moving some of the info into the java test file itself makes it more easily readable. Maybe.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=240148747
Updated the plugin to receive the "protocol"-like tag in the destination, so that you can choose whether to upload to GCS or just save it locally.
Possibly we might expand this in the future, but for now the goal was to allow saving our "internal" builds locally until we find a secure way to store AND BROWSE them remotely.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=238055136
The --continue argument makes gradle try and build every task whose dependencies finished successfully. This is in contrast to the "regular" behavior of stopping at the first failure.
This is useful because it will show us all the failed tests (from all test tasks) instead of just the first test task that failed.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=237792667
This removes the "create Domain/Host/Contact" forms that were supposed to be used instead of regular EPPs for CC-TLD that wanted to support it.
We're removing it because we don't use it and want to reduce unneeded code for the registry 3.0 migration.
Also, this is a security risk, as it allowed to do "billable actions" (creating a new domain for example) with the only authentication being access to the registrar's G Suite account.
This bypassed the certificate, IP whitelist, and EPP password, which is bad.
PUBLIC:
Remove the web console EPP endpoint
This removes the "create Domain/Host/Contact" forms that were supposed to be used instead of regular EPPs for CC-TLD that wanted to support it.
We're removing it because we don't use it and want to reduce unneeded code for the registry 3.0 migration.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=236244195
This is mostly for build reports that use xml or other non-browsable
format. Most notable - the JUnit xml test results.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=236118885
We set the initial value to the "icann referral email", but registrars can change it later if they want.
Although this value isn't strictly required, we assume it exists in the spec11 report.
Also changed the name of the contact email from "email" to "consoleUserEmail"
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=235734200
This uploads all the report files to GCS, and also creates a cover page that
summarises all the task states.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=234601015
Icann reports have 3 parameter-provided injections:
- yearMonth
- subdir
- reportType
We move all of them away from the "inner classes" and only @Inject them in the Actions themselves.
This has 2 benefits:
- it's much clearer what all the parameter inputs of the Actions are
- the "inner injected classes" don't assume anything about the Action that uses them - they will work just as well for JSON actions as for "regular" actions.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=233625765
The goal of this CL is to set up the build environment to allow plugins to work.
We have a trivial plugin that doesn't do anything (yet) - it just sets itself as the finalizer of all Reporting tasks.
Eventually, this plugin will upload all reports to GCS, and even create a "cover page" linking to each one of them.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=233617499
We want to make it clear what query (or POST) inputs the user needs to / can give for each Action. That means moving all the @Injects of these parameters to the Actions themselves instead of injecting them in "hidden" indirect dependencies.
This has the extra benefit of allowing these indirect dependencies to work for JSON Actions as well, since the "regular" way we @Inject parameters can corrupt the POST JSON data.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=232540758
This console is only to be used by Admins (either GAE admins for this project, or Support accounts). It is for "internal" use only, not for use by the registrars themselves.
To prevent abuse, the registrar is created in a non-functional PENDING state and can only be made functional from the nomulus shell tool.
While in "PENDING" state, the registrar can be updated from the registrar-console by admins.
Also - moving all the web consoles to the same directory (moving the otesetup/* files into registrar/)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229681011
Currently, the TLD is a "hidden" part of the metric - we record it, but by looking at the call site you can't see that we record it.
Also, it's injected from the query / POST parameter, so we might not even be aware of what the value is and it might not be the value we wanted.
Instead, making it explicit in the Metric call. That way it's also more similar to the "logging" statements that record the same data but have to explicitly output the TLD.
It also makes the tests better, as we test that we indeed record the correct TLD
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229550115
The "lenient" bit must be the same between RdeStagingMapper and
RdeStagingReducer, but this is hidden by the Reducer receiving the bit in a
completely different way than the mapper.
There are 2 ways to do this:
- add a "setLenient" function to RdeStagingReducer that we MUST call, or else
get a runtime error. This is the simplest solution
- have a RdeStagingReducerBuilder you can inject, and that requires the
"lenient" value to actually build the RdeStagingReducer. This prevents bugs
at compile-time but is "more complicated"
I'm going with the second one here, but feel free to ask for the first one.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229423590
Our goal is to be able to address every Action by looking at the class itself, and to make it clearer at a glance what you need to access the Action's endpoint
Currently, we can know from the @Action annotation:
- the endpoint path
- the Method needed
- the authentication level needed
This CL adds the service where the Action is hosted, which also translates to the URL.
NOTE - currently we don't have any Action hosted on multiple services. I don't think we will ever need it (since they do the same thing no matter which service they are on, so why host it twice?), but if we do we'll have to update the code to allow it.
The next step after this is to make sure all the @Parameters are defined on the Action itself, and then we will be able to craft access to the endpoint programatically (or at least verify at run-time we crafted a correct URL)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229375735
Also using Service instead of serviceId String - because it helps readability.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228952033
Added a separator between the fields, and marked required fields as "required", so you can't submit without them
Also - changed from base64 to base58 in for the auto-generated password. It's conceivable that someone might need to read it outloud to someone else - and not having "visually similar" characters (like O and 0) can be helpful.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228810158
Generated code is now also covered by @VisibleForTesting, including Dagger @Inject
This CL is a cleanup of auto-generated code by ghm@ from the Error Prone team
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228748874
Certain flows need to launch batched jobs. Logically this would mean that flows
depend on batch.
However, the current state of dependency was the other way around, and the
reason for that was ResourceFlowUtils.java that had in it some utility
functions that weren't used in the flows and were needed in the batch jobs.
This CL removes these utility functions from the /flows/ directory, letting us
reverse the dependency edge between flows/ and batch/
Part of this was moving the flows/async/ code into batch/ - which also makes sense because flows/async/ just "enqueued" tasks that would then be run by actions in batch/
It makes sense that the code that enqueues the tasks and the code that dequeues the tasks sit in the same library.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228698761
Currently, you have to set "--ip_whitelist=null", which is unintuitive.
This adds the option to just give an empty string: "--ip_whitelist="
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=227550896
We create a new endpoint with a simple form that will let admins (including
support) setup OT&E for registrars.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=226570568
see b/34094769 for context
The webdriver tests don't choose a correct font when we specify "monospace". As a result, we don't render correctly pages that use monospace.
Here we instead explicitly reference a monospace font we know exists in the webdriver: Courier New.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=226233831
A few nullable parameters were not marked as nullable, which causes exceptions
to be thrown in debug mode.
This had no effect in the deployed web server, because these assert sanity
checks aren't performed - but on our local test server this failed.
Note that all these fields are checked for "nullness" in the code itself. It's
just an oversight in the declaration.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=226187227
SendEmailUtils is a general utility of the web console, and not specifically "only"
to the Registrar console.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=226187094
HTML headers can only be sent via JS, we need this change to allow
secure POST form submission.
The form itself will have a hidden "input" tag with the XSRF token in it. This
is how other framework do it as well - see
https://en.wikipedia.org/wiki/Cross-site_request_forgery#Synchronizer_token_pattern
This is in preparation for the OT&E setup page, which will be a simple form
with a "submit" button, so using JS for it is overkill.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=226178070
Currently there's a margin on the top, making the textarea be unaligned with
the text naming it. This is annoying on the eye, and will be more annoying in
the OT&E cl that will be added soon.
- So why not just do this change in that CL?
- Because the changes in the Screenshot tests here are irrelevant to that CL
and I found make it harder to actually review the actual screenshots we're
adding there.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=226057985
This is in preparation for having a web-console endpoint to create OTE.
In addition - we streamline the code:
- we remove support for different premium lists
- we remove support for different DNS writers - we never want a "real" DnsWriter for OTE
- we remove support of --eap_only, because we don't need it anymore
- We use a single password for all the Registrars
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225841694
In addition to just making good sense to not have support group for some
environments (local? unittest? crash?) - connecting with G Suit requires
additional permissions that are harder to find.
Specifically, it requires the Json Credentials that just aren't set in the
Dummy Keyring used on some environments.
So we make sure to not even *try* to create the credentials if the support
email isn't set
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225589255
We have a retrier wanting to call a function, retrying on
IllegalStateExceptions (thrown by checkState()).
However, one of the checkStates is called inside a Concurrent.transform, so
when the checkState fails, the resulting IllegalStateException is wrapped in an
UncheckedExecutionException and isn't caught by the retrier.
We unwrap the IllegalStateException to make sure it's caught.
Q: Why not just catch UncheckedExecution exception?
A: Because it might wrap a different exception which we don't want to retry on.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224862573
Currently the /registrar-settings backend endpoint will fail to update any
OWNER fields that a non-OWNER tries to change.
However, the front-end (soy, js) still allow non-OWNERs to try and change
these fields (there's the "edit" or "add" button, and it only fails when you try to "save")
This CL changes the front-end to remove the ability for non-OWNERs to even try
and change these fields. However, it will still let them *view* these fields as
it has interesting and important information.
-------------------------------
In addition - it changes the webdriver tests to include the "edit buttons". Those were never tested before, and now we will test to see if they are indeed displayed or not.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=223845883
content-type needs to be parsed (no automatic parsing from String)
data was splitting on commas, meaning --data="key=value1,value2" was sent to the server as "key=value1&value2"
NOTE - you'd expect there to already be a "do nothing splitter", right? But there isn't :/
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=223346887
The console will have 2 different "updatable things":
- only ADMINs (GAE-admins and users in the support G-Suite group) can change the things in the "admin settings" tab (currently just the allowed TLDs)
- only OWNERs can change things from the other tabs: WHOIS info, certificates, whitelisted IPs, contacts etc.
Also, all ADMINs are now OWNERS of "non-REAL" registrars. Meaning - we're only
preventing ADMINs from editing "REAL" registrars (usually in production).
Specifically, OTE registrars on sandbox are NOT "REAL", meaning ADMINS will
still be able to update them.
This only changes the backend (registrar-settings endpoint). As-is, the console
website will still make ADMINs *think* they can change everything, but if they
try - they will get an error.
Changing the frontend will happen in the next CL - because I want to get this
out this release cycle and getting JS reviewed takes a long time :(
TESTED=deployed to alpha, and saw I can't update fields even as admin on REAL
registrars, but could change it on non-REAL registrars. Also checked that I can
update the allowed TLDs on REAL registrars
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=222698270
EppConsoleAction still "manually" checks access by going over the
RegistrarContacts. We need it to use AuthenticatedRegistrarAccessor just like
every other part of the registrar console.
We still need to remove the (now unneeded) login EPP sent by the console, but that's left for a followup CL.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=222404208
This is a temporary measure until we implement access control for Support.
Once we implement access control, we will only block Support from removing TLDs
on production.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=222180321
It is starting to be used in more places than just ur/server/registrar. Even now it's used in the RDAP, and we are going to start using it for the registrar-xhr endpoint meaning it will be used in EPP flows as well.
Also logically - this is part of the request authentication.
While moving - we also refactor it to make it easier to use in tests. Instead of mocking, we will be able to create instances with arbitrary roles.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=221645055
Currently, OTE creates an invalid address "state", which doesn't pass validation when trying to update it on the console.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=221372006
After this CL, "support" accounts (accounts that are part of the "support" G-Suite group) will the same access to the registrar console as GCP "admins". However, they don't won't have access to the GCP project itself.
We could give them their own Role in the future (say SUPPORT) and give them different access than "admins", but right now we don't need it and YAGNI or something :)
NOTE: we identify users by their email (they need to be logged in to a google account). I don't know if that's best practice, since I guess different google accounts might have the same email address. However, G-Suite groups' membership is by email so there's not much we can do about it if we want to use G-Suite groups.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=220804273
This tab will set the "allowedTlds", but might have other functionality in the
future.
It is based on (branches from) the security-settings tab, because I'm copying the functionality of the "whitelisted IPs" to the "allowed TLDs": they are both lists of "arbitrary" strings that you can remove from and add to.
There are a lot of moving parts in this CL, because of how all the different elements need to interact, and how intertwined they are (for example, we need to disable the admin-settings view for non admins both in the soy and in the JS code)
It's really time to refactor the console given all we've learned... :/
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=220373443
This is in preparation for having other "command changing things" like
redirecting to file and maybe variable substitutions in the arguments.
"On the way" added a
RUNNING "some_command" "--some_flag" "some_value"
to the output encapsulator so that if we run multiple commands, we know what
command was called where.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=219837452
Currently, all 4 servlets (backend, frontend, pubapi, tools) have duplicates of
the same exact code.
That's an anti-pattern!
Created a ServletBase they can all extend which has the duplicate code.
As a bonus, the tools servlet now runs the metric reporter, meaning tool
related metrics will now be reported!
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=219792176
The cardinality of this new metric is:
buckets - 16
path + method - around 100 (the number of Actions we have)
authLevel - at most 3
success - 2
Total: 16*100*3*2 = 9,600
This is still low, especially for the value it could give in understanding our system (graphs of all endpoints, how often are they called, how long they take, how often do they fail)
Instead of "success true/false", we might want to give the actual status code. This can be a bit annoying because HttpServletResponse doesn't have a getStatus.
But it's possible, and worth considering.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=219312400
Having a log at the very begining of "our" code helped us find issues with
App-Engine's dispatcher, where we could clearly see in the logs that "our" code
started many seconds after the request came in.
We now suspect there is something similar going on after the end of "our" code
- where the reply is sent back many seconds after our code finished running.
To make sure - we add a log statement at the very last line of "our" code, so
we know exactly when it ended.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=219309962
AppEngineConnection can now connect to all services and not just the tools.
The default is still the tools.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=218734983
Cardinality of this metric:
clientId: there are currently 650 (on sandbox, because of OTE), and 200 on production.
explicitClientId: 2
roles: 2 now, might be 3 soon if we add vendors
status: 2
So we're talking about a cardinality of 2,000-8,000. Less when you consider that registrars only seldom actually need to access the console (certainly not daily or even weekly).
Compare with, e.g., the /epp/processing_time from the above EppMetrics.java which has:
Epp commands: 26 (manual counting)
client IDs: 200 on prod
status: the actual status CODE of the command. Can have many values, but looking at the past few weeks' metrics I counted 20
Note that not every command results in every status. Looking a few weeks back we can see around 80-100 (commands+status) combination.
buckets: 16
so that's over 250,000-1,000,000 cardinality, on a very high-volume metric.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=218699280
This CL removes the "READ vs UPDATE" feature completely. Now anyone with access
has full read+write access.
We still keep track of which role a user has (did they get access "explicitly"
because they are an "allowed access" contact? Or do they have access because
they are admins?) for the logs and UI, and also so we could in the (very near)
future have features only available to admins.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=218169608
All RDAP actions inherit from a common RdapActionBase class.
All RDAP search actions inherit from a comman RdapSearchActionBase class, which inherits from RdapActionBase
Each of the base classes has @Before initialization needed for the tests, as well as utility functions (such as login and logout).
Currently, these were copied in all test classes. Instead, we created a similar test inheritance tree to centralize the initialization and place common utility functions.
This way, the @Before of every test only needs to initialize the variables new to the specific action, making the code somewhat clearer.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=217856844
When creating the various registrar objects in Sandbox for OTE, we also give access to all the registrars' data to a given google account (identified by the email)
This email has to belong to the registry's G-Suite account, just like in the registrar_contact command.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=217728407
Whenever a registrar is changed via the registrar console, we send out a
notification of that change.
Since we're going to allow Admins and soon Vendors to use the console in
addition to the registrars, it becomes important to know who actually performed
the changes if the registrars complain.
In addition, we will now send notifications for changes in Sandbox since we're
going to actually allow registrars to update sandbox data.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=217539534
SessionUtil is renames AuthenticatedRegistrarAccessor, as it's used to access a registrar for an authenticated user.
It will now be injected with the AuthResult instead of receiving it in every function call, since there's only one "legal" AuthResult to use.
The AccessType names are changed from READ_ONLY/READ_WRITE to READ/UPDATE, as it was confusing that a user could have both READ_ONLY AND READ_WRITE access to the same registrar.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=216958306
We want to be able to view / test / debug how the registrar console looks for our clients.
However, we don't want to accidentally change the data for registrars, especially in a "non-accountable" way (where we later don't know who did that change)
So we do 2 things here:
- Add a "mode" (read-only and read-write) to the getRegistrarForUser function. We set it according to what we want to do with the registrar. Currently, read-write is only requested for the "update" RegistrarSetting action. Admins will have read-only access to all registrars, but read-write access only to the "admin registrar" (or whatever registrar they are contacts for).
- Support an undocumented "clientId=XXX" query param that replaces the "guessClientIdForUser" function in the original page load. We can then set it when we want to view a different account.
We also change the navigation links on the HTML page to preserve the query.
-------------------------
This might be used also for a better user experience for our clients, especially those with multiple "clientId"s (some registrar entities have multiple "registrar" objects)
Currently, they have to have a separate user for each clientId, and only have one user allowed which has both read and write permissions.
Using this change, we can give them the possibility to add users on their own, some with read-only access (to view billing information without being able to change anything), and use a single user for all their clientIds.
-------------------------
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=215480610
RegistrarSettingsAction is a JSON in / JSON out endpoint, meaning the reply is consumed as JSON.
The current state is that if an error occurs, there are two possible replies:
- a JSON error reply is sent out, or
- a 402 HTML reply is sent out with the exception.getMessage()
The difference is only - do we actively catch the exception to translate it to JSON or not.
This fix catches ALL exceptions and translates them to JSON format. Note that there's no security change by giving the getMessage in the JSON reply since we were returning that anyway (in the HTML).
In addition - changed the "gaeUserId" to "user.getEmail" as the identifier, since it's clearer to the users who see that error - and I do want to transition to a more "email identifier" way of checking access (since that's what users put in the registrar contact info)
This too isn't leaking new information because
- the initial HTML page load already gives the user's email, and
- the logs already log the user's email for every request
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=215213807
Replaced the plethora of inter winding access functions and inputs in SessionUtils with just 2 functions, that both accept the same type for the user (AuthResult):
guessRegistrarForUser: given an AuthResult, finds a registrar that they have access to. If none is found - a ForbiddenException is thrown.
getRegistrarForUser[Cached]: (maybe should be called getRegistrarOnBehalfOfUser?) given an AuthResult and a clientId, loads and returns the registrar ONLY IF the user has access to it. Otherwise throws a ForbiddenException.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=214630657
This is an intermediate CL, part of the Registrar Console cleanup.
TL;DR:
- the current state: resource.js points to a resource TYPE on the server (only registrars can be resources right now), but the specific resource is selected based on the user (we select the "first resource of this type that the user has access to)
- new state: resource.js points to a SPECIFIC resource (TYPE + ID).
In this CL the server still chooses the resource like before (first one that user has access to) but we make sure the returned resource is the same one we requested.
In a subsequent CL we will use the requested ID to load the resource, and then make sure the user has access to that resource.
---------------------------
When loading the RegistrarConsole HTML page, the server determines which clientId belongs to the user ("guesses" it by looking for the first registrar that has this user as contact). It sends the relevant clientId back with the page load.
However, this information isn't currently used in the JS requests to read / update the registrar. Instead, currently the client ID is guessed again for each JS access to the server. It is also saved again in the client's "session" cookie.
As a result, it is theoretically possible to have the JS access a different clientID than the original page load (not likely, since it requires a single user registered for multiple registrars AND that the contacts change for the original registrar).
So our goal is to only have a single clientID "value" instead of the 3 we currently have for JS requests (the one from the initial page load, the one saved in the session cookie, the one guessed on the JS request)
As a first step, we send over the "initial page load" clientId on every JS request, and make sure the "session + guessed" value is equal to that one. Later we will remove the "session+guessed" values from the RegistrarSettings, using the "initial page load" clientID instead.
In addition to the "nicer code" implications, having the clientID from the initial page load always used means it'll be easy to have a clientID selection option for users who have access to multiple clientIDs (such as admins)
SECURITY NOTE:the choice of clientID has no security implication since we make sure the user has access to the clientID no matter how we actually choose the clientID on every single server request.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=214459506
*** Reason for rollback ***
Automated tools sometimes don't have default credentials, and can't set them up. We should redo this CL once we figure out the credential thing.
*** Original change description ***
Add metrics for the command used in the registry CLI tool
Puts the metric in <project>/tools/commands_called
It counts the use of the tool, with the following labels:
- environment
- tool (nomulus/gtech)
- command called (class name)
- success true/false
- from the shell true/false
***
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=214048616
We have several "consoles" ("views" in the registrar console). They all affect the same resource - the registrar itself.
Currently, each view creates its own "RESTful resource", even though it's the same resource for all of them, meaning we have copied code and copied "URL endpoint" across multiple files.
I assume this was made so that IF one day we have a "view" to another resource, we can easily add it. But we currently don't have any such view, nor plans to add any.
So according to the YAGNI paradigm, it's better to move the resource creation outside of the console.
Also, IF we do add a view to a different resource - it'll still be more readable to have a map from the "view" to the "resource URL endpoint" alongside the existing map from the "view" to the "console"...
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=213992967
The original thought was that the actions you can do on a resource is:
- create it
- read it
- update it
(I guess you should have "delete" as well, but that isn't currently there)
Although we use "read" and "update", we never use "create". So having it goes against the YAGNI principle :)
Also, it had a bug: when sending a "create", the opt_newId in send_() would
permanentily change the uri of the request, causing any subsequent request to
go to the wrong endpoint.
By removing the "create" we can simplify the rest of the code (the send_() function).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=213029499
Puts the metric in <project>/tools/commands_called
It counts the use of the tool, with the following labels:
- environment
- tool (nomulus/gtech)
- command called (class name)
- success true/false
- from the shell true/false
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=212879670
According to RFC 2046, the body of the multipart contains:
multipart-body := [preamble CRLF]
dash-boundary transport-padding CRLF
body-part *encapsulation
close-delimiter transport-padding
[CRLF epilogue]
The preemble and epilogue are optional, and ignored. However, it's not 100%
explicit whether the CRLFs after the preamble and before the epilogue are
required. The one after the preemble is often not given if there's no preemble,
so it's conceivable that you don't *have* to give the CRLF before the epilogue
if there's no epilogue (it's also enclosed in the [], making it part of the
"optional")
However, it seems that when the TMDB "migrated to the cloud" (as they
describe it) on Aug. 13 2018, they started requiring that CRLF.
TESTED=connected to a TMDB-whitelisted server, used CURL to manually create the
message as we currently send it (without the final CRLF) with junk data and got
the error from the bug. Then sent the exact same message with the additional
CRLF, and got a different error that directly relates to the content of the
junk data.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=212637246
The "tar file encoding" saves the file + metadata (filename and modification) in a "tar" format that is required in the RDE spec, even though it only contains a single file.
This is only relevant for RyDE, and not for Ghostryde. In fact, the only reason Ghostryde exists is to not have the TAR layer.
Currently we only encrypt RyDE, so we only need the TAR encoding. We plan to add decryption ability so we can test files we sent to IronMountain if there's a problem - so we will need TAR decoding for that.
The new file - RydeTar.java - has both encoding and decoding. We keep the format used for all other Input/OutputStreams for consistency, even though in this case it could be a private part of the RyDE encoder / decoder.
This is one of a series of CLs - each merging a single "part" of the encoding.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=208056757
The "file encoding" saves the file + metadata (filename and modification) in a "blob" format that PGP knows how to read.
Merges the file-encoder creation between RyDE and Ghostryde.
The new file - RydeFileEncoding.java - is a merge of the removed functions in
Ghostryde.java and the RydePgpFileOutputStream.java.
This is one of a series of CLs - each merging a single "part" of the encoding.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=205295756
Merges the encryptor creation between RyDE and Ghostryde.
The new file - RydeEncryption.java - is a merge of the removed functions in
Ghostryde.java and the RydePgpEncryptionOutputStream.java.
This is one of a series of CLs - each merging a single "part" of the encoding.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=205246053
Merges the compressor creation between RyDE and Ghostryde. Note that GhostRyde
will now compress with ZIP rather than the previous ZLIB. This is backwards
compatible because the decompression algorithm works with either, so files
created by the old version (with ZLIB) can still be opened by the new version,
and vice-versa.
The new file - RydeCompression.java - is a merge of the removed functions in Ghostryde.java and the RydePgpCompressionOutputStream.java.
This is one of a series of CLs - each merging a single "part" of the encoding.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=205102150
Ghostryde.java has a lot of duplicate code with RydeEncoder and the future
RydeDecoder - the encryption/decryption, compression/decompression, file
encoding/decoding. The "de-XXX" part of each of these pairs needs to read a PGP
object from a stream using PGPObjectFactory.
Since we want to move the duplicate code into their own files, we will need to
move the "read PGP objects from stream" functions to a common utility class.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=205092800
Second step of RDE encoding refactoring.
Creates a single OutputStream encode RyDE files.
This replaces the 5 OutputStreams that were needed before.
Also removes all the factories that were injected. It's an encoding, there's no point in injecting it.
Finally, removed the buffer-size configuration and replaced with a static final
const value in each individual OutputStream.
This doesn't yet include a decoder (InputStream). And there's still a lot of overlap between the Ryde and the Ghostryde code. Both of those are left for the next CLs.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=204898369
First step of RDE encoding refactoring.
Creates a single InputStream (OutputStream) to decode (encode) Ghostryde files.
This replaces the 3 InputStreams (OutputStreams) that were needed before.
Also removes a lot of classes, and removes the "injection" of the Ghostryde
class. It's an encoding, there's no point in injecting it.
Finally, removed the buffer-size configuration and replaced with a static final
const value. It's just a buffer size - it doesn't actually affect much. There
are much more "important" fields that weren't configured (such as the
compression algorithm and whether or not to do integrity checks)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202319102
RdeUpload works on alpha and sandbox by sending the data to a google internal
server. It is still running (and succeeding) after each RdeStaging, but the
lack of the cron job means it is currently lagging behind.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202181148
guyben13andLai Jiang
