Commit Graph
68 Commits
Author SHA1 Message Date
shikhmanandBen McIlwain 2b1ab579dc Move metrics shutdown behavior to a ShutdownHook
HttpServlet#destroy() is not called on instance shutdown, so the metrics code was never shut down correctly before.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156360866
2017-05-23 17:22:49 -04:00
shikhmanandBen McIlwain 413a634533 Automated g4 rollback of changelist 156106183.
*** Reason for rollback ***

This change is crashing alpha

*** Original change description ***

Change metrics ThreadFactory to produce daemon threads

***

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156336467
2017-05-17 15:19:10 -04:00
shikhmanandBen McIlwain e1f4df86bd Change metrics ThreadFactory to produce daemon threads
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156106183
2017-05-17 12:21:12 -04:00
shikhmanandBen McIlwain 5779cc988b Refactor RdeKeyringModule into FakeKeyringModule
A fake Keyring implementation is needed to test Keyring related tools now, so
I've refactored the fake RdeKeyring to support all Keyring API methods and
moved it to the common testing package.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149977968
2017-03-21 14:53:25 -04:00
shikhmanandBen McIlwain 9eddbe2b6e Add test which verifies that all commands have descriptions
Besides being good practice, this works around a bug in JCommander 1.48, which we are stuck with for now
since newer versions of the library are Java 8 only:
https://github.com/cbeust/jcommander/issues/270

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149565357
2017-03-13 10:44:12 -04:00
shikhmanandBen McIlwain 1dfb6afa4b Add command description to get_resource_by_key command
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149561503
2017-03-13 10:41:15 -04:00
shikhmanandBen McIlwain b05afd2153 Fix nomulus tool NPEs on login/logout commands when printing usage
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149535358
2017-03-13 10:35:23 -04:00
shikhmanandBen McIlwain 51a613bca6 Fix bugs in KmsConnectionImpl
A few errors emerged when doing an integration test against the actual API. I've updated the unit tests to reflect the correct behavior.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148793856
2017-03-07 13:25:51 -05:00
shikhmanandBen McIlwain ddfe4c8a47 Add KMS parameters in configuration
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148778412
2017-03-07 13:21:23 -05:00
shikhmanandBen McIlwain 388dd1055e Abstract KMS code with KmsConnection and create a fake KmsConnection
This simplifies the tests for KmsKeyring and KmsUpdater.

This is a followup to []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148496758
2017-02-27 11:30:45 -05:00
shikhmanandBen McIlwain be30ecdf66 Add Cloud KMS based secret storage
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147791972
2017-02-17 12:22:12 -05:00
shikhmanandBen McIlwain 0916219c3e Fix javadoc placement
The prior placement was error prone.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146825173
2017-02-08 17:42:49 -05:00
shikhmanandBen McIlwain 37a4f389ef Fix incorrect description in javadoc
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=145836628
2017-01-30 15:03:53 -05:00
shikhmanandBen McIlwain 3f9c53b850 Refactor Stackdriver-over-REST specific code to a different package
Downstream users who use gRPC rather than REST don't want to pull down
rest-related dependencies.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=145834701
2017-01-30 15:03:53 -05:00
shikhmanandBen McIlwain 9adfd7a353 Switch PO number for Google Domains for 2017
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=144764336
2017-01-30 15:01:35 -05:00
shikhmanandBen McIlwain d3fe6be385 Remove ECatcher from GAE code
It's superceded by Stackdriver Trace.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=145440930
2017-01-25 12:25:41 -05:00
shikhmanandBen McIlwain e981bad0b3 Rename all encoding methods to start with the word encode for consistency
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143973446
2017-01-12 13:58:31 -05:00
shikhmanandBen McIlwain 7794fb04d1 Add documentation on Bazel to the code structure docs
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136083155
2016-10-14 17:00:32 -04:00
shikhmanandBen McIlwain 9695024f1b Add Stackdriver Monitoring info to the operational documentation
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136066350
2016-10-14 16:58:07 -04:00
shikhmanandBen McIlwain 77e826997d Add documentation about Guava to the code structure document
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136057514
2016-10-14 16:58:07 -04:00
shikhmanandBen McIlwain f76bc70f91 Preserve test logs and test summary output for Kokoro CI runs
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=135494972
2016-10-14 16:57:43 -04:00
shikhmanandBen McIlwain b62e60ed7e Fix documentation error in architecture doc
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=135411618
2016-10-07 15:29:48 -04:00
shikhmanandBen McIlwain adced3a5bc Make /bin/bash explicit in make_pkginfo
/bin/sh is not always /bin/bash, and double bracket evaluations don't work on all sh implementations.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=134998874
2016-10-03 16:45:21 -04:00
shikhmanandBen McIlwain aca4d569bd Add documentation about keeping the maven mirror up to date
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=134836487
2016-10-03 16:34:55 -04:00
shikhmanandBen McIlwain 9ea1c5f0d1 Update fastutil maven dependency
The version of fastutil (required by appengine-[]) that we included as an external maven dependency wasn't in the supported version range for appengine-[].

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=134813255
2016-10-03 16:15:31 -04:00
shikhmanandBen McIlwain e19546ffb4 Fix encoding error for Stackdriver metrics of cumulative points
The Stackdriver API requires that the end time always be greater than the start
time for cumulative metric points.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=134396192
2016-10-03 15:43:16 -04:00
shikhmanandBen McIlwain cee08d48f2 Remove unused dns-cron queue
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132884314
2016-09-14 15:44:28 -04:00
shikhmanandBen McIlwain bd887e857e Fix incorrect field name in EppMetric
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132688882
2016-09-14 15:35:25 -04:00
shikhmanandBen McIlwain aa4ca42cdd Add EPP metrics to flows
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132591518
2016-09-14 15:32:27 -04:00
shikhmanandBen McIlwain 4652688585 Add default DistributionFitter implementation to EventMetric
This DistributionFitter is suitable for tracking the latency of network calls.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132400538
2016-09-07 12:20:46 -04:00
shikhmanandBen McIlwain 42a39b0ddc Refactor EppMetrics into the EppMetric value type
This change refactors EppMetrics from the mutable self-exporting thing that it
was into a real value type EppMetric, and delegates exporting functionality to the
BigQueryMetricsEnqueuer.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132387660
2016-09-07 12:19:22 -04:00
shikhmanandBen McIlwain e5a0392994 Make StackdriverWriter#flush() return early when attempting to flush empty
timeseries

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132359115
2016-09-07 12:10:49 -04:00
shikhmanandBen McIlwain 0e8d9e3859 Force endTimestamp to be strictly greater than or equal to startTimestamp
This fixes an error that we've been seeing from Stackdriver.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132358790
2016-09-07 12:09:23 -04:00
shikhmanandBen McIlwain dbb9ef80c5 Refactor ConcurrentHashMap creation code to a helper method
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132352263
2016-09-07 12:06:26 -04:00
shikhmanandBen McIlwain 0538c1e210 Add method to create and register EventMetrics
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132346375
2016-09-07 12:03:28 -04:00
shikhmanandBen McIlwain 2ba9b01a13 Add the EventMetric metric type
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132345599
2016-09-07 12:01:58 -04:00
shikhmanandBen McIlwain 969d9483ae Add encoding for the Stackdriver Distribution value type to StackdriverWriter
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132327732
2016-09-07 11:58:58 -04:00
shikhmanandBen McIlwain 93bf68039b Add lock.unlock() to finally clause in Counter
It is good practice to unlock a lock in a finally clause.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132302413
2016-09-07 11:56:00 -04:00
shikhmanandBen McIlwain dcb189943b Add the Distribution data type for instrumentation
This is one of a series of CLs adding a new metric type, EventMetric, which
is used for tracking numerical distributions.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132103552
2016-09-07 11:54:26 -04:00
shikhmanandBen McIlwain 180240ae04 Wire in MetricsReporter to FrontendServlet
This work is identical to the work done for BackendServlet in
[]

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132100448
2016-09-07 11:52:59 -04:00
shikhmanandBen McIlwain 7d1a30d8d6 Remove backfill for DnsWriter and PricingEngine
The backfill for these is no longer necessary as all of the Registry entities in all environments have been backfilled.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=131984138
2016-09-02 13:53:28 -04:00
shikhmanandBen McIlwain 5ee78c505d Add metric related unit tests to EppControllerTest and FlowRunnerTest
This is one of a series of CLs which will refactor EppMetrics into a value type
and Metrics into a stateless class which will have an export(EppMetrics requestDetails)
method to export EPP metrics in a stateless way. Once EppMetrics is a value
type, I will create a new StackdriverEppMetrics that will also accept the value
type via an incrementRequests(EppMetrics requestDetails), allowing us to
monitor EPP via BigQuery and Stackdriver with minimum code duplication.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=131973288
2016-09-02 13:51:48 -04:00
shikhmanandBen McIlwain 1a60073b24 Small fixes in StackdriverModule
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=131955504
2016-09-02 13:48:47 -04:00
shikhmanandBen McIlwain c11ac3129f Factor out labelValue length check to abstract base class and small name fix
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=131902964
2016-09-02 13:47:21 -04:00
shikhmanandBen McIlwain c309560892 Rename fields in EppMetrics and EntityIntegrityAlertsSchema
This is one of a series of CLs which will refactor EppMetrics into a value type
and Metrics into a stateless class which will have an export(EppMetrics requestDetails)
method to export EPP metrics in a stateless way. Once EppMetrics is a value
type, I will create a new StackdriverEppMetrics that will also accept the value
type via an incrementRequests(EppMetrics requestDetails), allowing us to
monitor EPP via BigQuery and Stackdriver with minimum code duplication.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=131711810
2016-08-30 14:18:07 -04:00
shikhmanandBen McIlwain d77dced024 Fix NPE in StackdriverWriter
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=131603566
2016-08-30 14:13:50 -04:00
shikhmanandBen McIlwain 1c5b4e16c6 Add metrics to PublishDnsUpdatesAction
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=131190862
2016-08-26 09:43:30 -04:00
shikhmanandBen McIlwain e6beba2d39 Change resource type for Stackdriver to fix metrics push
Currently, it's forbidden for custom metrics to write to the gae_app resource
type.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=131076789
2016-08-23 15:48:49 -04:00
shikhmanandBen McIlwain 87314c4469 Add missing dependency to open source build
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130981257
2016-08-22 17:38:40 -04:00
shikhmanandBen McIlwain b04e54994e Fix open source build
Add missing dependencies to open source build.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130978680
2016-08-22 17:37:01 -04:00
shikhmanandBen McIlwain 91f8b6da38 Change to metrics to keep track of when the metric value was first set
This CL also adds IncrementableMetric#reset() methods to allow resetting the
value and start timestamp of IncrementableMetrics.

This is necessary because some backends, like Stackdriver, use non-monotonic
changes in cumulative metric values to detect timeseries restarts. Tracking and
re-setting the start timestamp allows users to track mostly monotonic metrics
which may have non-monotonic discontinuities.

See https://cloud.google.com/monitoring/api/ref_v3/rest/v3/TimeSeries#Point for
more details.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130795229
2016-08-22 14:44:28 -04:00
shikhmanandBen McIlwain b6eaba08eb Improve service handling and add unit tests for threading behavior in MetricExporter
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130790170
2016-08-22 14:43:05 -04:00
shikhmanandBen McIlwain f10a7d8fb0 Enforce monotonicity for IncrementableMetrics
This change enforces that IncrementableMetrics should only monotonically
increase in value, and adds a new increment() method to increment by one, which is slightly faster than incrementBy (due to a lack of non-negative checking) in the common case that the counter should only be incremented by one.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130578421
2016-08-22 14:20:33 -04:00
shikhmanandBen McIlwain 01e38790fd Fix visibility on MetricsReporter#startMetricExporter()
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130576851
2016-08-22 14:19:06 -04:00
shikhmanandBen McIlwain 7a041d066f Improve StackdriverWriter code
Add additional logic to handle cases when a MetricDescriptor is already defined
on the server, fix an NPE in related code, and add additional tests to ensure
that TimeSeries created from native MetricPoints are well-formed.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130555708
2016-08-22 14:17:38 -04:00
shikhmanandBen McIlwain 564bbdef83 Enforce very basic URL-like metric names
This change enforces URL-like metric names to match the behavior of
Stackdriver. The StackdriverWriter no longer prepends a slash, which was a
crutch.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130537347
2016-08-22 14:11:58 -04:00
shikhmanandBen McIlwain b45c2ca2ee Wire in MetricReporter into the backend servlet to publish metrics
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130468842
2016-08-22 14:07:45 -04:00
shikhmanandBen McIlwain 9b4ef23d92 Change project to projectResource to better match Stackdriver sample code
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130432024
2016-08-22 14:01:50 -04:00
shikhmanandBen McIlwain 64abebec82 Add StackDriver implementation, in monitoring/metrics package
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130287319
2016-08-15 17:12:35 -04:00
shikhmanandBen McIlwain 82ab624b36 Add instrumentation implementation
The instrumentation implementation adds the ability to track counters for dimensional data in the codebase. Instrumentation points will be added to different parts of the codebase in upcoming CLs. The ability to export the data to Stackdriver will also be added in an upcoming CL.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130195199
2016-08-15 16:58:42 -04:00
shikhmanandBen McIlwain e55ed209c5 Fix NPE in registry_tool when creating or updating a TLD with a dns writer
I forgot to inject the CreateTldCommand/UpdateTldCommand commands, which now
need to be injected so that the list of available DnsWriter implementations is
instantiated. This CL also adds a new DI Set<String> with just the name of the
writer, so that the instantiated writer map (which may have many DI dependencies)
doesn't need to be pulled in.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130042215
2016-08-15 16:48:20 -04:00
Greg ShikhmanandJustine Tunney 751df4b488 Re-indent module list in BackendComponent
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=129453164
2016-08-05 20:40:54 -04:00
Greg ShikhmanandJustine Tunney a620d06239 Add dagger map for injecting DnsWriter implementations
This is one of several CLs in a sequence for allowing per-TLD DNS
implementations.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=129445641
2016-08-05 20:38:21 -04:00
Greg ShikhmanandJustine Tunney 0e4244a03d Add DnsWriter for Cloud DNS
The plumbing to support different DnsWriter implementations per TLD
using DI will come in a later CL.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=129375856
2016-08-05 20:36:52 -04:00
Greg ShikhmanandJustine Tunney 1ba739a6b6 Refactor DnsWriter into the model package
This is one of several CLs in order to support per-TLD DnsWriter
implementations, modeled on the work done for PremiumPricingEngine.

Since DnsWriters will be set inside the Registry object, the DnsWriter
interface definition needs to be moved to models to create minimal
dependency on the rest of the registry codebase to avoid cyclic
dependency.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=128711643
2016-08-02 19:10:49 -04:00
Greg ShikhmanandJustine Tunney 6b5c00e143 Fix javadoc link to refer to correct annotation name
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=128364461
2016-08-02 19:08:35 -04:00
Greg ShikhmanandJustine Tunney 5522135f5a Change value type class to @AutoValue to fix errorprone warning
AutoValue classes by default do not allow null parameters, so I've removed the unnecessary null checks.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=127870611
2016-07-21 14:54:42 -04:00
shikhmanandBen McIlwain aa1b5229a0 Create Stackdriver module and inject it into backend module
This adds Stackdriver as a DI module into the registry codebase. This is a
necessary prerequisite for using Stackdriver based monitoring. For now, I'm
only adding it to the backend module as it will be more work to add it to the
frontend module.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=126730925
2016-07-13 15:22:19 -04:00