Commit Graph
797 Commits
Author SHA1 Message Date
chrislu b28b1a3402 3.77 2024-10-10 01:28:06 -07:00
chrislu 82ed61c655 3.76 2024-10-06 19:37:03 -07:00
chrislu 117c3aebda 3.75 2024-10-03 08:33:35 -07:00
chrislu e3fa87bcc1 3.74 2024-09-30 11:29:01 -07:00
BruceandGitHub 5428229347 fix file read crash (#6021) 2024-09-14 08:33:35 -07:00
chrislu 6063a889ed 3.73 2024-09-12 13:11:55 -07:00
Eugeniy E. MikhailovandGitHub dab0bb8097 Feature limit caching to prescribed number of bytes per file (#6009)
* feature: we can check if a fileId is already in the cache

We using this to protect cache from adding the same needle to
the cache over and over.

* fuse mount: Do not start dowloader if needle is already in the cache

* added maxFilePartSizeInCache property to ChunkCache

If file very large only first maxFilePartSizeInCache bytes
are going to be put to the cache (subject to the needle size
constrains).

* feature: for large files put in cache no more than prescribed number of bytes

Before this patch only the first needle of a large file was intended for
caching. This patch uses maximum prescribed amount of bytes to be put in
cache. This allows to bypass default 2MB maximum for a file part stored
in the cache.

* added dummy mock methods to satisfy interfaces of ChunkCache
2024-09-11 21:09:20 -07:00
Eugeniy E. MikhailovandGitHub c04edeed68 bug fix in the data received from cache processing (#6002)
The patch addresses #3745.

The cache should return the exact amount of data requested by the buffer.
By construction of the cache it is always all requested data range
or we have error happening.

The old use of minsize miscalculate the requested data size,
if non zero offset is requested.
2024-09-10 13:30:18 -07:00
chrislu 18afdb15b6 Revert "weed mount, weed dav add option to force cache"
This reverts commit 7367b976b0.
2024-09-04 01:38:29 -07:00
chrislu 7367b976b0 weed mount, weed dav add option to force cache 2024-09-04 01:19:14 -07:00
chrislu e50d85c0f3 3.72 2024-08-18 21:51:53 -07:00
chrislu 57dc39c451 randomizing next file handle id 2024-08-07 10:42:00 -07:00
chrislu ed7e721efe 3.71 2024-07-21 21:03:27 -07:00
chrislu 4020149690 3.70 2024-07-21 20:27:39 -07:00
chrislu 9dd008f8f1 add version to filer 2024-07-21 20:13:51 -07:00
vadimartynovandGitHub 86d92a42b4 Added tls for http clients (#5766)
* Added global http client

* Added Do func for global http client

* Changed the code to use the global http client

* Fix http client in volume uploader

* Fixed pkg name

* Fixed http util funcs

* Fixed http client for bench_filer_upload

* Fixed http client for stress_filer_upload

* Fixed http client for filer_server_handlers_proxy

* Fixed http client for command_fs_merge_volumes

* Fixed http client for command_fs_merge_volumes and command_volume_fsck

* Fixed http client for s3api_server

* Added init global client for main funcs

* Rename global_client to client

* Changed:
- fixed NewHttpClient;
- added CheckIsHttpsClientEnabled func
- updated security.toml in scaffold

* Reduce the visibility of some functions in the util/http/client pkg

* Added the loadSecurityConfig function

* Use util.LoadSecurityConfiguration() in NewHttpClient func
2024-07-16 23:14:09 -07:00
vadimartynovandGitHub b796c21fa9 Added loadSecurityConfigOnce (#5792) 2024-07-16 09:15:55 -07:00
chrislu 33b11f92bc Squashed commit of the following:
commit 465b47b8f72668dfc12eaa72befb232513c9e1db
Merge: f39272b54 9c440d472
Author: chrislu <chris.lu@gmail.com>
Date:   Wed Jul 10 23:56:51 2024 -0700

    Merge branch 'master' into merged-persisted-logs

commit f39272b5416bdd4d1714ce3e99aded0fe62e7d89
Author: chrislu <chris.lu@gmail.com>
Date:   Wed Jul 10 23:49:26 2024 -0700

    merge meta logs from multiple files
2024-07-11 00:04:24 -07:00
chrislu 114ac39917 add a simple test 2024-07-07 23:44:36 -07:00
Konstantin LebedevandGitHub 5ffacbb6ea refactor all methods strings to const (#5726) 2024-07-01 01:00:39 -07:00
chrislu fbe1316c92 3.69 2024-06-30 23:08:49 -07:00
chrislu c030cb3ce9 bootstrap filer from one peer 2024-06-28 14:57:20 -07:00
chrislu a9cd9b0542 3.68 2024-06-09 21:10:46 -07:00
chrislu d218fe54fa go fmt 2024-05-20 11:03:56 -07:00
chrislu d3032d1e80 3.67 2024-05-07 08:47:30 -07:00
Patrick SchmidtandGitHub 96c48bd0da Fix deadlock in lock table locks (#5566) 2024-05-07 05:44:53 -07:00
chrislu 3682eb929f 3.66 2024-05-05 22:23:18 -07:00
chrislu a5645d50a7 ensure head index is within range 2024-05-05 12:20:08 -07:00
chrislu 8948fb51e3 Merge branch 'master' into mq-subscribe 2024-04-25 23:42:19 -07:00
M@andGitHub abf01a0eb7 Fixes unlocked read from logBuffer.LastTsNs that is racey. (#5536) 2024-04-25 15:46:12 -07:00
Konstantin LebedevandGitHub 2f3fee9bb9 filer read chunk retry if status code 499 (#5528) 2024-04-24 06:51:05 -07:00
chrislu ce2b2fa9b2 Merge branch 'master' into mq-subscribe 2024-04-15 20:09:14 -07:00
chrislu 0edc5aad9a 3.65 2024-04-14 20:15:46 -07:00
chrislu e53a972483 Merge branch 'master' into mq-subscribe 2024-04-11 19:48:00 -07:00
Konstantin LebedevandGitHub 5189a09de0 [volume] Reduce the number of buffers for uploading one chunk (#5458) 2024-04-11 04:47:21 -07:00
chrislu cc1c69f312 Merge branch 'master' into mq-subscribe 2024-04-08 11:03:14 -07:00
Henco AppelandGitHub 5c8e6014ba fix: filer authenticate with with volume server (#5480) 2024-04-08 07:27:00 -07:00
clonefetchandGitHub 9e07a87fcb chore: fix function names in comment (#5478) 2024-04-08 07:19:02 -07:00
chrislu ccdd9cd8de refactor 2024-04-02 16:25:43 -07:00
chrislu d935f70e3c shutdown follower when leader stops 2024-04-01 22:32:30 -07:00
chrislu f07875e8e1 send flush message to follower before shutting down logBuffer 2024-04-01 16:01:26 -07:00
chrislu 03f18e43df clean up logs 2024-03-21 23:30:49 -07:00
chrislu 26e24db721 adjust log level 2024-03-21 23:27:19 -07:00
chrislu 5c0fcae039 clean up logs 2024-03-21 23:24:18 -07:00
chrislu 407a52fc93 adjust log level 2024-03-21 23:23:26 -07:00
chrislu 50023fe1cf adjust log level 2024-03-21 08:07:56 -07:00
chrislu 3c559dc294 fix test2 2024-03-16 23:48:31 -07:00
chrislu b74e8082ba 3.64 2024-03-16 11:46:40 -07:00
chrislu 054d0b3487 wait for data 2024-03-16 11:14:14 -07:00
chrislu f8957b84c1 rename to LastTsNs 2024-03-16 10:52:47 -07:00