Commit Graph

713 Commits

Author SHA1 Message Date
Simone Gotti aff44f7e89
Merge pull request #174 from sgotti/datamanager_test_increas_etcd_waitdown_timeout
datamanager tests: increase etcd waitdown timeout
2019-11-12 11:02:12 +01:00
Simone Gotti dfd0f8c806 datamanager tests: increase etcd waitdown timeout 2019-11-12 10:47:28 +01:00
Simone Gotti 581a610523
Merge pull request #169 from sgotti/improve_error_handling
*: improve error handling
2019-11-11 14:08:39 +01:00
Simone Gotti 72f279c4c3 *: improve error handling
* objectstorage: remove `types` package and move `ErrNotExist` in base package
* objectstorage: Implement .Is and add helper `IsErrNotExist` for `ErrNotExist`
* util: Rename `ErrNotFound` to `ErrNotExist`
* util: Add `IsErr*` helpers and use them in place of `errors.Is()`
* datamanager: add `ErrNoDataStatus` to report when there's not data status in ost
* runservice/common: remove `ErrNotExist` and use errors in util package
2019-11-11 12:17:35 +01:00
Simone Gotti 89eb87a827
Merge pull request #170 from sgotti/objectstorage_use_single_package
objectstorage: use a single package
2019-11-11 10:24:54 +01:00
Simone Gotti 5af07d0852 objectstorage: use a single package
remove all the subpackages and just use a single package
2019-11-08 16:31:48 +01:00
Simone Gotti 35e1ec0e15 datamanager: remove old storage wals
Remove all wals not required by the existing data status files and not existing
in etcd.
2019-11-08 15:39:17 +01:00
Simone Gotti 9fbfeb9d13
Merge pull request #168 from sgotti/datamanager_refactor_readwal
datamanager: refactor ReadWal
2019-11-08 14:26:55 +01:00
Simone Gotti 9c0eb3d7ef datamanager: refactor ReadWal
make ReadWal directly return a *WalHeader
2019-11-08 13:24:43 +01:00
Simone Gotti cd5c5a20ed
Merge pull request #165 from sgotti/datamanager_check_wal_previousequence_is_correct
datamanager: check wal previouswalsequence is correct in initEtcd
2019-11-08 09:40:44 +01:00
Simone Gotti 9c1f3b2a69 datamanager: check wal previouswalsequence is correct in initEtcd 2019-11-07 17:05:40 +01:00
Simone Gotti 33e2d50036
Merge pull request #164 from sgotti/datamanager_dont_create_ost_checkpointed_files
datamanager: don't create ost wal checkpointed files
2019-11-07 14:48:12 +01:00
Simone Gotti acd62a3f90 datamanager: don't create ost wal checkpointed files
currently creating .checkpointed files in the ost isn't useful.
We already have the data snapshot that reports the last checkpointed wal.
2019-11-07 10:32:41 +01:00
Simone Gotti ae8eec94b5
Merge pull request #160 from sgotti/datamanager_remove_old_data_files
datamanager: clean old data files
2019-11-07 10:30:54 +01:00
Simone Gotti 4fcb067052 datamanager: clean old data files
keep the last n (now set to 3) data status files and remove all other data status
files and unneeded data files.
2019-11-07 10:30:31 +01:00
Simone Gotti 156995a804
Merge pull request #163 from sgotti/datamanager_fix_index_pos
datamanager: fix index creation on multiple data files
2019-11-06 17:50:12 +01:00
Simone Gotti 52cb683267 datamanager: fix index creation on multiple data files
When during a checkpoint more than one file is created the entries position in
the index is not right since it's not reset at every new index.

Fix it and add related tests.
2019-11-06 15:33:40 +01:00
Simone Gotti 778bc93755
Merge pull request #162 from camandel/config_enabled_services
services: check config only for enabled services
2019-11-06 13:58:21 +01:00
Carlo Mandelli aab2321d58 services: check config only for enabled services 2019-11-05 09:25:22 +01:00
Simone Gotti b78e0b5f33
Merge pull request #158 from sgotti/datamanager_add_datasequence_datafile_name
datamanager: add data sequence to data file name
2019-11-04 10:20:49 +01:00
Simone Gotti 4b4416fc99 datamanager: add data sequence to data file name
When creating a datafile name make it start with the current data sequence. This
is useful in future to know which data sequence created a new data file.
2019-11-04 09:23:12 +01:00
Simone Gotti e0346617ab
Merge pull request #159 from sgotti/sequence_tests
sequence: add tests for String and Parse methods
2019-11-04 09:22:32 +01:00
Simone Gotti e06dc332e2 sequence: add tests for String and Parse methods
Add tests to sequence String() and Parse(string) methods.
2019-10-31 16:53:57 +01:00
Simone Gotti d73344cf8c
Merge pull request #150 from sgotti/update_deps
go.mod: update dependencies
2019-10-30 17:47:52 +01:00
Simone Gotti e18794764e go.mod: update dependencies
Update all the updatable dependencies
2019-10-29 09:31:38 +01:00
Simone Gotti ec67b26d21
Merge pull request #149 from camandel/add_tty_option
run config: add tty option for run steps
2019-10-29 09:31:12 +01:00
Carlo Mandelli 7a51404b71 run config: add tty option for run steps 2019-10-28 16:58:54 +01:00
Simone Gotti 0558e78e82
Merge pull request #156 from camandel/readme_issue_link
README: issue link points to templates
2019-10-28 10:18:06 +01:00
Carlo Mandelli 93ad75127e README: issue link points to templates
[ci skip]
2019-10-26 11:21:11 +02:00
Simone Gotti 0fd76b2b4c
Merge pull request #155 from sgotti/objectstorage_add_writeobject_test
objectstorage: add WriteObject tests
2019-10-25 16:28:08 +02:00
Simone Gotti 1eb16886d8 objectstorage: add WriteObject tests
Test WriteObject using different size values: unspecified, equal to the buffer
size or less than the buffer size.
2019-10-25 15:16:42 +02:00
Simone Gotti 2e520dae55
Merge pull request #154 from sgotti/objectstorage_object_size
objectstorage: return object size in objectinfo
2019-10-25 12:23:48 +02:00
Simone Gotti 4c88bb75a3
Merge pull request #153 from sgotti/objectstorage_posix_limitreader_only_size_gt_0
objectstorage posix: use limitreader only when size is specified.
2019-10-25 12:23:35 +02:00
Simone Gotti 8a32e2251e
Merge pull request #152 from sgotti/objectstorage_s3_use_limitreader
objectstorage s3: use limitreader in write object
2019-10-25 12:23:19 +02:00
Simone Gotti ae1e92b115 objectstorage: return object size in objectinfo
Return object size in object info.
2019-10-25 10:53:36 +02:00
Simone Gotti 58f68601e6 objectstorage posix: use limitreader only when size is specified.
Use limitreader only when size is specified (greater or equal to 0).
When size is unknown (less than 0) limitreader will immediately return EOF
instead of writing the whole data.
2019-10-25 10:26:59 +02:00
Simone Gotti 0388003d09 objectstorage s3: use limitreader in write object
If size is specified limit reads to size bytes.
2019-10-25 10:06:22 +02:00
Simone Gotti 3808c71b37
Merge pull request #148 from sgotti/release_v0.3.0
Release v0.3.0
2019-10-24 11:34:52 +02:00
Simone Gotti 43c858ac7d Release v0.3.0 2019-10-24 09:35:59 +02:00
Simone Gotti a0450a5e69
Merge pull request #146 from sgotti/gitea_fix_getref
gitea: use GetRepoRefs instead of GetRepoRef
2019-10-23 10:29:23 +02:00
Simone Gotti 400de5f720
Merge pull request #147 from sgotti/fix_when_reftype_tag
when: fix typo in tag matching
2019-10-23 10:25:49 +02:00
Simone Gotti 11e803d6f0 when: fix typo in tag matching
Add related tests
2019-10-23 09:39:32 +02:00
Simone Gotti 1ed584ed1c
Merge pull request #144 from sgotti/cmd_run_create
cmd: add run create command
2019-10-22 16:48:18 +02:00
Simone Gotti 98912f1187 cmd: add run create command
Add a run create command to manually create a run providing the project ref, the
branch/tag/ref and an optional commit sha.
2019-10-22 10:47:20 +02:00
Simone Gotti e79337c3a7
Merge pull request #143 from sgotti/gateway_fix_project_create_run_http_method
gateway: fix project create run http method
2019-10-22 10:45:24 +02:00
Simone Gotti f0e4bbfeeb gitea: use GetRepoRefs instead of GetRepoRef
Looks like GetRepoRef doesn't correcly handle gitea repo refs response expecting
a single entry. Instead, at least with latest gitea version, the response is
always an array of refs. So use GetRepoRefs.
2019-10-22 10:17:00 +02:00
Simone Gotti 446e626f9f gateway: fix project create run http method
Make it a POST instead of a PUT.
2019-10-22 09:50:01 +02:00
Simone Gotti fc283e70d3
Merge pull request #142 from camandel/fix_multiple_volumes
docker: mount multiple volumes
2019-10-17 15:57:49 +02:00
Carlo Mandelli 6fccb935c4 docker: mount multiple volumes 2019-10-17 09:21:46 +02:00
Simone Gotti 2d813461d9
Merge pull request #140 from sgotti/gitsource_pr_dont_set_branch
gitsources: don't set branch value when in a pull request
2019-10-15 09:15:05 +02:00