a/agola
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
642 Commits 1 Branch 3 Tags 2.3 MiB
8bbdc0a36e
Commit Graph

642 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Carlo Mandelli
8bbdc0a36e cmd: add TaskID to run list output 2019-12-02 08:51:11 +01:00
Simone Gotti
2232e2895e
Merge pull request #185 from camandel/add_run_list_info 
cmd: add details to run list
 2019-11-29 16:21:45 +01:00
Carlo Mandelli
a15a769610 cmd: add details to run list 2019-11-29 14:52:04 +01:00
Simone Gotti
05984f0f62
Merge pull request #187 from camandel/cli_log_management 
cmd: add log get and delete subcommands
 2019-11-29 10:41:31 +01:00
Carlo Mandelli
8aade1859b cmd: add log get and delete subcommands 2019-11-28 13:39:00 +01:00
Simone Gotti
cc70a55a59
Merge pull request #186 from camandel/fix_tests_api_delete 
gateway: fix api used by GetRunTask
 2019-11-27 08:52:37 +01:00
Carlo Mandelli
4464c9a6f2 gateway: fix api used by GetRunTask 2019-11-25 14:42:34 +01:00
Simone Gotti
c55f1f2e16
Merge pull request #184 from camandel/fix_change_visibility 
cmd: fix project visibility switch
 2019-11-21 15:22:05 +01:00
Carlo Mandelli
42e302df50 cmd: fix project visibility switch 2019-11-21 14:44:50 +01:00
Simone Gotti
074f346c78
Merge pull request #182 from sgotti/release_v0.4.0 
Release v0.4.0
 2019-11-19 10:40:30 +01:00
Simone Gotti
e077ac8ba9 Release v0.4.0 2019-11-19 09:52:33 +01:00
Simone Gotti
f5998a78d4
Merge pull request #157 from camandel/api_delete_logs 
gateway/runservice: add api to delete step logs
 2019-11-18 13:58:25 +01:00
Carlo Mandelli
3e47bc601a gateway/runservice: add api to delete step logs 2019-11-18 10:34:56 +01:00
Simone Gotti
3ff18510a6
Merge pull request #181 from sgotti/runservice_logshandler_improve_errors_02 
runservice: improve errors in logsHandler
 2019-11-15 16:46:40 +01:00
Simone Gotti
7e8f7155d7 runservice: improve errors in logsHandler 
return errNotExist in readTaskLogs when the run,task or step doesn't exist.
 2019-11-15 15:50:58 +01:00
Simone Gotti
5fe3ceb60c
Merge pull request #180 from sgotti/api_write_flush_logs_headers 
*: write and flush header on log handlers
 2019-11-15 10:18:28 +01:00
Simone Gotti
f7d0950ca1 *: write and flush header on log handlers 
Explicitly write and flush the headers in the various services LogHandlers.

Currently the 200 response and the other headers will be automatically written
by the golang http implementation only when we send something in the body. But if
there's nothing to send (no logs yet written) the client will never receive the
headers and cannot know if the request was successful.
 2019-11-14 10:52:45 +01:00
Simone Gotti
32a08ec5c8
Merge pull request #179 from sgotti/runservice_logshandler_improve_errors 
runservice: improve errors in logsHandler
 2019-11-14 09:56:24 +01:00
Simone Gotti
eeacc296d0
Merge pull request #177 from sgotti/runservice_use_trylock_on_fetching 
runservice: use etcd mutex TryLock on fetching
 2019-11-13 18:02:34 +01:00
Simone Gotti
66e182a55d runservice: improve errors in logsHandler 
* return errNotExist in readTaskLogs when the executor task doesn't exist: so
the client will receive a 404 instead of a 500 (since a generic error will be
mapped to a 500).
* Wrap the errNotExist returned by readTaskLogs with a new ErrNotExits reporting
"log doesn't exist"
 2019-11-13 15:50:20 +01:00
Simone Gotti
07cde065c8 runservice: use etcd mutex TryLock on fetching 
When fetching avoid concurrent fetches from multiple runservices using an etcd
mutex TryLock.
 2019-11-13 11:53:54 +01:00
Simone Gotti
9fd4b662a8
Merge pull request #175 from camandel/api_logarchived 
gateway: add api to get log status
 2019-11-13 11:45:31 +01:00
Simone Gotti
4497e196ba
Merge pull request #176 from sgotti/use_etcd_mutex_trylock 
*: use etcd mutex TryLock
 2019-11-13 11:32:29 +01:00
Carlo Mandelli
8ed813e722 gateway: add api to get log status 2019-11-13 10:01:51 +01:00
Simone Gotti
5ab9f7c970 *: use etcd mutex TryLock 
etcd PR 11104 (https://github.com/etcd-io/etcd/pull/11104) implemented mutex
TryLock. Since it's only available in etcd master just copy relevant code and
add a TODO to remove it when updating the etcd client to a version implementing
TryLock.

Use TryLock everywhere where it'll be useful.
 2019-11-12 22:27:17 +01:00
Simone Gotti
24a9563872
Merge pull request #167 from sgotti/datamanager_writedatasnapshot_skip_already_checkpointed_wals 
datamanager: skip already applied wals in writeDataSnapshot
 2019-11-12 14:57:42 +01:00
Simone Gotti
2076b171eb
Merge pull request #172 from sgotti/readdb_improve_handleevents_goroutines_exit 
readdb: improve HandleEvents goroutine exiting
 2019-11-12 14:57:22 +01:00
Simone Gotti
d679254516 readdb: improve HandleEvents goroutine exiting 
Rename errCh to doneCh (error is not needed) and always send to it when one of
the HandleEvents functions exits (not only on error).

This will ensure that all the goroutines will be stopped also if one of them
returns without an error.
 2019-11-12 11:03:21 +01:00
Simone Gotti
1e70e3404b datamanager: skip already applied wals in writeDataSnapshot 
As an optimization don't apply already applied wals.
 2019-11-12 11:02:52 +01:00
Simone Gotti
aff44f7e89
Merge pull request #174 from sgotti/datamanager_test_increas_etcd_waitdown_timeout 
datamanager tests: increase etcd waitdown timeout
 2019-11-12 11:02:12 +01:00
Simone Gotti
dfd0f8c806 datamanager tests: increase etcd waitdown timeout 2019-11-12 10:47:28 +01:00
Simone Gotti
581a610523
Merge pull request #169 from sgotti/improve_error_handling 
*: improve error handling
 2019-11-11 14:08:39 +01:00
Simone Gotti
72f279c4c3 *: improve error handling 
* objectstorage: remove `types` package and move `ErrNotExist` in base package
* objectstorage: Implement .Is and add helper `IsErrNotExist` for `ErrNotExist`
* util: Rename `ErrNotFound` to `ErrNotExist`
* util: Add `IsErr*` helpers and use them in place of `errors.Is()`
* datamanager: add `ErrNoDataStatus` to report when there's not data status in ost
* runservice/common: remove `ErrNotExist` and use errors in util package
 2019-11-11 12:17:35 +01:00
Simone Gotti
89eb87a827
Merge pull request #170 from sgotti/objectstorage_use_single_package 
objectstorage: use a single package
 2019-11-11 10:24:54 +01:00
Simone Gotti
5af07d0852 objectstorage: use a single package 
remove all the subpackages and just use a single package
 2019-11-08 16:31:48 +01:00
Simone Gotti
35e1ec0e15 datamanager: remove old storage wals 
Remove all wals not required by the existing data status files and not existing
in etcd.
 2019-11-08 15:39:17 +01:00
Simone Gotti
9fbfeb9d13
Merge pull request #168 from sgotti/datamanager_refactor_readwal 
datamanager: refactor ReadWal
 2019-11-08 14:26:55 +01:00
Simone Gotti
9c0eb3d7ef datamanager: refactor ReadWal 
make ReadWal directly return a *WalHeader
 2019-11-08 13:24:43 +01:00
Simone Gotti
cd5c5a20ed
Merge pull request #165 from sgotti/datamanager_check_wal_previousequence_is_correct 
datamanager: check wal previouswalsequence is correct in initEtcd
 2019-11-08 09:40:44 +01:00
Simone Gotti
9c1f3b2a69 datamanager: check wal previouswalsequence is correct in initEtcd 2019-11-07 17:05:40 +01:00
Simone Gotti
33e2d50036
Merge pull request #164 from sgotti/datamanager_dont_create_ost_checkpointed_files 
datamanager: don't create ost wal checkpointed files
 2019-11-07 14:48:12 +01:00
Simone Gotti
acd62a3f90 datamanager: don't create ost wal checkpointed files 
currently creating .checkpointed files in the ost isn't useful.
We already have the data snapshot that reports the last checkpointed wal.
 2019-11-07 10:32:41 +01:00
Simone Gotti
ae8eec94b5
Merge pull request #160 from sgotti/datamanager_remove_old_data_files 
datamanager: clean old data files
 2019-11-07 10:30:54 +01:00
Simone Gotti
4fcb067052 datamanager: clean old data files 
keep the last n (now set to 3) data status files and remove all other data status
files and unneeded data files.
 2019-11-07 10:30:31 +01:00
Simone Gotti
156995a804
Merge pull request #163 from sgotti/datamanager_fix_index_pos 
datamanager: fix index creation on multiple data files
 2019-11-06 17:50:12 +01:00
Simone Gotti
52cb683267 datamanager: fix index creation on multiple data files 
When during a checkpoint more than one file is created the entries position in
the index is not right since it's not reset at every new index.

Fix it and add related tests.
 2019-11-06 15:33:40 +01:00
Simone Gotti
778bc93755
Merge pull request #162 from camandel/config_enabled_services 
services: check config only for enabled services
 2019-11-06 13:58:21 +01:00
Carlo Mandelli
aab2321d58 services: check config only for enabled services 2019-11-05 09:25:22 +01:00
Simone Gotti
b78e0b5f33
Merge pull request #158 from sgotti/datamanager_add_datasequence_datafile_name 
datamanager: add data sequence to data file name
 2019-11-04 10:20:49 +01:00
Simone Gotti
4b4416fc99 datamanager: add data sequence to data file name 
When creating a datafile name make it start with the current data sequence. This
is useful in future to know which data sequence created a new data file.
 2019-11-04 09:23:12 +01:00