Commit Graph

478 Commits

Author SHA1 Message Date
Simone Gotti
19706fe485
Merge pull request #57 from sgotti/runservice_stop_also_if_result_is_not_set
runservice: stop run also if result is not set
2019-07-23 15:09:58 +02:00
Simone Gotti
85876310af readdb: fix deadlock in Run method
In runservice readdb Run method we could end with a deadlock if two of the
goroutines that call HandleEvents.* try to write to the errCh at the same
time before the errCh is read. If this happens one of the two will be blocked on
writing to the channel but the read won't happen since it'll blocked by
wg.Wait().

Fix this doing:
* use a buffered channel large as the number of executed goroutines.
* create a new errCh at every loop (so we'll ignore later errors after the first
one)

Note: we could also use a non blocking send to avoid this situation but we
should also start the wg.Wait before the goroutines or earlier errors could be
lost causing another kind of hang.
2019-07-23 14:56:26 +02:00
Simone Gotti
75d68b2b52 runservice: stop run also if result is not set 2019-07-23 12:11:01 +02:00
Simone Gotti
ea3e0d1d7c
Merge pull request #56 from sgotti/datamanager_multiple_improvements
datamanager: multiple improvements
2019-07-19 11:12:16 +02:00
Simone Gotti
3a963ef95f readdb: error if there's no wal in etcd 2019-07-18 16:44:28 +02:00
Simone Gotti
3f64bda0cc readdb: save walSequence provided by data file 2019-07-18 16:44:28 +02:00
Simone Gotti
16820e9033 readdb: insert current wal sequence after checking wal status 2019-07-18 16:44:28 +02:00
Simone Gotti
f7175c4592 datamanager: save previous wal in waldata 2019-07-18 16:44:28 +02:00
Simone Gotti
18c5ae0492 datamanager: error if there're wals but not a datastatus in ost 2019-07-18 16:44:28 +02:00
Simone Gotti
c034819087 datamanager: accept optional datastatus in initEtcd 2019-07-18 16:44:28 +02:00
Simone Gotti
df716fccc6 datamanager: create a new wal and checkpoint in initEtcd
When doing an initEtcd (new instance or etcd reset) create a new wal (that will
have a new sequence epoch) and do a checkpoint.

In this way:

* readdb will detect that an epoch change and do a full resync
* we always have a data file (also if empty) that provides the last checkpointed
wal. This information could be used by readdb to resync
2019-07-18 16:44:28 +02:00
Simone Gotti
cb2a871be6 datamanager: start initEtcd from last datastatus 2019-07-18 16:44:28 +02:00
Simone Gotti
445ef24daa datamanager: add option to force a checkpoint 2019-07-18 16:44:27 +02:00
Simone Gotti
512162bf98 datamanager: clean etcd data before reinitialization 2019-07-18 16:42:17 +02:00
Simone Gotti
8fbb9fdcbe datamanager: add deleteEtcd method 2019-07-18 16:40:28 +02:00
Simone Gotti
21964f90fd etcd: add DeletePrefix method 2019-07-18 16:40:28 +02:00
Simone Gotti
1927a17857
Merge pull request #52 from sgotti/readme_add_features
README: Add "features" and "documentation" sections
2019-07-15 17:31:55 +02:00
Simone Gotti
e3e9e162cd README: Add "features" and "documentation" sections 2019-07-15 17:18:09 +02:00
Simone Gotti
44adad61a4 README: add link to agola introduction post 2019-07-15 00:15:40 +02:00
Simone Gotti
94462e6ee4 README: add screenshot 2019-07-14 18:15:00 +02:00
Simone Gotti
86d822a247 service: handle cors config and use it only on gateway
* Don't make cors enabled on all (*) by default.
* Handle related web.allowedOrigins options
* Only the gateway api should be called by a browser so setup the cors handler
only on it
2019-07-13 23:15:00 +02:00
Simone Gotti
5483620fb0 gateway: return project if the project is public
If the project is public don't require the user to be logged in to show the
project.
2019-07-12 23:41:34 +02:00
Simone Gotti
f61a6068c2 cmd/serve: don't make --config required 2019-07-12 17:26:00 +02:00
Simone Gotti
973c72976c README: add contacts and contributing info 2019-07-12 10:13:53 +02:00
Simone Gotti
cf0b7f8ded add initial integration tests 2019-05-29 13:35:20 +02:00
Simone Gotti
b80710e2f0 k8s driver: start goroutines after setup 2019-07-10 21:55:26 +02:00
Simone Gotti
940264e413 runservice: add lock around compatchangegroups
just to avoid concurrency errors when multiple instances are running
2019-07-10 10:20:35 +02:00
Simone Gotti
c064e490b2 datamanager: add lock around compatchangegroups and initetcd
just to avoid concurrency errors when multiple instances are running
2019-07-10 10:20:03 +02:00
Simone Gotti
276c045c3b configstore: add parenttype to related tables
Add the parenttype column to project, projectgroup, secret, variable tables.
2019-07-09 10:16:45 +02:00
Simone Gotti
8b7a5602e0 configstore: fix project/projectgroup rename
project: check name exists only if the name has changed
projectgroup: add missing duplicate name check
2019-07-08 18:12:42 +02:00
Simone Gotti
25322a6d81 github: fix retrieving commit sha for signed tags
if a tag is a signed tag, github's hook.After points to the signed tag object
sha and not the related commit sha. In this case use hook.HeadCommit.ID.
2019-07-08 17:21:24 +02:00
Simone Gotti
11a2ff48d6 runservice: delete executor task early
currently we are deleting the executor tasks only when all the run tasks
log/archives were fetched. But it'll better to remove a single executor task
when the task fetching is finished.

This could also fix possible issues on k8s since we are scheduling tasks but the
k8s scheduler may not schedule them if there aren't enough resources causing a
scheduling deadlock since we won't remove finished pods because their related
tasks are not removed and k8s cannot start new pods since it has no resources.
2019-07-08 16:03:14 +02:00
Simone Gotti
7aea06fce8 k8s driver: use right node selector on old k8s versions
Before kubernetes 1.14 nodes were labeled with the "beta.kubernetes.io/arch"
label instead of the "kubernetes.io/arch".

Current k8s version (v1.15) labels nodes with both labels but it's
deprecated and will removed in future versions.

At driver start get the current k8s api version and choose the right label to
use as node selector based on it.
2019-07-08 13:51:31 +02:00
Simone Gotti
378ee352c0 cmd: implement secret update 2019-07-08 10:32:59 +02:00
Simone Gotti
34d36e9e24 gateway: implement secret update 2019-07-08 10:32:45 +02:00
Simone Gotti
289e691a58 configstore: implement secret update 2019-07-08 10:32:32 +02:00
Simone Gotti
6508fd2b53 secret create: read values from yaml file/stdin 2019-07-08 09:58:11 +02:00
Simone Gotti
95f0df8576 README: add agola run status badge 2019-07-08 09:39:52 +02:00
Simone Gotti
45a460ebc0 runservice: handle run not existing
Check if the response from the readdb is null and return an http not found error
2019-07-08 09:30:15 +02:00
Simone Gotti
8c2f37d47c cmd: implement variable update 2019-07-06 15:25:57 +02:00
Simone Gotti
49c1c263fd gateway: implement variable update 2019-07-06 15:25:45 +02:00
Simone Gotti
ac0c1fc2bd configstore: implement variable update 2019-07-06 15:25:24 +02:00
Simone Gotti
28c44ce1fc configstore: update/fix remotesource rename
* Override the provided remotesource id with the current one (it could not be
provided or provided with a different id but the remotesource ref is the way to
get the current remote source).

* When changing remotesource name check that a remote source with the new name
does not already exist.
2019-07-06 14:40:31 +02:00
Simone Gotti
2c042fb4bd configstore: add remote source tests 2019-07-06 14:39:37 +02:00
Simone Gotti
c7aca42f1f variable create: read values from yaml file/stdin 2019-07-05 18:34:40 +02:00
Simone Gotti
ee5c829653 *: add remote source option to disable registration/login
* Make the new fields RegistrationEnabled/LoginEnabled in types.RemoteSource
bool pointers (since they are new fields that don't exist in previously saved
remote sources) and default them to true if null when unmarshaling (or existing
remotesources will have registration and login disabled)

* Add options to cmd remotesource create/update to set the registration/login
disabled.
2019-07-05 13:33:29 +02:00
Simone Gotti
e7864d0d84 gateway: add getruns filter by result 2019-07-05 10:42:45 +02:00
Simone Gotti
04ef20464d runservice: add getruns filter by result 2019-07-05 10:32:51 +02:00
Simone Gotti
9d71f32368 cmd: don't set remotesourcecreate api-url flag as required
check it later since for github it could not be provided and will default to
"https://api.github.com"
2019-07-05 10:16:37 +02:00
Simone Gotti
f861e2a647 cmd: fix run list 2019-07-04 16:22:11 +02:00