Commit Graph

722 Commits

Author SHA1 Message Date
Simone Gotti 16820e9033 readdb: insert current wal sequence after checking wal status 2019-07-18 16:44:28 +02:00
Simone Gotti f7175c4592 datamanager: save previous wal in waldata 2019-07-18 16:44:28 +02:00
Simone Gotti 18c5ae0492 datamanager: error if there're wals but not a datastatus in ost 2019-07-18 16:44:28 +02:00
Simone Gotti c034819087 datamanager: accept optional datastatus in initEtcd 2019-07-18 16:44:28 +02:00
Simone Gotti df716fccc6 datamanager: create a new wal and checkpoint in initEtcd
When doing an initEtcd (new instance or etcd reset) create a new wal (that will
have a new sequence epoch) and do a checkpoint.

In this way:

* readdb will detect that an epoch change and do a full resync
* we always have a data file (also if empty) that provides the last checkpointed
wal. This information could be used by readdb to resync
2019-07-18 16:44:28 +02:00
Simone Gotti cb2a871be6 datamanager: start initEtcd from last datastatus 2019-07-18 16:44:28 +02:00
Simone Gotti 445ef24daa datamanager: add option to force a checkpoint 2019-07-18 16:44:27 +02:00
Simone Gotti 512162bf98 datamanager: clean etcd data before reinitialization 2019-07-18 16:42:17 +02:00
Simone Gotti 8fbb9fdcbe datamanager: add deleteEtcd method 2019-07-18 16:40:28 +02:00
Simone Gotti 21964f90fd etcd: add DeletePrefix method 2019-07-18 16:40:28 +02:00
Simone Gotti 1927a17857
Merge pull request #52 from sgotti/readme_add_features
README: Add "features" and "documentation" sections
2019-07-15 17:31:55 +02:00
Simone Gotti e3e9e162cd README: Add "features" and "documentation" sections 2019-07-15 17:18:09 +02:00
Simone Gotti 44adad61a4 README: add link to agola introduction post 2019-07-15 00:15:40 +02:00
Simone Gotti 94462e6ee4 README: add screenshot 2019-07-14 18:15:00 +02:00
Simone Gotti 86d822a247 service: handle cors config and use it only on gateway
* Don't make cors enabled on all (*) by default.
* Handle related web.allowedOrigins options
* Only the gateway api should be called by a browser so setup the cors handler
only on it
2019-07-13 23:15:00 +02:00
Simone Gotti 5483620fb0 gateway: return project if the project is public
If the project is public don't require the user to be logged in to show the
project.
2019-07-12 23:41:34 +02:00
Simone Gotti f61a6068c2 cmd/serve: don't make --config required 2019-07-12 17:26:00 +02:00
Simone Gotti 973c72976c README: add contacts and contributing info 2019-07-12 10:13:53 +02:00
Simone Gotti cf0b7f8ded add initial integration tests 2019-05-29 13:35:20 +02:00
Simone Gotti b80710e2f0 k8s driver: start goroutines after setup 2019-07-10 21:55:26 +02:00
Simone Gotti 940264e413 runservice: add lock around compatchangegroups
just to avoid concurrency errors when multiple instances are running
2019-07-10 10:20:35 +02:00
Simone Gotti c064e490b2 datamanager: add lock around compatchangegroups and initetcd
just to avoid concurrency errors when multiple instances are running
2019-07-10 10:20:03 +02:00
Simone Gotti 276c045c3b configstore: add parenttype to related tables
Add the parenttype column to project, projectgroup, secret, variable tables.
2019-07-09 10:16:45 +02:00
Simone Gotti 8b7a5602e0 configstore: fix project/projectgroup rename
project: check name exists only if the name has changed
projectgroup: add missing duplicate name check
2019-07-08 18:12:42 +02:00
Simone Gotti 25322a6d81 github: fix retrieving commit sha for signed tags
if a tag is a signed tag, github's hook.After points to the signed tag object
sha and not the related commit sha. In this case use hook.HeadCommit.ID.
2019-07-08 17:21:24 +02:00
Simone Gotti 11a2ff48d6 runservice: delete executor task early
currently we are deleting the executor tasks only when all the run tasks
log/archives were fetched. But it'll better to remove a single executor task
when the task fetching is finished.

This could also fix possible issues on k8s since we are scheduling tasks but the
k8s scheduler may not schedule them if there aren't enough resources causing a
scheduling deadlock since we won't remove finished pods because their related
tasks are not removed and k8s cannot start new pods since it has no resources.
2019-07-08 16:03:14 +02:00
Simone Gotti 7aea06fce8 k8s driver: use right node selector on old k8s versions
Before kubernetes 1.14 nodes were labeled with the "beta.kubernetes.io/arch"
label instead of the "kubernetes.io/arch".

Current k8s version (v1.15) labels nodes with both labels but it's
deprecated and will removed in future versions.

At driver start get the current k8s api version and choose the right label to
use as node selector based on it.
2019-07-08 13:51:31 +02:00
Simone Gotti 378ee352c0 cmd: implement secret update 2019-07-08 10:32:59 +02:00
Simone Gotti 34d36e9e24 gateway: implement secret update 2019-07-08 10:32:45 +02:00
Simone Gotti 289e691a58 configstore: implement secret update 2019-07-08 10:32:32 +02:00
Simone Gotti 6508fd2b53 secret create: read values from yaml file/stdin 2019-07-08 09:58:11 +02:00
Simone Gotti 95f0df8576 README: add agola run status badge 2019-07-08 09:39:52 +02:00
Simone Gotti 45a460ebc0 runservice: handle run not existing
Check if the response from the readdb is null and return an http not found error
2019-07-08 09:30:15 +02:00
Simone Gotti 8c2f37d47c cmd: implement variable update 2019-07-06 15:25:57 +02:00
Simone Gotti 49c1c263fd gateway: implement variable update 2019-07-06 15:25:45 +02:00
Simone Gotti ac0c1fc2bd configstore: implement variable update 2019-07-06 15:25:24 +02:00
Simone Gotti 28c44ce1fc configstore: update/fix remotesource rename
* Override the provided remotesource id with the current one (it could not be
provided or provided with a different id but the remotesource ref is the way to
get the current remote source).

* When changing remotesource name check that a remote source with the new name
does not already exist.
2019-07-06 14:40:31 +02:00
Simone Gotti 2c042fb4bd configstore: add remote source tests 2019-07-06 14:39:37 +02:00
Simone Gotti c7aca42f1f variable create: read values from yaml file/stdin 2019-07-05 18:34:40 +02:00
Simone Gotti ee5c829653 *: add remote source option to disable registration/login
* Make the new fields RegistrationEnabled/LoginEnabled in types.RemoteSource
bool pointers (since they are new fields that don't exist in previously saved
remote sources) and default them to true if null when unmarshaling (or existing
remotesources will have registration and login disabled)

* Add options to cmd remotesource create/update to set the registration/login
disabled.
2019-07-05 13:33:29 +02:00
Simone Gotti e7864d0d84 gateway: add getruns filter by result 2019-07-05 10:42:45 +02:00
Simone Gotti 04ef20464d runservice: add getruns filter by result 2019-07-05 10:32:51 +02:00
Simone Gotti 9d71f32368 cmd: don't set remotesourcecreate api-url flag as required
check it later since for github it could not be provided and will default to
"https://api.github.com"
2019-07-05 10:16:37 +02:00
Simone Gotti f861e2a647 cmd: fix run list 2019-07-04 16:22:11 +02:00
Simone Gotti 929a6fb654 service/*: log error only if nil 2019-07-04 15:50:37 +02:00
Simone Gotti 0ec0a37fa3 config: remove unused gatewayURL from gitserver config 2019-07-04 10:55:12 +02:00
Simone Gotti ea025638ff configstore: set base path for datamanager
Don't put datamanager base dirs inside the root of the ost but use a base path.

Let's do it now before releasing since this is a breaking change that requires
moving the ost data to the new path
2019-07-03 17:19:40 +02:00
Simone Gotti 5db23410d0 runservice: set base path for datamanager
Don't put datamanager base dirs inside the root of the ost but use a base path.

Let's do it now before releasing since this is a breaking change that requires
moving the ost data to the new path
2019-07-03 17:18:21 +02:00
Simone Gotti d989fe9639 datamanager: always handle basepath
Currently we aren't setting a basepath and it wasn't always correctly handled.
Fix missing basepath handling and improve tests to also use a non empty
basepath.
2019-07-03 17:03:37 +02:00
Simone Gotti abab40c6f5 gateway/cmd: generate CacheGroup for user direct runs
Since the user direct runs all belong to the same run group (the user id) all
the user direct runs will share the same caches. To distinguish between the
different caches we need to use something in addition to the user id. In this
case we are usin the local repo uuid generated by the direct run start command.
2019-07-03 15:19:52 +02:00