Commit Graph

31 Commits

Author SHA1 Message Date
Simone Gotti
53dad95cd0 cmd: fix variable create/update
In c1ff28ef9f we exported various types. Unfortunately the types used by cmd
variable create/update are the wrong types and marshalling fails. Fix it using
the right type. In future this internal types should be exported.
2019-08-29 16:38:19 +02:00
Simone Gotti
2676770336 userdirectrun: add options to define variables
Add a --var and --var-file options (repeatable multiple times) to define the
variables to be used in the run.
2019-08-06 16:58:00 +02:00
Simone Gotti
e31b0b47ef executor: listen on wildcard address
Since the current logic is to use the first available private ip address as the
advertized address we have to listen on wildcard since a different host provided
in web.ListenAddress will make the executor unreachable.

In future improve this to let the user to manually define the bind and the
advertized address (perhaps using go-sockaddr templates like done by consul) to
also support nat between the schedulers and the executors.
2019-08-06 13:42:42 +02:00
Simone Gotti
1c96b5fbff
Merge pull request #81 from sgotti/docker_driver_use_fixed_client_api_version
docker driver: use fixed client api version
2019-08-04 23:56:17 +02:00
Simone Gotti
df66cfc736 docker driver: use fixed client api version
Set the client required api version to 1.26. In this way we'll work with docker
>= 1.13.1
2019-08-04 23:38:20 +02:00
Simone Gotti
b3672bf927 docker driver: use toolbox exec
Older version of docker doesn't support the exec api Env and WorkingDir options.

Support these versions by doing the same we already do with the k8s driver: use
the `toolbox exec` command that will set the provided Env, change the cwd to the
WorkingDir and the exec the wanted command.
2019-08-04 18:09:34 +02:00
Simone Gotti
c1ff28ef9f *: export clients and related types
Export clients and related packages.

The main rule is to not import internal packages from exported packages.

The gateway client and related types are totally decoupled from the gateway
service (not shared types between the client and the server).

Instead the configstore and the runservice client currently share many types
that are now exported (decoupling them will require that a lot of types must be
duplicated and the need of functions to convert between them, this will be done
in future when the APIs will be declared as stable).
2019-08-02 12:02:01 +02:00
Simone Gotti
d0c5621201 util: remove time.go
The same function is already provided by pointer.go
2019-08-01 14:14:56 +02:00
Simone Gotti
e48a28d5b9
Merge pull request #71 from sgotti/executor_set_task_endtime_when_marking_task_failed
executor: set task endTime when marking as failed
2019-07-29 14:43:13 +02:00
Simone Gotti
0ecfc24def executor: fix typo in setting task endTime when setup failed
There was a typo so we weren't setting the task endTime when the setup step
failed.

Also unify all logic to just use `et` (instead of a mix of `et` or `rt.et`)
2019-07-29 10:00:32 +02:00
Simone Gotti
1707be9528 executor: set task endTime when marking as failed
Add missing set of task endTime when the executor is marking the task as failed
due to no related running task (usually after executor restart).
2019-07-29 09:58:17 +02:00
Simone Gotti
6f3798e8fe *: use sleep timer in loops
So we'll react instantly to a context cancel instead of waiting on time.Sleep
returning.
2019-07-25 16:22:54 +02:00
Simone Gotti
b80710e2f0 k8s driver: start goroutines after setup 2019-07-10 21:55:26 +02:00
Simone Gotti
7aea06fce8 k8s driver: use right node selector on old k8s versions
Before kubernetes 1.14 nodes were labeled with the "beta.kubernetes.io/arch"
label instead of the "kubernetes.io/arch".

Current k8s version (v1.15) labels nodes with both labels but it's
deprecated and will removed in future versions.

At driver start get the current k8s api version and choose the right label to
use as node selector based on it.
2019-07-08 13:51:31 +02:00
Simone Gotti
929a6fb654 service/*: log error only if nil 2019-07-04 15:50:37 +02:00
Simone Gotti
f152b6a6da executor: fix linter errors
Fix errors reported by default golangci-lint linters
2019-07-02 15:18:52 +02:00
Simone Gotti
8d67844cc4 *: use vanity url
use agola.io domain
2019-07-01 11:40:20 +02:00
Simone Gotti
0296d594b5 executor: add option to allow privileged containers
* add a config option allowPrivilegedContainers
* fail task setup if privileged containers are requested but they aren't
allowed.
* report if privileged containers are allowed to the runservice
2019-06-13 18:31:08 +02:00
Simone Gotti
57446f7dcd executor: fix task status update when runservice is unavailable
* don't remove the runningTask when executeTask finishes but just mark the
runningTask a not executing
* add a loop to periodically update executorTask status and remove the
runningTask if not executing and status update was successful
* remove runningTask when it disappears from the runservice
2019-06-13 12:39:34 +02:00
Simone Gotti
d6feb10e8f executor: rename/refactor executeTasksInternal to executeTaskSteps
* directly pass runningTask instead of executorTask
2019-06-13 12:34:31 +02:00
Simone Gotti
11ae921364 docker driver: allow multiple executors on same docker daemon
This is useful primarily for testing purposes.

Add an executorID label to created container to filter out containers not
matching our executorID.
2019-06-12 17:29:43 +02:00
Simone Gotti
629cf2be82 executor: reuse taskPath function 2019-06-12 16:16:53 +02:00
Simone Gotti
9b2ce717c7 *: migrate to "golang.org/x/xerrors"
Just a raw replace of "github.com/pkg/errors".

Next steps will improve errors (like remote errors, api errors, not exist errors
etc...) to leverage its functionalities
2019-05-23 11:23:14 +02:00
Simone Gotti
0e10a406f9 *: remove server sent events from logs handlers
Just use basic http streaming and send all the data as it's available without
splitting by new lines
2019-05-19 14:35:04 +02:00
Simone Gotti
faded9b809 docker driver: handle already deleted containers
don't expect that all the containers of a pod are available. Some could be
already be removed. Improve the logic of sorting containers by their index.
2019-05-15 15:55:08 +02:00
Simone Gotti
821e371cd8 executor: don't start task if max tasks limit has been reached 2019-05-15 15:06:46 +02:00
Simone Gotti
caa9d74b72 runservice: implement additional service containers
This was already defined in the config but not implemented in the executor and
drivers.

All the containers defined in the runtime after the first one will be "service"
containers. They will share the same network namespace with the other containers
in the "pod" so they can communicate between themself on loopback
2019-05-10 17:31:23 +02:00
Simone Gotti
7adca3ea30 runservice drivers: define ExecConfig AttachStdin 2019-05-10 16:50:31 +02:00
Simone Gotti
da224c7580 executor: make docker driver logger sugared 2019-05-09 23:44:41 +02:00
Simone Gotti
4e785e4851 runservice: build and use multiple toolboxes per architecture 2019-05-09 12:36:30 +02:00
Simone Gotti
1e34dca95d runservice: split and simplify scheduler and executor naming
Also if they are logically part of the runservice the names runserviceExecutor
and runserviceScheduler are long and quite confusing for an external user

Simplify them separating both the code parts and updating the names:

runserviceScheduler -> runservice
runserviceExecutor -> executor
2019-05-07 23:56:10 +02:00