skip fetching of tasks with status skipped, not only tasks marked as skip.
This avoid many wrong an noisy logs of type "executor task with id taskid
doesn't exist. This shouldn't happen. Skipping fetching"
updateRunTaskStatus should also accept transitions from not started to a
finished state like "success", "failed", "stopped" since we could miss some
status updates from the executor for many reasons.
rename activeExecutorTasks to scheduledExecutorTasks and don't filter out
finished tasks.
In some logic we need all the scheduled tasks and not only the not finished
ones.
taskUpdater will be called serially and won't block. It'll execute a goroutine
for executing the task and for sending the task state to the scheduler.
executeTask will just start task execution, all the logic of choosing if
starting a task is moved inside taskUpdater
In this way we avoid concurrency issues when handling the same executorTask
in parallel
Since the executor only periodically updates its state we could end up
scheduling much more tasks than the executor ActiveTasksLimit. This will happen
in the case of many parallel tasks that can all start at the same time.
To avoid this also considere the executor tasks saved in etcd that represent
the real view of scheduled tasks.
Currently when a run is marked to stop we are going to stop currently running
tasks and then their childs will be marked as skipped.
But tasks not depending on a stopped task (root task or childs with a finished
parent) that are just waiting for an executor slot, will be scheduled when
there will be a free slot also if the run is marked to stop (and then the
scheduler will stop them after some seconds).
This patch will mark all not started tasks as skipped when the run is marked to
stop.
Use default public github api only if no api-url is provided.
Use default public github ssh host key only if the public github api url is
provided and no ssh host key is provided.
There's no test script. Full tests can be easily executed inside a local
running agola instance using an agola "direct run".
Single package tests can be executed using a manual go test invocation based on
the kind of package we want to test (since some of them require a running etcd,
a running docker, a running k8s cluster, a gitea binary etc...)
During tests provide a zaptest Logger so all services output will be redirected
to golang testing logger.
When multiple services of the same type are provided add a unique name field to
distinguish them.
Instead of doing the current hack of copying the agola toolbox inside the host
tmp dir (always done but only needed when running the executor inside a docker
container) that has different issues (like tmp file removal done by
tmpwatch/systemd-tmpfiles), use a solution similar to the k8s driver: for every
pod create a volume containing the agola-toolbox and remove it at pod removal.
We could also use a single "global" volume but we should handle cases like
volume removal (i.e. a docker volume prune command). So for now just create a
dedicated per pod volume.
Simone Gotti
6543
Carlo Mandelli
MitaliBo