Implement a new error handling library based on pkg/errors. It provides
stack saving on wrapping and exports some function to add stack saving
also to external errors.
It also implements custom zerolog error formatting without adding too
much verbosity by just printing the chain error file:line without a full
stack trace of every error.
* Add a --detailed-errors options to print error with they full chain
* Wrap all error returns. Use errors.WithStack to wrap without adding a
new messsage and error.Wrap[f] to add a message.
* Add golangci-lint wrapcheck to check that external packages errors are
wrapped. This won't check that internal packages error are wrapped.
But we want also to ensure this case so we'll have to find something
else to check also these.
Replace zap with zerolog.
zerolog has a cleaner interface and can be easily configured with custom
error chain printing using a new error handling library that will be
implemented in another PR.
* Create an APIError that should only be used for api returned errors.
It'll wrap an error and can have different Kinds and optional code and
message.
* The http handlers will use the first APIError available in the
error chain and generate a json response body containing the code and
the user message. The wrapped error is internal and is not sent in the
response.
If no api error is available in the chain a generic internal
server error will be returned.
* Add a RemoteError type that will be created from remote services calls
(runservice, configstore). It's similar to the APIError but a
different type to not propagate to the caller response and it'll not
contain any wrapped error.
* Gateway: when we call a remote service, by default, we'll create a
APIError using the RemoteError Kind (omitting the code and the
message that usually must not be propagated).
This is done for all the remote service calls as a starting point, in
future, if this default behavior is not the right one for a specific
remote service call, a new api error with a different kind and/or
augmented with the calling service error codes and user messages could
be created.
* datamanager: Use a dedicated ErrNotExist (and converting objectstorage
ErrNotExist).
Replace https://github.com/satori/go.uuid with maintained version at
https://github.com/gofrs/uuid
Since the new version uuid.NewV4 returns an error when failing to read
from the random source reader we use uuid.Must to panic on error since
it's considered an unrecoverable error.
In future, if needed, we could handle the error instead of panicking.
During tests provide a zaptest Logger so all services output will be redirected
to golang testing logger.
When multiple services of the same type are provided add a unique name field to
distinguish them.
Instead of doing the current hack of copying the agola toolbox inside the host
tmp dir (always done but only needed when running the executor inside a docker
container) that has different issues (like tmp file removal done by
tmpwatch/systemd-tmpfiles), use a solution similar to the k8s driver: for every
pod create a volume containing the agola-toolbox and remove it at pod removal.
We could also use a single "global" volume but we should handle cases like
volume removal (i.e. a docker volume prune command). So for now just create a
dedicated per pod volume.
* Add a generic container volume option that currently only support tmpfs. In
future it could be expanded to use of host volumes or other kind of volumes (if
supported by the underlying executor)
* Implement creation of tmpfs volumes in docker and k8s drivers.
In c1ff28ef9f we exported various types. Unfortunately the types used by cmd
variable create/update are the wrong types and marshalling fails. Fix it using
the right type. In future this internal types should be exported.
Older version of docker doesn't support the exec api Env and WorkingDir options.
Support these versions by doing the same we already do with the k8s driver: use
the `toolbox exec` command that will set the provided Env, change the cwd to the
WorkingDir and the exec the wanted command.
Export clients and related packages.
The main rule is to not import internal packages from exported packages.
The gateway client and related types are totally decoupled from the gateway
service (not shared types between the client and the server).
Instead the configstore and the runservice client currently share many types
that are now exported (decoupling them will require that a lot of types must be
duplicated and the need of functions to convert between them, this will be done
in future when the APIs will be declared as stable).
Before kubernetes 1.14 nodes were labeled with the "beta.kubernetes.io/arch"
label instead of the "kubernetes.io/arch".
Current k8s version (v1.15) labels nodes with both labels but it's
deprecated and will removed in future versions.
At driver start get the current k8s api version and choose the right label to
use as node selector based on it.
Just a raw replace of "github.com/pkg/errors".
Next steps will improve errors (like remote errors, api errors, not exist errors
etc...) to leverage its functionalities
This was already defined in the config but not implemented in the executor and
drivers.
All the containers defined in the runtime after the first one will be "service"
containers. They will share the same network namespace with the other containers
in the "pod" so they can communicate between themself on loopback
Also if they are logically part of the runservice the names runserviceExecutor
and runserviceScheduler are long and quite confusing for an external user
Simplify them separating both the code parts and updating the names:
runserviceScheduler -> runservice
runserviceExecutor -> executor
Simone Gotti
Carlo Mandelli