Commit Graph

80 Commits

Author SHA1 Message Date
Simone Gotti d2b09d854f *: use new errors handling library
Implement a new error handling library based on pkg/errors. It provides
stack saving on wrapping and exports some function to add stack saving
also to external errors.
It also implements custom zerolog error formatting without adding too
much verbosity by just printing the chain error file:line without a full
stack trace of every error.

* Add a --detailed-errors options to print error with they full chain
* Wrap all error returns. Use errors.WithStack to wrap without adding a
  new messsage and error.Wrap[f] to add a message.
* Add golangci-lint wrapcheck to check that external packages errors are
  wrapped. This won't check that internal packages error are wrapped.
  But we want also to ensure this case so we'll have to find something
  else to check also these.
2022-02-28 12:49:13 +01:00
Simone Gotti d1b4ab4296 *: use zerolog for logging
Replace zap with zerolog.

zerolog has a cleaner interface and can be easily configured with custom
error chain printing using a new error handling library that will be
implemented in another PR.
2022-02-28 10:40:55 +01:00
Simone Gotti c1da3ab566 *: Improve error handling
* Create an APIError that should only be used for api returned errors.
  It'll wrap an error and can have different Kinds and optional code and
  message.
* The http handlers will use the first APIError available in the
  error chain and generate a json response body containing the code and
  the user message. The wrapped error is internal and is not sent in the
  response.
  If no api error is available in the chain a generic internal
  server error will be returned.
* Add a RemoteError type that will be created from remote services calls
  (runservice, configstore). It's similar to the APIError but a
  different type to not propagate to the caller response and it'll not
  contain any wrapped error.
* Gateway: when we call a remote service, by default, we'll create a
  APIError using the RemoteError Kind (omitting the code and the
  message that usually must not be propagated).
  This is done for all the remote service calls as a starting point, in
  future, if this default behavior is not the right one for a specific
  remote service call, a new api error with a different kind and/or
  augmented with the calling service error codes and user messages could
  be created.
* datamanager: Use a dedicated ErrNotExist (and converting objectstorage
  ErrNotExist).
2022-02-25 16:11:19 +01:00
Simone Gotti 87f182a0c9 *: use errors.Is/errors.As to handle wrapped error checking
Enable golangci-lint errorlint linter to check proper use of errors.Is
and error.As instead of direct comparison or error type casting.
2022-02-24 17:07:29 +01:00
Simone Gotti b7c797bde1 gateway: use dedicated auth context keys type
Use auth context keys dedicated type instead of strings and avoid code
duplication by moving shared code under a common package.
2022-02-24 13:41:57 +01:00
Tullio 7160194104 gateway: add api to get authenticated user organizations
gateway: add api to get authenticated user organizations

gateway: add api to get authenticated user organizations
2022-02-23 15:31:34 +01:00
Simone Gotti 64680468fd *: migrate to golang-jwt/jwt
Migrate to the new community maintained version of github.com/dgrijalva/jwt-go
2021-09-10 12:33:59 +02:00
Simone Gotti d91ec09d7d config: add starlark config support
Handle `.agola/config.star` files in starlark config format.
To provide a context like done for jsonnet we require that the starlark agola
config file contains a main function that will receive a config context as a
dict.
We also had to implement our own json conversion from a starlark dict since go
starlark removed its own function.
2020-03-10 13:29:20 +01:00
Simone Gotti 7094de908b run: export also ref type environment variable to runs 2020-03-09 10:47:58 +01:00
Simone Gotti 627f40987e run: export also pull request id environment variable to runs 2020-03-09 09:42:51 +01:00
Simone Gotti 1820c0247c config: provide jsonnet context top level argument 2020-03-06 11:19:49 +01:00
Carlo Mandelli 182eb14b20 cmd: project option to disable passing variables to PR from forked repo 2020-01-28 09:02:37 +01:00
Carlo Mandelli 3e47bc601a gateway/runservice: add api to delete step logs 2019-11-18 10:34:56 +01:00
Simone Gotti 72f279c4c3 *: improve error handling
* objectstorage: remove `types` package and move `ErrNotExist` in base package
* objectstorage: Implement .Is and add helper `IsErrNotExist` for `ErrNotExist`
* util: Rename `ErrNotFound` to `ErrNotExist`
* util: Add `IsErr*` helpers and use them in place of `errors.Is()`
* datamanager: add `ErrNoDataStatus` to report when there's not data status in ost
* runservice/common: remove `ErrNotExist` and use errors in util package
2019-11-11 12:17:35 +01:00
Simone Gotti e18794764e go.mod: update dependencies
Update all the updatable dependencies
2019-10-29 09:31:38 +01:00
Simone Gotti fa4b41ab74 when: match only the current ref type
Only match the current ref type, ie: don't match a branch when the ref type is a
tag or pull request.

Ref is always matched because it's not related to a specific ref type.
2019-10-14 17:08:44 +02:00
Carlo Mandelli 4fc8f3ebed cmd: add version command 2019-10-05 12:11:18 +02:00
Simone Gotti 11da186913 gateway: add parentRef field to project group update api
and make all the request fields optional
2019-10-01 14:39:45 +02:00
Simone Gotti 8b9464486d gateway: add parentRef field to project update api
and make all the request fields optional
2019-09-27 17:10:04 +02:00
Simone Gotti f5c0e91f39 gateway: api implement get secrets removeoverridden
Implement missing removeoverridden option for get secrets.
2019-09-20 15:39:21 +02:00
Carlo Mandelli 436aa8f1de skip run with special commit 2019-09-18 11:53:22 +02:00
Simone Gotti 70eeddb719 types: use a global When type
Currently we are using different `When` types for every service and convert
between them. This is a good approach if we want to keep isolated all the
services (like if we were using different repos for every service instead of the
current monorepo).

But currently, since When is identical between all the services, simplify this by
using a common When type.
2019-09-05 09:37:27 +02:00
Simone Gotti 2676770336 userdirectrun: add options to define variables
Add a --var and --var-file options (repeatable multiple times) to define the
variables to be used in the run.
2019-08-06 16:58:00 +02:00
Simone Gotti db742a6cd6 config: add run when field
Don't create a run if a when condition is defined and it doesn't match.
2019-08-05 16:07:47 +02:00
Simone Gotti 4ec0b33eb4 userdirectrun: allow setting destination branch/tag/ref
Allow setting the destination branch/tag/ref so users can test the run
conditions based on the branch/tag/ref.

To simulate a pull request an user can define a ref that matches one of these
regular expressions: `refs/pull/(\d+)/head`, `refs/merge-requests/(\d+)/head`
2019-08-05 14:45:34 +02:00
Simone Gotti c1ff28ef9f *: export clients and related types
Export clients and related packages.

The main rule is to not import internal packages from exported packages.

The gateway client and related types are totally decoupled from the gateway
service (not shared types between the client and the server).

Instead the configstore and the runservice client currently share many types
that are now exported (decoupling them will require that a lot of types must be
duplicated and the need of functions to convert between them, this will be done
in future when the APIs will be declared as stable).
2019-08-02 12:02:01 +02:00
Simone Gotti fd26e617b3 configstore: move configstore types inside configstore package
Since they're not types common to all the services but belongs to the
configstore.

Next step will be to make them local to the configstore and not directly used by
other services since these types are also stored.
2019-08-02 10:05:47 +02:00
Simone Gotti bd035e9840 util: use context in backoff 2019-07-26 10:36:11 +02:00
Simone Gotti 34d36e9e24 gateway: implement secret update 2019-07-08 10:32:45 +02:00
Simone Gotti 49c1c263fd gateway: implement variable update 2019-07-06 15:25:45 +02:00
Simone Gotti ee5c829653 *: add remote source option to disable registration/login
* Make the new fields RegistrationEnabled/LoginEnabled in types.RemoteSource
bool pointers (since they are new fields that don't exist in previously saved
remote sources) and default them to true if null when unmarshaling (or existing
remotesources will have registration and login disabled)

* Add options to cmd remotesource create/update to set the registration/login
disabled.
2019-07-05 13:33:29 +02:00
Simone Gotti 04ef20464d runservice: add getruns filter by result 2019-07-05 10:32:51 +02:00
Simone Gotti abab40c6f5 gateway/cmd: generate CacheGroup for user direct runs
Since the user direct runs all belong to the same run group (the user id) all
the user direct runs will share the same caches. To distinguish between the
different caches we need to use something in addition to the user id. In this
case we are usin the local repo uuid generated by the direct run start command.
2019-07-03 15:19:52 +02:00
Simone Gotti c5abbee3d8 gateway: fix linter errors
Fix errors reported by default golangci-lint linters
2019-07-02 15:35:14 +02:00
Simone Gotti 5c2bf8642f gateway: show run/tasks/steps if the project is public
If the project is public don't require the user to be logged in to show the
run/tasks/logs
2019-07-01 16:40:02 +02:00
Simone Gotti 8d67844cc4 *: use vanity url
use agola.io domain
2019-07-01 11:40:20 +02:00
Simone Gotti 0c30839f5c gateway: cleanup remote repo on project delete 2019-06-14 15:17:19 +02:00
Simone Gotti f2be16ce4d gateway: cleanup project if remote setup fails
If the remote setup steps fails (create webhook and deploy key) try to delete
project and cleanup remote repo.
2019-06-14 14:57:59 +02:00
Simone Gotti 5e21089baa *: remove unneeded logging
remove many log.Info entries that where old debugging entries and move some of
them to the Debug level.
2019-06-14 11:28:00 +02:00
Simone Gotti 2a95c93f0d gateway: add UserCreateRun api and related action
add an api to execute a user direct run. This method will replace the current
custom webhook way to create a direct run.
2019-06-11 15:09:41 +02:00
Simone Gotti cff6b8d531 *: rename user local run to user direct run 2019-06-11 12:09:57 +02:00
Simone Gotti 98c2f76f5d gateway: add manual run creation
Add an API and related action to manually create a run from a git branch/tag/ref
with optional commitSHA.

Currently only branches and tags are supported (no pull requests).
If not commitSHA is provided the commit sha referenced by the provided branch/tag/ref is
used.
2019-06-11 11:08:40 +02:00
Simone Gotti 8242dc3a9d gateway: move create run to own action
* Move all run creation logic to the action handler.
* Cleanup webhook to use it
2019-06-11 09:31:12 +02:00
Simone Gotti 4d7605a86b gateway: improve ErrFromRemote handling
Don't create an ErrFromRemote wrapping the returned error but
wrap the ErrFromRemote

Also use xerrors Is/As to get the underlying error to return to api clients
while maintaining context for logging
2019-05-23 12:59:11 +02:00
Simone Gotti 9b2ce717c7 *: migrate to "golang.org/x/xerrors"
Just a raw replace of "github.com/pkg/errors".

Next steps will improve errors (like remote errors, api errors, not exist errors
etc...) to leverage its functionalities
2019-05-23 11:23:14 +02:00
Simone Gotti 8f1225da76 *: implement remote source update 2019-05-23 10:29:03 +02:00
Simone Gotti 933dfae658 cmd/api: add skipVerify remote source option
Add an option to set skipVerify on remote source to disable tls cert
verification on remote source api endpoint
2019-05-22 16:28:42 +02:00
Simone Gotti 0e10a406f9 *: remove server sent events from logs handlers
Just use basic http streaming and send all the data as it's available without
splitting by new lines
2019-05-19 14:35:04 +02:00
Simone Gotti eabf6c8a92 gateway: fix wrong error message 2019-05-15 23:22:05 +02:00
Simone Gotti bda7a3eb8b *: add run cancel action
and remove unused change phase to finished
2019-05-15 14:42:50 +02:00