Commit Graph

183 Commits

Author SHA1 Message Date
Finn-Thorben Sell 8cb8973846 add content-type header for /config.js 2020-10-30 19:41:52 +01:00
Simone Gotti d91ec09d7d config: add starlark config support
Handle `.agola/config.star` files in starlark config format.
To provide a context like done for jsonnet we require that the starlark agola
config file contains a main function that will receive a config context as a
dict.
We also had to implement our own json conversion from a starlark dict since go
starlark removed its own function.
2020-03-10 13:29:20 +01:00
Simone Gotti 7094de908b run: export also ref type environment variable to runs 2020-03-09 10:47:58 +01:00
Simone Gotti 627f40987e run: export also pull request id environment variable to runs 2020-03-09 09:42:51 +01:00
Simone Gotti 1820c0247c config: provide jsonnet context top level argument 2020-03-06 11:19:49 +01:00
Simone Gotti eb48e73a54 gateway: move authentication apis to /api/v1alpha/auth
Move the various authentication apis to /api/v1alpha/auth since they should be
versioned like other apis.
2020-02-19 10:48:14 +01:00
Carlo Mandelli 182eb14b20 cmd: project option to disable passing variables to PR from forked repo 2020-01-28 09:02:37 +01:00
Simone Gotti 2de91549a3 tests: improve services logging
During tests provide a zaptest Logger so all services output will be redirected
to golang testing logger.

When multiple services of the same type are provided add a unique name field to
distinguish them.
2020-01-15 12:30:34 +01:00
Carlo Mandelli 3e47bc601a gateway/runservice: add api to delete step logs 2019-11-18 10:34:56 +01:00
Simone Gotti f7d0950ca1 *: write and flush header on log handlers
Explicitly write and flush the headers in the various services LogHandlers.

Currently the 200 response and the other headers will be automatically written
by the golang http implementation only when we send something in the body. But if
there's nothing to send (no logs yet written) the client will never receive the
headers and cannot know if the request was successful.
2019-11-14 10:52:45 +01:00
Carlo Mandelli 8ed813e722 gateway: add api to get log status 2019-11-13 10:01:51 +01:00
Simone Gotti 72f279c4c3 *: improve error handling
* objectstorage: remove `types` package and move `ErrNotExist` in base package
* objectstorage: Implement .Is and add helper `IsErrNotExist` for `ErrNotExist`
* util: Rename `ErrNotFound` to `ErrNotExist`
* util: Add `IsErr*` helpers and use them in place of `errors.Is()`
* datamanager: add `ErrNoDataStatus` to report when there's not data status in ost
* runservice/common: remove `ErrNotExist` and use errors in util package
2019-11-11 12:17:35 +01:00
Simone Gotti e18794764e go.mod: update dependencies
Update all the updatable dependencies
2019-10-29 09:31:38 +01:00
Simone Gotti 446e626f9f gateway: fix project create run http method
Make it a POST instead of a PUT.
2019-10-22 09:50:01 +02:00
Simone Gotti fa4b41ab74 when: match only the current ref type
Only match the current ref type, ie: don't match a branch when the ref type is a
tag or pull request.

Ref is always matched because it's not related to a specific ref type.
2019-10-14 17:08:44 +02:00
Carlo Mandelli 4fc8f3ebed cmd: add version command 2019-10-05 12:11:18 +02:00
Simone Gotti 11da186913 gateway: add parentRef field to project group update api
and make all the request fields optional
2019-10-01 14:39:45 +02:00
Simone Gotti 8b9464486d gateway: add parentRef field to project update api
and make all the request fields optional
2019-09-27 17:10:04 +02:00
Simone Gotti f5c0e91f39 gateway: api implement get secrets removeoverridden
Implement missing removeoverridden option for get secrets.
2019-09-20 15:39:21 +02:00
Carlo Mandelli 436aa8f1de skip run with special commit 2019-09-18 11:53:22 +02:00
Simone Gotti 9cfb21d365 gateway: run api return more step data
For every step return the step type.

For a run step also return the shell and exit status.
2019-09-17 17:43:48 +02:00
Simone Gotti 70eeddb719 types: use a global When type
Currently we are using different `When` types for every service and convert
between them. This is a good approach if we want to keep isolated all the
services (like if we were using different repos for every service instead of the
current monorepo).

But currently, since When is identical between all the services, simplify this by
using a common When type.
2019-09-05 09:37:27 +02:00
Simone Gotti 53dad95cd0 cmd: fix variable create/update
In c1ff28ef9f we exported various types. Unfortunately the types used by cmd
variable create/update are the wrong types and marshalling fails. Fix it using
the right type. In future this internal types should be exported.
2019-08-29 16:38:19 +02:00
Simone Gotti 2676770336 userdirectrun: add options to define variables
Add a --var and --var-file options (repeatable multiple times) to define the
variables to be used in the run.
2019-08-06 16:58:00 +02:00
Simone Gotti db742a6cd6 config: add run when field
Don't create a run if a when condition is defined and it doesn't match.
2019-08-05 16:07:47 +02:00
Simone Gotti 4ec0b33eb4 userdirectrun: allow setting destination branch/tag/ref
Allow setting the destination branch/tag/ref so users can test the run
conditions based on the branch/tag/ref.

To simulate a pull request an user can define a ref that matches one of these
regular expressions: `refs/pull/(\d+)/head`, `refs/merge-requests/(\d+)/head`
2019-08-05 14:45:34 +02:00
Simone Gotti c1ff28ef9f *: export clients and related types
Export clients and related packages.

The main rule is to not import internal packages from exported packages.

The gateway client and related types are totally decoupled from the gateway
service (not shared types between the client and the server).

Instead the configstore and the runservice client currently share many types
that are now exported (decoupling them will require that a lot of types must be
duplicated and the need of functions to convert between them, this will be done
in future when the APIs will be declared as stable).
2019-08-02 12:02:01 +02:00
Simone Gotti fd26e617b3 configstore: move configstore types inside configstore package
Since they're not types common to all the services but belongs to the
configstore.

Next step will be to make them local to the configstore and not directly used by
other services since these types are also stored.
2019-08-02 10:05:47 +02:00
Simone Gotti bd035e9840 util: use context in backoff 2019-07-26 10:36:11 +02:00
Simone Gotti b8c2b4020a db: use context functions
Use the go sql context functions (ExecContext, QueryContext etc...)

The context is saved inside Tx so the library users should only pass it one time
to the db.Do function.
2019-07-25 14:49:53 +02:00
Simone Gotti 86d822a247 service: handle cors config and use it only on gateway
* Don't make cors enabled on all (*) by default.
* Handle related web.allowedOrigins options
* Only the gateway api should be called by a browser so setup the cors handler
only on it
2019-07-13 23:15:00 +02:00
Simone Gotti 5483620fb0 gateway: return project if the project is public
If the project is public don't require the user to be logged in to show the
project.
2019-07-12 23:41:34 +02:00
Simone Gotti 34d36e9e24 gateway: implement secret update 2019-07-08 10:32:45 +02:00
Simone Gotti 49c1c263fd gateway: implement variable update 2019-07-06 15:25:45 +02:00
Simone Gotti ee5c829653 *: add remote source option to disable registration/login
* Make the new fields RegistrationEnabled/LoginEnabled in types.RemoteSource
bool pointers (since they are new fields that don't exist in previously saved
remote sources) and default them to true if null when unmarshaling (or existing
remotesources will have registration and login disabled)

* Add options to cmd remotesource create/update to set the registration/login
disabled.
2019-07-05 13:33:29 +02:00
Simone Gotti e7864d0d84 gateway: add getruns filter by result 2019-07-05 10:42:45 +02:00
Simone Gotti 04ef20464d runservice: add getruns filter by result 2019-07-05 10:32:51 +02:00
Simone Gotti 929a6fb654 service/*: log error only if nil 2019-07-04 15:50:37 +02:00
Simone Gotti abab40c6f5 gateway/cmd: generate CacheGroup for user direct runs
Since the user direct runs all belong to the same run group (the user id) all
the user direct runs will share the same caches. To distinguish between the
different caches we need to use something in addition to the user id. In this
case we are usin the local repo uuid generated by the direct run start command.
2019-07-03 15:19:52 +02:00
Simone Gotti c5abbee3d8 gateway: fix linter errors
Fix errors reported by default golangci-lint linters
2019-07-02 15:35:14 +02:00
Simone Gotti 5c2bf8642f gateway: show run/tasks/steps if the project is public
If the project is public don't require the user to be logged in to show the
run/tasks/logs
2019-07-01 16:40:02 +02:00
Simone Gotti 8d67844cc4 *: use vanity url
use agola.io domain
2019-07-01 11:40:20 +02:00
Simone Gotti 0c30839f5c gateway: cleanup remote repo on project delete 2019-06-14 15:17:19 +02:00
Simone Gotti f2be16ce4d gateway: cleanup project if remote setup fails
If the remote setup steps fails (create webhook and deploy key) try to delete
project and cleanup remote repo.
2019-06-14 14:57:59 +02:00
Simone Gotti 5e21089baa *: remove unneeded logging
remove many log.Info entries that where old debugging entries and move some of
them to the Debug level.
2019-06-14 11:28:00 +02:00
Simone Gotti 48411f4c64 gateway: move webhook inside api package 2019-06-11 17:07:08 +02:00
Simone Gotti 369f116bfc gateway webhook: remove handling of user direct runs 2019-06-11 15:17:08 +02:00
Simone Gotti 2a95c93f0d gateway: add UserCreateRun api and related action
add an api to execute a user direct run. This method will replace the current
custom webhook way to create a direct run.
2019-06-11 15:09:41 +02:00
Simone Gotti cff6b8d531 *: rename user local run to user direct run 2019-06-11 12:09:57 +02:00
Simone Gotti 98c2f76f5d gateway: add manual run creation
Add an API and related action to manually create a run from a git branch/tag/ref
with optional commitSHA.

Currently only branches and tags are supported (no pull requests).
If not commitSHA is provided the commit sha referenced by the provided branch/tag/ref is
used.
2019-06-11 11:08:40 +02:00