a
/
agola
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
agola/internal/config/config_test.go

616 lines
19 KiB
Go
Raw Normal View History

Add initial agola config format and handling
2019-03-07 13:42:32 +00:00
// Copyright 2019 Sorint.lab
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied
// See the License for the specific language governing permissions and
// limitations under the License.
package config
import (
"fmt"
"testing"
*: use vanity url use agola.io domain
2019-07-01 09:40:20 +00:00
"agola.io/agola/internal/util"
types: use a global When type Currently we are using different `When` types for every service and convert between them. This is a good approach if we want to keep isolated all the services (like if we were using different repos for every service instead of the current monorepo). But currently, since When is identical between all the services, simplify this by using a common When type.
2019-09-05 07:29:20 +00:00
"agola.io/agola/services/types"
*: initial implementation of when conditions
2019-03-07 17:01:34 +00:00
"github.com/google/go-cmp/cmp"
*: migrate to "golang.org/x/xerrors" Just a raw replace of "github.com/pkg/errors". Next steps will improve errors (like remote errors, api errors, not exist errors etc...) to leverage its functionalities
2019-05-23 09:23:14 +00:00
errors "golang.org/x/xerrors"
*: implement ability to add tmpfs volumes to containers * Add a generic container volume option that currently only support tmpfs. In future it could be expanded to use of host volumes or other kind of volumes (if supported by the underlying executor) * Implement creation of tmpfs volumes in docker and k8s drivers.
2019-10-08 13:52:35 +00:00
"k8s.io/apimachinery/pkg/api/resource"
Add initial agola config format and handling
2019-03-07 13:42:32 +00:00
)
func TestParseConfig(t *testing.T) {
tests := []struct {
name string
in string
err error
}{
{
runservice: rename config pipeline(s) to run(s) No need for different names. We are always talking about a run. In the config it's the run definition, in the service is the run execution.
2019-04-15 21:42:10 +00:00
name: "test no runs 1",
Add initial agola config format and handling
2019-03-07 13:42:32 +00:00
in: ``,
runservice: rename config pipeline(s) to run(s) No need for different names. We are always talking about a run. In the config it's the run definition, in the service is the run execution.
2019-04-15 21:42:10 +00:00
err: fmt.Errorf(`no runs defined`),
Add initial agola config format and handling
2019-03-07 13:42:32 +00:00
},
{
runservice: rename config pipeline(s) to run(s) No need for different names. We are always talking about a run. In the config it's the run definition, in the service is the run execution.
2019-04-15 21:42:10 +00:00
name: "test no runs 2",
Add initial agola config format and handling
2019-03-07 13:42:32 +00:00
in: `
runservice: rename config pipeline(s) to run(s) No need for different names. We are always talking about a run. In the config it's the run definition, in the service is the run execution.
2019-04-15 21:42:10 +00:00
runs:
Add initial agola config format and handling
2019-03-07 13:42:32 +00:00
`,
runservice: rename config pipeline(s) to run(s) No need for different names. We are always talking about a run. In the config it's the run definition, in the service is the run execution.
2019-04-15 21:42:10 +00:00
err: fmt.Errorf(`no runs defined`),
Add initial agola config format and handling
2019-03-07 13:42:32 +00:00
},
{
runservice: rename config pipeline(s) to run(s) No need for different names. We are always talking about a run. In the config it's the run definition, in the service is the run execution.
2019-04-15 21:42:10 +00:00
name: "test empty run",
Add initial agola config format and handling
2019-03-07 13:42:32 +00:00
in: `
runservice: rename config pipeline(s) to run(s) No need for different names. We are always talking about a run. In the config it's the run definition, in the service is the run execution.
2019-04-15 21:42:10 +00:00
runs:
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
-
Add initial agola config format and handling
2019-03-07 13:42:32 +00:00
`,
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
err: fmt.Errorf(`run at index 0 is empty`),
Add initial agola config format and handling
2019-03-07 13:42:32 +00:00
},
{
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
name: "test empty task",
in: `
runs:
- name: run01
tasks:
config: provide jsonnet context top level argument
2019-10-14 14:36:46 +00:00
-
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
`,
err: fmt.Errorf(`run "run01": task at index 0 is empty`),
},
config: check Runtime.Arch
2019-04-17 13:24:37 +00:00
{
name: "test empty runtime arch",
in: `
runs:
- name: run01
tasks:
- name: task01
runtime:
type: pod
containers:
- image: busybox
`,
},
{
name: "test invalid runtime arch",
in: `
runs:
- name: run01
tasks:
- name: task01
runtime:
type: pod
arch: invalidarch
containers:
- image: busybox
`,
err: fmt.Errorf(`task "task01" runtime: invalid arch "invalidarch"`),
},
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
{
name: "test missing task dependency",
Add initial agola config format and handling
2019-03-07 13:42:32 +00:00
in: `
runservice: rename config pipeline(s) to run(s) No need for different names. We are always talking about a run. In the config it's the run definition, in the service is the run execution.
2019-04-15 21:42:10 +00:00
runs:
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
- name: run01
tasks:
- name: task01
runtime:
type: pod
containers:
- image: busybox
Add initial agola config format and handling
2019-03-07 13:42:32 +00:00
depends:
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
- task02
Add initial agola config format and handling
2019-03-07 13:42:32 +00:00
`,
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
err: fmt.Errorf(`run task "task02" needed by task "task01" doesn't exist`),
Add initial agola config format and handling
2019-03-07 13:42:32 +00:00
},
{
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
name: "test circular dependency between 2 tasks a -> b -> a",
Add initial agola config format and handling
2019-03-07 13:42:32 +00:00
in: `
runservice: rename config pipeline(s) to run(s) No need for different names. We are always talking about a run. In the config it's the run definition, in the service is the run execution.
2019-04-15 21:42:10 +00:00
runs:
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
- name: run01
tasks:
- name: task01
runtime:
type: pod
containers:
- image: busybox
Add initial agola config format and handling
2019-03-07 13:42:32 +00:00
depends:
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
- task02
- name: task02
runtime:
type: pod
containers:
- image: busybox
Add initial agola config format and handling
2019-03-07 13:42:32 +00:00
depends:
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
- task01
Add initial agola config format and handling
2019-03-07 13:42:32 +00:00
`,
err: &util.Errors{
Errs: []error{
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
errors.Errorf("circular dependency between task %q and tasks %q", "task01", "task02"),
errors.Errorf("circular dependency between task %q and tasks %q", "task02", "task01"),
Add initial agola config format and handling
2019-03-07 13:42:32 +00:00
},
},
},
config: fix check on task and parents with common deps
2019-08-12 21:11:19 +00:00
{
name: "test task parent same dep a -> b -> c, a -> c",
in: `
runs:
- name: run01
tasks:
- name: task01
runtime:
type: pod
containers:
- image: busybox
- name: task02
runtime:
type: pod
containers:
- image: busybox
depends:
- task01
- name: task03
runtime:
type: pod
containers:
- image: busybox
depends:
- task02
- task01
`,
err: errors.Errorf("task %q and its dependency %q have both a dependency on task %q", "task03", "task02", "task01"),
},
{
name: "test task parent same dep a -> b -> c -> d, a -> d",
in: `
runs:
- name: run01
tasks:
- name: task01
runtime:
type: pod
containers:
- image: busybox
- name: task02
runtime:
type: pod
containers:
- image: busybox
depends:
- task01
- name: task03
runtime:
type: pod
containers:
- image: busybox
depends:
- task02
- name: task04
runtime:
type: pod
containers:
- image: busybox
depends:
- task03
- task01
`,
err: errors.Errorf("task %q and its dependency %q have both a dependency on task %q", "task04", "task03", "task01"),
},
Add initial agola config format and handling
2019-03-07 13:42:32 +00:00
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
config: provide jsonnet context top level argument
2019-10-14 14:36:46 +00:00
if _, err := ParseConfig([]byte(tt.in), ConfigFormatJSON, &ConfigContext{}); err != nil {
Add initial agola config format and handling
2019-03-07 13:42:32 +00:00
if tt.err == nil {
t.Fatalf("got error: %v, expected no error", err)
}
*: use errors.Is/errors.As to handle wrapped error checking Enable golangci-lint errorlint linter to check proper use of errors.Is and error.As instead of direct comparison or error type casting.
2022-02-24 11:56:52 +00:00
var errs *util.Errors
if errors.As(err, &errs) {
Add initial agola config format and handling
2019-03-07 13:42:32 +00:00
if !errs.Equal(tt.err) {
t.Fatalf("got error: %v, want error: %v", err, tt.err)
}
} else {
if err.Error() != tt.err.Error() {
t.Fatalf("got error: %v, want error: %v", err, tt.err)
}
}
*: initial implementation of when conditions
2019-03-07 17:01:34 +00:00
} else {
if tt.err != nil {
t.Fatalf("got nil error, want error: %v", tt.err)
}
}
})
}
}
func TestParseOutput(t *testing.T) {
tests := []struct {
name string
in string
out *Config
}{
{
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
name: "test task all options",
*: initial implementation of when conditions
2019-03-07 17:01:34 +00:00
in: `
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
runs:
- name: run01
runservice: update docker registry auth
2019-04-22 12:38:25 +00:00
docker_registries_auth:
index.docker.io:
username: username
password:
from_variable: password
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
tasks:
- name: task01
runservice: update docker registry auth
2019-04-22 12:38:25 +00:00
docker_registries_auth:
index.docker.io:
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
username: username
password:
from_variable: password
runservice: update docker registry auth
2019-04-22 12:38:25 +00:00
runtime:
type: pod
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
containers:
- image: image01
auth:
username:
from_variable: username2
password: password2
environment:
ENV01: ENV01
ENVFROMVARIABLE01:
from_variable: variable01
config environment from variable
2019-03-18 14:02:32 +00:00
environment:
ENV01: ENV01
ENVFROMVARIABLE01:
from_variable: variable01
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
steps:
# normal step definition
- type: clone
- type: run
command: command01
- type: run
name: name different than command
command: command02
- type: run
command: command03
environment:
ENV01: ENV01
ENVFROMVARIABLE01:
from_variable: variable01
- type: save_cache
key: cache-{{ arch }}
contents:
- source_dir: /go/pkg/mod/cache
*: initial implementation of when conditions
2019-03-07 17:01:34 +00:00
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
# simpler (for yaml not for json) steps definition
- clone:
- run: command01
- run:
name: name different than command
command: command02
- run:
command: command03
environment:
ENV01: ENV01
ENVFROMVARIABLE01:
from_variable: variable01
- save_cache:
key: cache-{{ arch }}
contents:
- source_dir: /go/pkg/mod/cache
*: initial implementation of when conditions
2019-03-07 17:01:34 +00:00
when:
branch: master
tag:
- v1.x
- v2.x
ref:
include: master
exclude: [ /branch01/ , branch02 ]
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
depends:
- task: task02
conditions:
- on_success
- on_failure
- task03
- task04:
- on_success
- name: task02
runtime:
type: pod
containers:
- image: image01
- name: task03
runtime:
type: pod
containers:
- image: image01
*: implement ability to add tmpfs volumes to containers * Add a generic container volume option that currently only support tmpfs. In future it could be expanded to use of host volumes or other kind of volumes (if supported by the underlying executor) * Implement creation of tmpfs volumes in docker and k8s drivers.
2019-10-08 13:52:35 +00:00
volumes:
- path: /mnt/tmpfs
tmpfs:
size: 1Gi
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
- name: task04
runtime:
type: pod
containers:
- image: image01
*: implement ability to add tmpfs volumes to containers * Add a generic container volume option that currently only support tmpfs. In future it could be expanded to use of host volumes or other kind of volumes (if supported by the underlying executor) * Implement creation of tmpfs volumes in docker and k8s drivers.
2019-10-08 13:52:35 +00:00
volumes:
- path: /mnt/tmpfs
tmpfs: {}
run config: add tty option for run steps
2019-10-23 06:23:41 +00:00
- name: task05
runtime:
type: pod
containers:
- image: image01
steps:
- type: run
name: command with default tty
command: command01
- type: run
name: command with tty as true
command: command02
tty: true
- type: run
name: command with tty as false
command: command03
tty: false
*: initial implementation of when conditions
2019-03-07 17:01:34 +00:00
`,
out: &Config{
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
Runs: []*Run{
&Run{
Name: "run01",
runservice: update docker registry auth
2019-04-22 12:38:25 +00:00
DockerRegistriesAuth: map[string]*DockerRegistryAuth{
"index.docker.io": {
Type: DockerRegistryAuthTypeBasic,
Username: Value{Type: ValueTypeString, Value: "username"},
Password: Value{Type: ValueTypeFromVariable, Value: "password"},
},
},
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
Tasks: []*Task{
&Task{
Name: "task01",
runservice: update docker registry auth
2019-04-22 12:38:25 +00:00
DockerRegistriesAuth: map[string]*DockerRegistryAuth{
"index.docker.io": {
Type: DockerRegistryAuthTypeBasic,
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
Username: Value{Type: ValueTypeString, Value: "username"},
Password: Value{Type: ValueTypeFromVariable, Value: "password"},
},
runservice: update docker registry auth
2019-04-22 12:38:25 +00:00
},
Runtime: &Runtime{
Type: "pod",
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
Arch: "",
Containers: []*Container{
&Container{
Image: "image01",
Environment: map[string]Value{
"ENV01": Value{Type: ValueTypeString, Value: "ENV01"},
"ENVFROMVARIABLE01": Value{Type: ValueTypeFromVariable, Value: "variable01"},
},
User: "",
},
},
runservice: implement docker registry auth By now just support default username/password login In future also support additional container registries with their own credential helpers
2019-04-10 15:13:51 +00:00
},
config: generalize and simplify value string/from_variable parsing
2019-04-10 15:12:39 +00:00
Environment: map[string]Value{
"ENV01": Value{Type: ValueTypeString, Value: "ENV01"},
"ENVFROMVARIABLE01": Value{Type: ValueTypeFromVariable, Value: "variable01"},
config environment from variable
2019-03-18 14:02:32 +00:00
},
runservice: set default config task working_dir Set default task working_dir to ~/project
2019-04-16 13:20:45 +00:00
WorkingDir: defaultWorkingDir,
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
Shell: "",
User: "",
config: refactor config unmarshal * Don't use a complex UnmarshalJSON for Task but use specific UnmarshalJSON for every type that requires custom unmarshaling
2019-06-08 14:07:15 +00:00
Steps: Steps{
&CloneStep{BaseStep: BaseStep{Type: "clone"}},
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
&RunStep{
config: refactor config unmarshal * Don't use a complex UnmarshalJSON for Task but use specific UnmarshalJSON for every type that requires custom unmarshaling
2019-06-08 14:07:15 +00:00
BaseStep: BaseStep{
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
Type: "run",
Name: "command01",
},
Command: "command01",
run config: add tty option for run steps
2019-10-23 06:23:41 +00:00
Tty: util.BoolP(true),
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
},
&RunStep{
config: refactor config unmarshal * Don't use a complex UnmarshalJSON for Task but use specific UnmarshalJSON for every type that requires custom unmarshaling
2019-06-08 14:07:15 +00:00
BaseStep: BaseStep{
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
Type: "run",
Name: "name different than command",
},
Command: "command02",
run config: add tty option for run steps
2019-10-23 06:23:41 +00:00
Tty: util.BoolP(true),
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
},
&RunStep{
config: refactor config unmarshal * Don't use a complex UnmarshalJSON for Task but use specific UnmarshalJSON for every type that requires custom unmarshaling
2019-06-08 14:07:15 +00:00
BaseStep: BaseStep{
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
Type: "run",
Name: "command03",
},
Command: "command03",
Environment: map[string]Value{
"ENV01": Value{Type: ValueTypeString, Value: "ENV01"},
"ENVFROMVARIABLE01": Value{Type: ValueTypeFromVariable, Value: "variable01"},
},
run config: add tty option for run steps
2019-10-23 06:23:41 +00:00
Tty: util.BoolP(true),
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
},
&SaveCacheStep{
config: refactor config unmarshal * Don't use a complex UnmarshalJSON for Task but use specific UnmarshalJSON for every type that requires custom unmarshaling
2019-06-08 14:07:15 +00:00
BaseStep: BaseStep{Type: "save_cache"},
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
Key: "cache-{{ arch }}",
Contents: []*SaveContent{&SaveContent{SourceDir: "/go/pkg/mod/cache", Paths: []string{"**"}}},
},
config: refactor config unmarshal * Don't use a complex UnmarshalJSON for Task but use specific UnmarshalJSON for every type that requires custom unmarshaling
2019-06-08 14:07:15 +00:00
&CloneStep{BaseStep: BaseStep{Type: "clone"}},
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
&RunStep{
config: refactor config unmarshal * Don't use a complex UnmarshalJSON for Task but use specific UnmarshalJSON for every type that requires custom unmarshaling
2019-06-08 14:07:15 +00:00
BaseStep: BaseStep{
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
Type: "run",
Name: "command01",
},
Command: "command01",
run config: add tty option for run steps
2019-10-23 06:23:41 +00:00
Tty: util.BoolP(true),
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
},
&RunStep{
config: refactor config unmarshal * Don't use a complex UnmarshalJSON for Task but use specific UnmarshalJSON for every type that requires custom unmarshaling
2019-06-08 14:07:15 +00:00
BaseStep: BaseStep{
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
Type: "run",
Name: "name different than command",
},
Command: "command02",
run config: add tty option for run steps
2019-10-23 06:23:41 +00:00
Tty: util.BoolP(true),
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
},
&RunStep{
config: refactor config unmarshal * Don't use a complex UnmarshalJSON for Task but use specific UnmarshalJSON for every type that requires custom unmarshaling
2019-06-08 14:07:15 +00:00
BaseStep: BaseStep{
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
Type: "run",
Name: "command03",
},
Command: "command03",
Environment: map[string]Value{
"ENV01": Value{Type: ValueTypeString, Value: "ENV01"},
"ENVFROMVARIABLE01": Value{Type: ValueTypeFromVariable, Value: "variable01"},
},
run config: add tty option for run steps
2019-10-23 06:23:41 +00:00
Tty: util.BoolP(true),
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
},
&SaveCacheStep{
config: refactor config unmarshal * Don't use a complex UnmarshalJSON for Task but use specific UnmarshalJSON for every type that requires custom unmarshaling
2019-06-08 14:07:15 +00:00
BaseStep: BaseStep{Type: "save_cache"},
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
Key: "cache-{{ arch }}",
Contents: []*SaveContent{&SaveContent{SourceDir: "/go/pkg/mod/cache", Paths: []string{"**"}}},
},
config environment from variable
2019-03-18 14:02:32 +00:00
},
*: initial implementation of when conditions
2019-03-07 17:01:34 +00:00
IgnoreFailure: false,
Approval: false,
config: refactor config unmarshal * Don't use a complex UnmarshalJSON for Task but use specific UnmarshalJSON for every type that requires custom unmarshaling
2019-06-08 14:07:15 +00:00
When: &When{
*: initial implementation of when conditions
2019-03-07 17:01:34 +00:00
Branch: &types.WhenConditions{
Include: []types.WhenCondition{
{Type: types.WhenConditionTypeSimple, Match: "master"},
},
},
Tag: &types.WhenConditions{
Include: []types.WhenCondition{
{Type: types.WhenConditionTypeSimple, Match: "v1.x"},
{Type: types.WhenConditionTypeSimple, Match: "v2.x"},
},
},
Ref: &types.WhenConditions{
Include: []types.WhenCondition{
{Type: types.WhenConditionTypeSimple, Match: "master"},
},
Exclude: []types.WhenCondition{
{Type: types.WhenConditionTypeRegExp, Match: "branch01"},
{Type: types.WhenConditionTypeSimple, Match: "branch02"},
},
},
},
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
Depends: []*Depend{
&Depend{TaskName: "task02", Conditions: []DependCondition{DependConditionOnSuccess, DependConditionOnFailure}},
&Depend{TaskName: "task03", Conditions: nil},
&Depend{TaskName: "task04", Conditions: []DependCondition{DependConditionOnSuccess}},
},
},
&Task{
Name: "task02",
Runtime: &Runtime{
Type: "pod",
Arch: "",
Containers: []*Container{
&Container{
Image: "image01",
},
},
},
runservice: set default config task working_dir Set default task working_dir to ~/project
2019-04-16 13:20:45 +00:00
WorkingDir: defaultWorkingDir,
config: refactor config unmarshal * Don't use a complex UnmarshalJSON for Task but use specific UnmarshalJSON for every type that requires custom unmarshaling
2019-06-08 14:07:15 +00:00
Steps: nil,
Depends: nil,
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
},
&Task{
Name: "task03",
Runtime: &Runtime{
Type: "pod",
Arch: "",
Containers: []*Container{
&Container{
*: implement ability to add tmpfs volumes to containers * Add a generic container volume option that currently only support tmpfs. In future it could be expanded to use of host volumes or other kind of volumes (if supported by the underlying executor) * Implement creation of tmpfs volumes in docker and k8s drivers.
2019-10-08 13:52:35 +00:00
Image: "image01",
Volumes: []Volume{{Path: "/mnt/tmpfs", TmpFS: &VolumeTmpFS{Size: resource.NewQuantity(1024*1024*1024, resource.BinarySI)}}},
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
},
},
},
runservice: set default config task working_dir Set default task working_dir to ~/project
2019-04-16 13:20:45 +00:00
WorkingDir: defaultWorkingDir,
config: refactor config unmarshal * Don't use a complex UnmarshalJSON for Task but use specific UnmarshalJSON for every type that requires custom unmarshaling
2019-06-08 14:07:15 +00:00
Steps: nil,
Depends: nil,
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
},
&Task{
Name: "task04",
Runtime: &Runtime{
Type: "pod",
Arch: "",
Containers: []*Container{
&Container{
*: implement ability to add tmpfs volumes to containers * Add a generic container volume option that currently only support tmpfs. In future it could be expanded to use of host volumes or other kind of volumes (if supported by the underlying executor) * Implement creation of tmpfs volumes in docker and k8s drivers.
2019-10-08 13:52:35 +00:00
Image: "image01",
Volumes: []Volume{{Path: "/mnt/tmpfs", TmpFS: &VolumeTmpFS{}}},
runservice: rework config format The current config format was thought for future extensions for reusing runtimes and job definitions adding some parameters. After a lot of thoughts this looks like a complex approach: the final result will be a sort of templating without a lot of powers. Other approach like external templating should be an alternative but I really don't think templating yaml is the way to go. A much better approach will to just use jsonnet when we need to create matrix runs and a lot of other use cases. So just make the config a simple yaml/json. User can generate their config using any preferred tool and in future we'll leverage jsonnet automated parsing and provide a lot of jsonnet based examples for most use cases. Main changes: * Runs are now an array and not a map. The run name is in the Name field * Tasks are now an array and not a map. The task name is in the Name field * Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
2019-04-16 09:01:02 +00:00
},
},
},
runservice: set default config task working_dir Set default task working_dir to ~/project
2019-04-16 13:20:45 +00:00
WorkingDir: defaultWorkingDir,
config: refactor config unmarshal * Don't use a complex UnmarshalJSON for Task but use specific UnmarshalJSON for every type that requires custom unmarshaling
2019-06-08 14:07:15 +00:00
Steps: nil,
Depends: nil,
*: initial implementation of when conditions
2019-03-07 17:01:34 +00:00
},
run config: add tty option for run steps
2019-10-23 06:23:41 +00:00
&Task{
Name: "task05",
Runtime: &Runtime{
Type: "pod",
Arch: "",
Containers: []*Container{
&Container{
Image: "image01",
},
},
},
WorkingDir: defaultWorkingDir,
Steps: Steps{
&RunStep{
BaseStep: BaseStep{
Type: "run",
Name: "command with default tty",
},
Command: "command01",
Tty: util.BoolP(true),
},
&RunStep{
BaseStep: BaseStep{
Type: "run",
Name: "command with tty as true",
},
Command: "command02",
Tty: util.BoolP(true),
},
&RunStep{
BaseStep: BaseStep{
Type: "run",
Name: "command with tty as false",
},
Command: "command03",
Tty: util.BoolP(false),
},
},
Depends: nil,
},
*: initial implementation of when conditions
2019-03-07 17:01:34 +00:00
},
},
},
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
config: provide jsonnet context top level argument
2019-10-14 14:36:46 +00:00
out, err := ParseConfig([]byte(tt.in), ConfigFormatJSON, &ConfigContext{})
*: initial implementation of when conditions
2019-03-07 17:01:34 +00:00
if err != nil {
t.Fatalf("unexpected error: %v", err)
Add initial agola config format and handling
2019-03-07 13:42:32 +00:00
}
*: implement ability to add tmpfs volumes to containers * Add a generic container volume option that currently only support tmpfs. In future it could be expanded to use of host volumes or other kind of volumes (if supported by the underlying executor) * Implement creation of tmpfs volumes in docker and k8s drivers.
2019-10-08 13:52:35 +00:00
if diff := cmp.Diff(tt.out, out, cmp.Comparer(func(x, y *resource.Quantity) bool {
if x == nil && y == nil {
return true
}
if x != nil && y != nil {
return x.Cmp(*y) == 0
}
return false
})); diff != "" {
*: initial implementation of when conditions
2019-03-07 17:01:34 +00:00
t.Error(diff)
Add initial agola config format and handling
2019-03-07 13:42:32 +00:00
}
})
}
}