run changegroup names are based on the run path but it will contain slashes and
could be very long. So calculate the sha256 sum of the path and use it as the
changegroup name.
* Rename to datamanager since it handles a complete "database" backed by an
objectstorage and etcd
* Don't write every single entry as a single file but group them in a single
file. In future improve this to split the data in multiple files of a max size.
`lts` was choosen to reflect a "long term storage" but currently it's just an
object storage implementation. So use this term and "ost" as its abbreviation
(to not clash with "os").
An executor can handle multiple archs (an executor that talks with a k8s cluster
with multi arch nodes). Don't use a label for archs but a custom executor
field.
* Add the concept of executor groups and siblings executors
* Add the concept of dynamic executor: an executor in an executor group that
doesn't need to be manually deleted from the scheduler since the other sibling
executors will take care of cleaning up its pods.
* Remove external labels visibility from pod.
* Add functions to return the sibling executors and the executor group
* Delete pods of disappeared sibling executors
take and change a copy of the current run so we'll change newRun and use curRun
status for logic decision. In this way result are reproducible or they will be
affected by the random run.Tasks map iteration order.
Handle config files with name `config.jsonnet`, `config.json` and
`config.yml` and take the first from the repository in this order
For a jsonnet file execute it and use the generated output as the config
The current config format was thought for future extensions for reusing runtimes
and job definitions adding some parameters.
After a lot of thoughts this looks like a complex approach: the final result
will be a sort of templating without a lot of powers.
Other approach like external templating should be an alternative but I really
don't think templating yaml is the way to go.
A much better approach will to just use jsonnet when we need to create matrix
runs and a lot of other use cases.
So just make the config a simple yaml/json. User can generate their config using
any preferred tool and in future we'll leverage jsonnet automated parsing and
provide a lot of jsonnet based examples for most use cases.
Main changes:
* Runs are now an array and not a map. The run name is in the Name field
* Tasks are now an array and not a map. The task name is in the Name field
* Use https://github.com/ghodss/yaml so we'll use json struct tags and unmarshall functions
Simone Gotti