Commit Graph

10 Commits

Author SHA1 Message Date
Simone Gotti 3139ef38d9 runservice readdb: get run from ost db if it's not in run db 2019-05-06 14:55:10 +02:00
Simone Gotti da6aefa7e2 runservice readdb: also resync changegroups 2019-04-29 10:16:19 +02:00
Simone Gotti 2c3e6bf9e4 wal: update and rename to datamanager
* Rename to datamanager since it handles a complete "database" backed by an
objectstorage and etcd

* Don't write every single entry as a single file but group them in a single
file. In future improve this to split the data in multiple files of a max size.
2019-04-26 16:00:03 +02:00
Simone Gotti 41e333d7ec *: rename "lts" to "ost"
`lts` was choosen to reflect a "long term storage" but currently it's just an
object storage implementation. So use this term and "ost" as its abbreviation
(to not clash with "os").
2019-04-27 15:16:48 +02:00
Simone Gotti 33c328b3f5 runservice: move all scheduler etcd data to own dir 2019-04-27 08:59:47 +02:00
Simone Gotti fc891409ca update wal and readdb 2019-04-01 12:54:43 +02:00
Simone Gotti 75b5b65da3 runservice: check that readdb is initialized 2019-03-29 12:20:54 +01:00
Simone Gotti e46766829c runservice: rework store and readdb logic
* Remove all the small index files on the lts
* Keep on s3 only a full index of all runs containing the runid, grouppath and phase
  million of runs can take only some hundred of megabytes
* Periodically create a new dump of the index
2019-03-29 12:15:48 +01:00
Simone Gotti 48ab496beb *: add api to query last run per group 2019-03-29 12:00:18 +01:00
Simone Gotti 36fc79dfc6 runservice: initial commit 2019-02-21 15:54:50 +01:00