Ingo Oppermann
129058e633
Improve FS.List and FS.RemoveList functions, improve CleanupFS
...
The FS.List and FS.RemoveList are up to 42x faster by precompiling the
globbing pattern.
The CleanupFS from restreamer/fs is up to 32x faster in the benchmarks
and cleanup is now only every 5 seconds instead of every second.
2023-06-12 11:36:07 +02:00
Ingo Oppermann
7e7d1caca7
Add EmergencyLeaderTimeout parameter, ignore throttling nodes, implement NodeRecoverTimeout, introduce processNodeMap in cluster DB
2023-06-07 22:08:07 +02:00
Ingo Oppermann
08bdf752f1
Fix tests
2023-06-06 22:07:21 +02:00
Ingo Oppermann
fc03bf73a2
Make resource manager a main module and expose more details
2023-06-06 21:28:08 +02:00
Ingo Oppermann
8829b8fff0
Respect domain in cluster DB, allow metadata in process config for cluster
2023-06-05 21:19:11 +02:00
Ingo Oppermann
e9ba394c2e
Fix process stored in DB
2023-06-02 14:16:12 +02:00
Ingo Oppermann
f59a63320c
Allow to pass metadata with process config, non-cluster only
2023-06-01 21:24:33 +02:00
Ingo Oppermann
d652fd213b
Check for identical configs on process update
2023-06-01 16:43:17 +02:00
Ingo Oppermann
a79cfa0c77
Merge branch 'clusteriam' into vod
2023-06-01 15:16:54 +02:00
Ingo Oppermann
89cc42c4b9
Add cluster IAM, update dependencies
2023-05-30 21:24:12 +02:00
Ingo Oppermann
e9034aa171
Create identity and access packages for IAM
2023-05-25 16:16:29 +02:00
Ingo Oppermann
10df8d51b1
Simplify IAM user config, only create users and policies if not previously run
2023-05-24 22:28:24 +02:00
Ingo Oppermann
fa5ae1d17b
Fix tests
2023-05-24 16:39:10 +02:00
Ingo Oppermann
1f1a124bb4
Check and lof resource specification
2023-05-24 14:53:33 +02:00
Ingo Oppermann
3c89cbb831
Create identities for basic auth access to mount points
2023-05-24 14:29:14 +02:00
Ingo Oppermann
ccac2ffd5d
Various updates
...
- rebrand group to domain
- move IAM to the API (rest and graph) for enforcing "process:" rules
- add abstraction layer for restream store in order to decouple internal format from format on disk
- move playout handler into restreamHandler
- remove user from restream interface
- add TaskID type that includes the process id and its domain
2023-05-23 15:47:06 +02:00
Ingo Oppermann
f03e2ca5c5
Merge branch 'dev' into iam
2023-05-16 13:19:45 +02:00
Ingo Oppermann
8c3b570ecc
Fix log transfer
2023-05-15 21:31:58 +02:00
Ingo Oppermann
ead6e64442
Merge branch 'dev' into iam
2023-05-15 21:17:23 +02:00
Ingo Oppermann
4c3c8d3bd4
Merge branch 'dev' into vod
2023-05-05 12:09:54 +02:00
Ingo Oppermann
ca261a56ee
Add looping_runtime to avstream status
2023-05-05 12:03:48 +02:00
Ingo Oppermann
6759ebc950
Fix keeping order on process update
2023-05-03 10:50:33 +02:00
Ingo Oppermann
4d5a1f9617
Fix too early history transfer to new parser during process update
2023-05-02 22:39:34 +02:00
Ingo Oppermann
3ee362a5fe
Fix race condition
2023-05-02 10:59:57 +02:00
Ingo Oppermann
7cca02f2b5
Add tests
2023-05-01 21:42:01 +02:00
Ingo Oppermann
ef138fb90f
Limit CPU and memory independently, release CPU throttling incremently
2023-05-01 16:29:18 +02:00
Ingo Oppermann
2376e43f96
Fix process restart to restart immediately regardless if reconnect is defined
2023-04-29 08:27:40 +02:00
Ingo Oppermann
f0a7564c55
Transfer report history when a process is reloaded
2023-04-29 08:12:45 +02:00
Ingo Oppermann
024b5710b5
Allow whitespaces in front of key/value pairs in process placeholders
2023-04-29 08:06:01 +02:00
Ingo Oppermann
f6d1cbbec2
WIP: release CPU throttling stepwise
2023-04-28 16:46:59 +02:00
Ingo Oppermann
74f98e8c7e
Define system CPU resources in percent 0-100, acquiring resources check with available system resources
2023-04-28 10:27:33 +02:00
Ingo Oppermann
7527f7833b
Add resource manager
2023-04-27 17:02:20 +02:00
Ingo Oppermann
b723f4e5fd
Propagate number of logical CPUs in resource usage
2023-04-27 10:58:41 +02:00
Ingo Oppermann
5e2060f785
WIP: add resource manager
2023-04-26 22:05:46 +02:00
Ingo Oppermann
1e35d29371
Fix tests
2023-04-26 16:12:41 +02:00
Ingo Oppermann
d59158de03
Allow hard and soft limiting a process
...
A hard limit will kill the process as soon as either CPU or memory
consumption are above a defined limit for a certain amount of time.
A soft limit will throttle the CPU usage if above a defined limit and
kill the process if memory consumption is above a defined limit. The
soft limit can be enabled/disabled on demand.
The default is hard limit.
2023-04-26 16:01:50 +02:00
Ingo Oppermann
0c497f4ea5
Merge branch 'dev' into vod
2023-04-26 10:08:18 +02:00
Ingo Oppermann
e45f80ed42
Fix tests
2023-04-26 09:50:09 +02:00
Ingo Oppermann
a2dab2682f
Fix not propagating process limits
2023-04-26 09:49:28 +02:00
Ingo Oppermann
2f9a997a8d
Add test for process limits
2023-04-25 21:59:22 +02:00
Ingo Oppermann
9b17ab2b29
Add global limits in config, fix not using process limits
2023-04-25 21:49:55 +02:00
Ingo Oppermann
b70382c40b
Expose resource usage in process state
2023-04-25 16:20:59 +02:00
Ingo Oppermann
3a1825cf61
Expose resource usage in report history
2023-04-25 15:55:32 +02:00
Ingo Oppermann
3e7e5d5c9c
Merge branch 'dev' into vod
2023-04-24 16:14:32 +02:00
Ingo Oppermann
061542645c
Fix test
2023-04-24 12:28:42 +02:00
Ingo Oppermann
c1bbb4065c
Merge branch 'dev' into vod
2023-04-24 12:16:52 +02:00
Ingo Oppermann
317d6eb4d9
Add updated_at field in process infos
2023-04-24 12:05:01 +02:00
Ingo Oppermann
6ddd58a124
Preserve process log history when updating a process
2023-04-24 11:59:09 +02:00
Ingo Oppermann
2b1459c5e8
Fix test
2023-04-17 15:24:39 +02:00
Ingo Oppermann
78686c81f4
Fix keeping matches in the report history
2023-04-17 15:03:25 +02:00
