71 Commits

Author SHA1 Message Date
Ingo Oppermann
8829b8fff0
Respect domain in cluster DB, allow metadata in process config for cluster 2023-06-05 21:19:11 +02:00
Ingo Oppermann
e9ba394c2e
Fix process stored in DB 2023-06-02 14:16:12 +02:00
Ingo Oppermann
f59a63320c
Allow to pass metadata with process config, non-cluster only 2023-06-01 21:24:33 +02:00
Ingo Oppermann
d652fd213b
Check for identical configs on process update 2023-06-01 16:43:17 +02:00
Ingo Oppermann
a79cfa0c77
Merge branch 'clusteriam' into vod 2023-06-01 15:16:54 +02:00
Ingo Oppermann
1f1a124bb4
Check and lof resource specification 2023-05-24 14:53:33 +02:00
Ingo Oppermann
ccac2ffd5d
Various updates
- rebrand group to domain
- move IAM to the API (rest and graph) for enforcing "process:" rules
- add abstraction layer for restream store in order to decouple internal format from format on disk
- move playout handler into restreamHandler
- remove user from restream interface
- add TaskID type that includes the process id and its domain
2023-05-23 15:47:06 +02:00
Ingo Oppermann
f03e2ca5c5
Merge branch 'dev' into iam 2023-05-16 13:19:45 +02:00
Ingo Oppermann
8c3b570ecc
Fix log transfer 2023-05-15 21:31:58 +02:00
Ingo Oppermann
ead6e64442
Merge branch 'dev' into iam 2023-05-15 21:17:23 +02:00
Ingo Oppermann
4c3c8d3bd4
Merge branch 'dev' into vod 2023-05-05 12:09:54 +02:00
Ingo Oppermann
6759ebc950
Fix keeping order on process update 2023-05-03 10:50:33 +02:00
Ingo Oppermann
4d5a1f9617
Fix too early history transfer to new parser during process update 2023-05-02 22:39:34 +02:00
Ingo Oppermann
ef138fb90f
Limit CPU and memory independently, release CPU throttling incremently 2023-05-01 16:29:18 +02:00
Ingo Oppermann
2376e43f96
Fix process restart to restart immediately regardless if reconnect is defined 2023-04-29 08:27:40 +02:00
Ingo Oppermann
f0a7564c55
Transfer report history when a process is reloaded 2023-04-29 08:12:45 +02:00
Ingo Oppermann
74f98e8c7e
Define system CPU resources in percent 0-100, acquiring resources check with available system resources 2023-04-28 10:27:33 +02:00
Ingo Oppermann
7527f7833b
Add resource manager 2023-04-27 17:02:20 +02:00
Ingo Oppermann
b723f4e5fd
Propagate number of logical CPUs in resource usage 2023-04-27 10:58:41 +02:00
Ingo Oppermann
5e2060f785
WIP: add resource manager 2023-04-26 22:05:46 +02:00
Ingo Oppermann
d59158de03
Allow hard and soft limiting a process
A hard limit will kill the process as soon as either CPU or memory
consumption are above a defined limit for a certain amount of time.

A soft limit will throttle the CPU usage if above a defined limit and
kill the process if memory consumption is above a defined limit. The
soft limit can be enabled/disabled on demand.

The default is hard limit.
2023-04-26 16:01:50 +02:00
Ingo Oppermann
a2dab2682f
Fix not propagating process limits 2023-04-26 09:49:28 +02:00
Ingo Oppermann
9b17ab2b29
Add global limits in config, fix not using process limits 2023-04-25 21:49:55 +02:00
Ingo Oppermann
b70382c40b
Expose resource usage in process state 2023-04-25 16:20:59 +02:00
Ingo Oppermann
3a1825cf61
Expose resource usage in report history 2023-04-25 15:55:32 +02:00
Ingo Oppermann
c1bbb4065c
Merge branch 'dev' into vod 2023-04-24 12:16:52 +02:00
Ingo Oppermann
317d6eb4d9
Add updated_at field in process infos 2023-04-24 12:05:01 +02:00
Ingo Oppermann
6ddd58a124
Preserve process log history when updating a process 2023-04-24 11:59:09 +02:00
Ingo Oppermann
76abf5474f
Fix compilation error 2023-04-13 15:43:59 +02:00
Ingo Oppermann
b96672e308
Forward keyframe and extradata_bytes counter to API 2023-04-11 08:59:24 +02:00
Ingo Oppermann
84817f137a
Merge branch 'dev' into iam 2023-04-03 15:44:16 +02:00
Ingo Oppermann
0a3117bbd0
Add log_patterns to process config
log_patterns allow to filter the FFmpeg log messages based on regular
expressions. Each entry of log_patterns is interpreted as regular
expression and matched against every non-progress log line emitted from
FFmpeg. All matching lines are returned in the matches array of the
report.
2023-03-27 15:50:25 +02:00
Ingo Oppermann
99c571d623
Add event filter support 2023-03-23 14:50:59 +01:00
Ingo Oppermann
b63b23527b
Add scheduler and timeout to processes
The scheduler allows to define when a process should run. It can be either
a timestamp in RFC3339 format or a crontab expression. If a scheduler is
given, reconnect and the reconnect delay will only apply to processes that
exited as failed.

The timeout allows to define when a process should be gracefully stopped.
It is measured from the actual start of that process including all reconnects
due to failures. If the process finished regularly, the timeout will be
reset.
2023-03-21 14:51:43 +01:00
Ingo Oppermann
1d298038dd
Add date of when a process exited in report history 2023-03-14 15:10:06 +01:00
Ingo Oppermann
6e93c1d5a1
Get rid of $localhost pseudo user 2023-03-09 21:10:04 +01:00
Ingo Oppermann
8755117e92
Add /api/v3/iam/user endpoints 2023-03-07 16:31:58 +01:00
Ingo Oppermann
175cfc2324
Fix using prefix syntax for filesystem operations 2023-03-03 16:18:46 +01:00
Ingo Oppermann
eb3f396793
Remove debug printf, add tests 2023-03-02 21:04:37 +01:00
Ingo Oppermann
0dedcddece
Add process report API 2023-03-02 11:15:57 +01:00
Ingo Oppermann
cceb39192a
Fix receiving proper current command args, simplifying retrieval of last log line 2023-03-01 20:13:49 +01:00
Ingo Oppermann
3151670829
Add descriptions to functions 2023-03-01 15:48:52 +01:00
Ingo Oppermann
86b3c053f1
Add exit state and last progress data to process report history 2023-03-01 15:28:28 +01:00
Ingo Oppermann
3cad139952
Create directories for output files to be written on disk 2023-03-01 11:08:20 +01:00
Ingo Oppermann
be718eac0a
Add support for date placeholder in process config
Because it doesn't make sense to replace the date placeholder at
process creation, it has to be replaced at every start of the process.

On process creation only the static placeholders (such as process ID)
are replaced. Dynamic placeholders (so far only "date") are not
replaced. On process start, a callback has been introduced that gives
the chance to change the command line. This is the point where
the restreamer replaces the date placeholders.
2023-02-28 17:46:08 +01:00
Ingo Oppermann
acc4d7ec5a
WIP: verify policies for processes, rtmp, and srt. rewriter for local urls 2023-02-15 21:43:37 +01:00
Ingo Oppermann
a9459bda7c
WIP enforcing policies on process level 2023-02-14 15:23:50 +01:00
Ingo Oppermann
2a3288ffd0
Use abstract filesystem for stores 2023-02-01 16:09:20 +01:00
Ingo Oppermann
f519acfd71
Add S3 storage support 2023-01-31 14:45:58 +01:00
Ingo Oppermann
b2cd8f7133
Allow probe with individual timeout 2023-01-23 17:09:55 +01:00