65 Commits

Author SHA1 Message Date
Ingo Oppermann
de9a30a108
Add internal mock modules 2024-10-29 14:55:55 +01:00
Ingo Oppermann
2ee7fa7e41
Make resources the only direct user of psutil 2024-10-29 12:25:39 +01:00
Ingo Oppermann
412fbedea3
Make psutil a submodule of resources, remove default psutil 2024-10-28 16:13:13 +01:00
Ingo Oppermann
2dbe5b5685
Add GPU support 2024-10-24 15:08:26 +02:00
Ingo Oppermann
df30a6b8e3
Replace timer-based SMA with a timer-less implementation 2024-10-23 11:08:13 +02:00
Ingo Oppermann
7831992936
Add parsing for track specific avstream progress data 2024-09-05 13:33:01 +02:00
Ingo Oppermann
b9baa17b0c
Fix writing string to prelude tail 2024-07-26 11:47:40 +02:00
Ingo Oppermann
70a49f8bdb
Process []byte instread of string in parser 2024-07-26 11:31:47 +02:00
Ingo Oppermann
7e90bb87ce
Allow to import report history for a process 2024-07-10 16:46:49 +02:00
Ingo Oppermann
480dbb7f53
Refactor cluster node code 2024-07-09 12:26:02 +02:00
Ingo Oppermann
03da97217b
Use faster JSON and gzip packages 2024-04-24 15:42:11 +02:00
Ingo Oppermann
b851249b98
Adjust process state if parser doesn't indicate that the process is producing output 2024-04-15 16:43:34 +02:00
Ingo Oppermann
18bf51d334
Rename Initialized to Started 2024-04-15 14:46:05 +02:00
Ingo Oppermann
6ebe008010
Fix parser tests 2024-04-12 15:55:25 +02:00
Ingo Oppermann
a6375da911
Add initialized field in progress data 2024-04-12 15:18:47 +02:00
Ingo Oppermann
74489775ca
Propagate avstream debug object as interface 2023-12-06 20:23:22 +01:00
Ingo Oppermann
b43fd64dce
Fix test 2023-12-04 16:54:38 +01:00
Ingo Oppermann
5d39620f6f
Parse out avstream debug infos 2023-12-01 12:08:56 +01:00
Ingo Oppermann
a416beea08
Add stream mapping to process state, adjust addresses and indexes for HLS outputs 2023-09-26 14:52:54 +02:00
Ingo Oppermann
ab7c9e448b
Add parser for ffmpeg and HLS stream mapping 2023-09-26 11:40:48 +02:00
Ingo Oppermann
48cc454314
Add expose 'mode' from avstream progress 2023-09-20 20:53:50 +02:00
Ingo Oppermann
ccecbe3c8e
Improve FFmpeg skills testing 2023-07-06 21:41:10 +02:00
Ingo Oppermann
6c2e8b0ec3
Make ffmpeg skills compareable 2023-07-06 10:27:56 +02:00
Ingo Oppermann
50e943a075
WIP: add session token, missing: writing sessions to log 2023-06-14 12:30:50 +02:00
Ingo Oppermann
4c3c8d3bd4
Merge branch 'dev' into vod 2023-05-05 12:09:54 +02:00
Ingo Oppermann
ca261a56ee
Add looping_runtime to avstream status 2023-05-05 12:03:48 +02:00
Ingo Oppermann
67cde54b90
Add test for progress data in history 2023-05-02 20:52:40 +02:00
Ingo Oppermann
da557271cb
Fix missing mutex for log history 2023-05-02 19:13:16 +02:00
Ingo Oppermann
7527f7833b
Add resource manager 2023-04-27 17:02:20 +02:00
Ingo Oppermann
b723f4e5fd
Propagate number of logical CPUs in resource usage 2023-04-27 10:58:41 +02:00
Ingo Oppermann
d59158de03
Allow hard and soft limiting a process
A hard limit will kill the process as soon as either CPU or memory
consumption are above a defined limit for a certain amount of time.

A soft limit will throttle the CPU usage if above a defined limit and
kill the process if memory consumption is above a defined limit. The
soft limit can be enabled/disabled on demand.

The default is hard limit.
2023-04-26 16:01:50 +02:00
Ingo Oppermann
a2dab2682f
Fix not propagating process limits 2023-04-26 09:49:28 +02:00
Ingo Oppermann
9b17ab2b29
Add global limits in config, fix not using process limits 2023-04-25 21:49:55 +02:00
Ingo Oppermann
74f582e4b4
Remove debug output 2023-04-25 16:25:21 +02:00
Ingo Oppermann
3a1825cf61
Expose resource usage in report history 2023-04-25 15:55:32 +02:00
Ingo Oppermann
c1bbb4065c
Merge branch 'dev' into vod 2023-04-24 12:16:52 +02:00
Ingo Oppermann
6ddd58a124
Preserve process log history when updating a process 2023-04-24 11:59:09 +02:00
Ingo Oppermann
78686c81f4
Fix keeping matches in the report history 2023-04-17 15:03:25 +02:00
Ingo Oppermann
76abf5474f
Fix compilation error 2023-04-13 15:43:59 +02:00
Ingo Oppermann
5d9db55032
Merge branch 'dev' into vod 2023-04-13 15:23:09 +02:00
Ingo Oppermann
d807becc8a
Add support for input framerate data from jsonstats patch 2023-04-13 15:22:33 +02:00
Ingo Oppermann
b96672e308
Forward keyframe and extradata_bytes counter to API 2023-04-11 08:59:24 +02:00
Ingo Oppermann
507b4f4b9a
Merge branch 'dev' into vod 2023-04-11 08:29:39 +02:00
Ingo Oppermann
7e9e6fce8d
Add number of keyframes and extradata size to process progress data 2023-04-04 20:44:57 +02:00
Ingo Oppermann
0a3117bbd0
Add log_patterns to process config
log_patterns allow to filter the FFmpeg log messages based on regular
expressions. Each entry of log_patterns is interpreted as regular
expression and matched against every non-progress log line emitted from
FFmpeg. All matching lines are returned in the matches array of the
report.
2023-03-27 15:50:25 +02:00
Ingo Oppermann
99c571d623
Add event filter support 2023-03-23 14:50:59 +01:00
Ingo Oppermann
b21aba5f9d
Remove /process/:id/report/:at endpoint, extend /process/:id/report endpoint 2023-03-22 12:31:41 +01:00
Ingo Oppermann
b63b23527b
Add scheduler and timeout to processes
The scheduler allows to define when a process should run. It can be either
a timestamp in RFC3339 format or a crontab expression. If a scheduler is
given, reconnect and the reconnect delay will only apply to processes that
exited as failed.

The timeout allows to define when a process should be gracefully stopped.
It is measured from the actual start of that process including all reconnects
due to failures. If the process finished regularly, the timeout will be
reset.
2023-03-21 14:51:43 +01:00
Ingo Oppermann
4ce8a0eaa3
Add config value for how many minimal process report should be kept in the history
A minimal history is a history entry without log and prelude.

The corresponding config entry is ffmpeg.log.max_minimal_history. This value is
added on top of the ffmpeg.log.max_history value. I.e. the latest max_history
entries contain the log and prelude, and the remaining entries don't have the
log and prelude. In total there are max_minimal_history+max_history history
entries.

If you want no history, set both values to 0.
If you want only full history, set max_minimal_history to 0.
If you want only minimal history, set max_history to 0.
2023-03-16 12:25:06 +01:00
Ingo Oppermann
206c81d7e8
Emit stale progress only if all outputs of a process are stale 2023-03-14 15:13:57 +01:00