247 Commits

Author SHA1 Message Date
Ingo Oppermann
b96672e308
Forward keyframe and extradata_bytes counter to API 2023-04-11 08:59:24 +02:00
Ingo Oppermann
507b4f4b9a
Merge branch 'dev' into vod 2023-04-11 08:29:39 +02:00
Ingo Oppermann
2c5f32fb26
Remove bytes.Buffer pool for memfs in order to be less a memory hog 2023-04-06 21:43:18 +02:00
Ingo Oppermann
d2e5720fa2
Fix limiting part sizes for S3 upload 2023-04-06 12:06:50 +02:00
Ingo Oppermann
98c561554d
Fix S3 storage parsing from environment variable 2023-04-05 14:43:45 +02:00
Ingo Oppermann
51cbb8e74e
Fix assembling return value for filtered report history 2023-04-05 13:58:41 +02:00
Ingo Oppermann
7372b3210d
Fix test 2023-04-05 12:21:19 +02:00
Ingo Oppermann
ec4607aec8
Fix process report selection 2023-04-05 12:01:21 +02:00
Ingo Oppermann
508d82d579
Fix high memory consumption when copying files to S3 2023-04-05 10:30:17 +02:00
Ingo Oppermann
7e9e6fce8d
Add number of keyframes and extradata size to process progress data 2023-04-04 20:44:57 +02:00
Ingo Oppermann
baf1c3391a
Deprecate ENV names that do not correspond to JSON name 2023-04-03 21:21:02 +02:00
Ingo Oppermann
0dd4a8fb60
Add TestFilesystemsPurgeCache 2023-04-03 14:02:41 +02:00
Ingo Oppermann
31a60598ee
Fix test 2023-04-03 10:57:43 +02:00
Ingo Oppermann
30be82c9fb
Merge branch 'dev' into vod 2023-04-03 10:29:01 +02:00
Ingo Oppermann
6eefa5ca2b
Fix purging default file from HTTP cache 2023-04-03 10:27:04 +02:00
Ingo Oppermann
75332ade0a
Merge branch 'dev' into vod 2023-04-03 10:25:06 +02:00
Ingo Oppermann
029bf208e7
Add bandwidth limit to filesystem operations, rename fields 2023-03-30 09:51:29 +02:00
Ingo Oppermann
4e7b84dd96
Fix tests 2023-03-28 14:31:43 +02:00
Ingo Oppermann
0a3117bbd0
Add log_patterns to process config
log_patterns allow to filter the FFmpeg log messages based on regular
expressions. Each entry of log_patterns is interpreted as regular
expression and matched against every non-progress log line emitted from
FFmpeg. All matching lines are returned in the matches array of the
report.
2023-03-27 15:50:25 +02:00
Ingo Oppermann
814975dfee
Fix duplicate ID in swagger annotations 2023-03-27 14:35:50 +02:00
Ingo Oppermann
83f94e1ef5
Fix concurrent map read and map write 2023-03-24 09:13:12 +01:00
Ingo Oppermann
d8c5b27d89
Fix write to closed channel 2023-03-24 08:59:32 +01:00
Ingo Oppermann
7188a7dcdd
Potentially fix panic on event subscriber disconnect 2023-03-23 21:07:21 +01:00
Ingo Oppermann
094e666926
Fix swagger documentation for /api/v3/events 2023-03-23 19:46:24 +01:00
Ingo Oppermann
99c571d623
Add event filter support 2023-03-23 14:50:59 +01:00
Ingo Oppermann
48678fb4c6
Fix purging default file from HTTP cache 2023-03-23 11:17:13 +01:00
Ingo Oppermann
b84fdddd81
Add /api/v3/events endpoint 2023-03-23 11:11:47 +01:00
Ingo Oppermann
2330270af2
Merge branch 'dev' into vod 2023-03-22 15:56:27 +01:00
Ingo Oppermann
52df872198
Update changelog 2023-03-22 15:52:06 +01:00
Ingo Oppermann
de207b02a1
Fix URL validation if the path contains FFmpeg specific placeholders 2023-03-22 15:49:52 +01:00
Ingo Oppermann
b21aba5f9d
Remove /process/:id/report/:at endpoint, extend /process/:id/report endpoint 2023-03-22 12:31:41 +01:00
Ingo Oppermann
d950c45eb0
Add comment 2023-03-21 18:41:07 +01:00
Ingo Oppermann
c92f2fd1df
Fix JSON name of process timeout 2023-03-21 14:54:33 +01:00
Ingo Oppermann
b63b23527b
Add scheduler and timeout to processes
The scheduler allows to define when a process should run. It can be either
a timestamp in RFC3339 format or a crontab expression. If a scheduler is
given, reconnect and the reconnect delay will only apply to processes that
exited as failed.

The timeout allows to define when a process should be gracefully stopped.
It is measured from the actual start of that process including all reconnects
due to failures. If the process finished regularly, the timeout will be
reset.
2023-03-21 14:51:43 +01:00
Ingo Oppermann
674c444d17
Merge branch 'dev' into vod 2023-03-17 18:41:16 +01:00
Ingo Oppermann
3149572a64
Fix freeing up S3 mounts 2023-03-17 18:40:20 +01:00
Ingo Oppermann
a692f88b69
Add DELETE /api/v3/fs/:storage endpoint
Works similar to the GET endpoint. It has the same query parameter (without
"sort" and "order"). A glob pattern has to be provided. To select all
files, use the glob pattern "/**".
2023-03-17 17:05:56 +01:00
Ingo Oppermann
7e7aadc6cb
Allow to bulk delete files based on ListOptions 2023-03-17 15:56:15 +01:00
Ingo Oppermann
ecfbbe3857
Allow to list files by ranges of size and/or lastmod
The listing options are implemented by the query parameters size_min,
size_max, lastmod_start, and lastmod_end, or by the new ListOptions
type. size_min and size_max expect a number of bytes, lastmod_start
and lastmod_end expect a unix timestamp. All values are inclusive.
2023-03-17 15:15:20 +01:00
Ingo Oppermann
2e8f8fb1a7
Fix swagger tags 2023-03-17 13:58:40 +01:00
Ingo Oppermann
ffaa3d9324
Merge branch 'dev' into vod 2023-03-17 13:56:07 +01:00
Ingo Oppermann
562b7aed92
Use better naming for storage endpoint documentation 2023-03-17 13:55:19 +01:00
Ingo Oppermann
4ce8a0eaa3
Add config value for how many minimal process report should be kept in the history
A minimal history is a history entry without log and prelude.

The corresponding config entry is ffmpeg.log.max_minimal_history. This value is
added on top of the ffmpeg.log.max_history value. I.e. the latest max_history
entries contain the log and prelude, and the remaining entries don't have the
log and prelude. In total there are max_minimal_history+max_history history
entries.

If you want no history, set both values to 0.
If you want only full history, set max_minimal_history to 0.
If you want only minimal history, set max_history to 0.
2023-03-16 12:25:06 +01:00
Ingo Oppermann
5b2b2243bb
Fix report history tests 2023-03-14 15:24:05 +01:00
Ingo Oppermann
206c81d7e8
Emit stale progress only if all outputs of a process are stale 2023-03-14 15:13:57 +01:00
Ingo Oppermann
1d298038dd
Add date of when a process exited in report history 2023-03-14 15:10:06 +01:00
Ingo Oppermann
b76e6a9b2c
Add Range support for HTTP GET requests on a mounted filesystem 2023-03-08 11:50:59 +01:00
Ingo Oppermann
8e29cbdd12
Update changelog 2023-03-06 10:24:19 +01:00
Ingo Oppermann
e434716fb2
Fix tests 2023-03-03 18:53:09 +01:00
Ingo Oppermann
81473e6adb
Fix placeholder parameter parsing 2023-03-03 17:26:59 +01:00