116 Commits

Author SHA1 Message Date
Ingo Oppermann
f37896a1e3
Implement certmagic.Storage on cluster 2023-06-23 21:00:45 +02:00
Ingo Oppermann
d3eed2a417
Allow to log each finished session to filesystem
By providing CORE_SESSIONS_SESSION_LOG_PATH_PATTERN (e.g. "/log/%Y-%m-%d.log") all
finished sessions will be logged to a file according to the provided strftime-pattern. The
actual value is calculated from when the session closed. CORE_SESSIONS_PERSIST must
be set. Default: not set.

Set CORE_SESSIONS_SESSION_LOG_BUFFER_SEC to the number of seconds the log should be
buffered in memory before persisted to disk. Default 15 seconds.
2023-06-20 15:19:25 +02:00
Ingo Oppermann
3cc8626ae1
Fix RTMP and SRT URL generation in placeholder 2023-06-15 14:19:02 +02:00
Ingo Oppermann
debcce192d
Add the process domain to the bas path in placeholders and cleanup rules 2023-06-14 22:30:00 +02:00
Ingo Oppermann
a728cc7839
Fix clusterNode/proxyNode 2023-06-14 22:05:59 +02:00
Ingo Oppermann
ccf4bee56d
WIP: cluster node revamp, add version and config comparison 2023-06-13 10:25:08 +02:00
Ingo Oppermann
7e7d1caca7
Add EmergencyLeaderTimeout parameter, ignore throttling nodes, implement NodeRecoverTimeout, introduce processNodeMap in cluster DB 2023-06-07 22:08:07 +02:00
Ingo Oppermann
bd75a5ad0f
Add SyncInterval and NodeRecoverTimeout to cluster config 2023-06-07 10:18:23 +02:00
Ingo Oppermann
fc03bf73a2
Make resource manager a main module and expose more details 2023-06-06 21:28:08 +02:00
Ingo Oppermann
3ac7ead20d
Add the resource limits to the metrics 2023-06-06 15:20:59 +02:00
Ingo Oppermann
1f6f7c9f59
Add API endpoint to leave cluster gracefully, hard leave on exit 2023-06-06 11:22:53 +02:00
Ingo Oppermann
8829b8fff0
Respect domain in cluster DB, allow metadata in process config for cluster 2023-06-05 21:19:11 +02:00
Ingo Oppermann
747e443e25
Fix tests 2023-06-02 16:51:23 +02:00
Ingo Oppermann
a79cfa0c77
Merge branch 'clusteriam' into vod 2023-06-01 15:16:54 +02:00
Ingo Oppermann
f21ca2055e
Use IAM for placeholders, simplify proxy nodes 2023-05-31 21:49:25 +02:00
Ingo Oppermann
89cc42c4b9
Add cluster IAM, update dependencies 2023-05-30 21:24:12 +02:00
Ingo Oppermann
e9034aa171
Create identity and access packages for IAM 2023-05-25 16:16:29 +02:00
Ingo Oppermann
710d5c595f
Merge branch 'iam' into clusteriam 2023-05-25 11:05:26 +02:00
Ingo Oppermann
10df8d51b1
Simplify IAM user config, only create users and policies if not previously run 2023-05-24 22:28:24 +02:00
Ingo Oppermann
26d556d8b8
Merge branch 'dev' into vod 2023-05-24 14:42:41 +02:00
Ingo Oppermann
3c89cbb831
Create identities for basic auth access to mount points 2023-05-24 14:29:14 +02:00
Ingo Oppermann
ccac2ffd5d
Various updates
- rebrand group to domain
- move IAM to the API (rest and graph) for enforcing "process:" rules
- add abstraction layer for restream store in order to decouple internal format from format on disk
- move playout handler into restreamHandler
- remove user from restream interface
- add TaskID type that includes the process id and its domain
2023-05-23 15:47:06 +02:00
Ingo Oppermann
ead6e64442
Merge branch 'dev' into iam 2023-05-15 21:17:23 +02:00
Ingo Oppermann
adf43eaf01
Merge branch 'dev' into cluster 2023-05-12 12:59:30 +02:00
Ingo Oppermann
e0fdc37e8d
Bump version to 16.13.0 2023-05-08 12:52:14 +02:00
Ingo Oppermann
cd58cf3df7
Fix panics if cluster is not enabled 2023-05-05 11:56:39 +02:00
Ingo Oppermann
7f59c188cf
Remove node storage, use raft configuration instead; re-establish file and stream proxying 2023-05-04 19:49:53 +02:00
Ingo Oppermann
d201921a33
Allow to provide complete cluster configuration
Replace CORE_CLUSTER_JOIN_ADDRESS with CORE_CLUSTER_PEERS. This is
a comma separated list of cluster members with their IDs of the form
ID@host:port

On startup the node tries to connect to all the peers. In case of
sudden deaths of a node this will allow to find back into the
cluster. The list in CLUSTER_PEERS is a starting point of known
peers. Other node that are not in that list can still join the
cluster.

File and stream proxy has been moved to the Proxy type.
2023-05-03 16:13:05 +02:00
Ingo Oppermann
4eb0258ba4
Add automatic bootstrap and recovery 2023-05-02 15:49:46 +02:00
Ingo Oppermann
7527f7833b
Add resource manager 2023-04-27 17:02:20 +02:00
Ingo Oppermann
3a1825cf61
Expose resource usage in report history 2023-04-25 15:55:32 +02:00
Ingo Oppermann
386bdc0b6e
WIP: add emergency leader, api circuit breaker 2023-04-21 17:19:21 +02:00
Ingo Oppermann
27f19f9188
WIP: internal API 2023-04-17 14:01:20 +02:00
Ingo Oppermann
7643959bf8
WIP: raft leadership 2023-04-13 21:44:24 +02:00
Ingo Oppermann
e6b64dbe97
WIP: raft 2023-04-13 09:42:19 +02:00
Ingo Oppermann
ffdf6d3323
Merge branch 'dev' into cluster 2023-04-12 15:27:38 +02:00
Ingo Oppermann
8326160b28
Add option to set GOMAXPROCS automatically
Use the debug.auto_max_procs config value or the CORE_DEBUG_AUTO_MAX_PROCS
environment variable. Set it to "true" to enable this feature.
2023-04-11 17:12:01 +02:00
Ingo Oppermann
0bd68d6dc3
Fix debug.force_gc behaviour 2023-04-11 11:08:11 +02:00
Ingo Oppermann
507b4f4b9a
Merge branch 'dev' into vod 2023-04-11 08:29:39 +02:00
Ingo Oppermann
84817f137a
Merge branch 'dev' into iam 2023-04-03 15:44:16 +02:00
Ingo Oppermann
99c571d623
Add event filter support 2023-03-23 14:50:59 +01:00
Ingo Oppermann
b84fdddd81
Add /api/v3/events endpoint 2023-03-23 11:11:47 +01:00
Ingo Oppermann
674c444d17
Merge branch 'dev' into vod 2023-03-17 18:41:16 +01:00
Ingo Oppermann
3149572a64
Fix freeing up S3 mounts 2023-03-17 18:40:20 +01:00
Ingo Oppermann
7e7aadc6cb
Allow to bulk delete files based on ListOptions 2023-03-17 15:56:15 +01:00
Ingo Oppermann
4ce8a0eaa3
Add config value for how many minimal process report should be kept in the history
A minimal history is a history entry without log and prelude.

The corresponding config entry is ffmpeg.log.max_minimal_history. This value is
added on top of the ffmpeg.log.max_history value. I.e. the latest max_history
entries contain the log and prelude, and the remaining entries don't have the
log and prelude. In total there are max_minimal_history+max_history history
entries.

If you want no history, set both values to 0.
If you want only full history, set max_minimal_history to 0.
If you want only minimal history, set max_history to 0.
2023-03-16 12:25:06 +01:00
Ingo Oppermann
6e93c1d5a1
Get rid of $localhost pseudo user 2023-03-09 21:10:04 +01:00
Ingo Oppermann
a0ef3ab5ee
WIP: add /api/v3/iam/group/* endpoints 2023-03-09 19:03:58 +01:00
Ingo Oppermann
d101a76e9e
Change anon user to localhost user only if DisableLocalhost is set 2023-03-08 15:21:55 +01:00
Ingo Oppermann
8755117e92
Add /api/v3/iam/user endpoints 2023-03-07 16:31:58 +01:00