Ingo Oppermann
29143753f6
Minor adjustments to modern go
2025-12-05 16:38:05 +01:00
Ingo Oppermann
e077cd48a6
Integrate media events into cluster
2025-10-06 17:36:23 +02:00
Ingo Oppermann
86437171f3
Add timestamp to fs events, allow to provide a glob pattern
2025-09-23 12:50:23 +02:00
Ingo Oppermann
5c7187e373
Add events to disk filesystem
2025-09-22 21:51:37 +02:00
Ingo Oppermann
bbf40b6e50
Add filesystem event handler
2025-09-22 17:10:05 +02:00
Ingo Oppermann
062204eca9
Add filesystem events
2025-09-18 22:28:07 +02:00
Ingo Oppermann
46810bf64d
Limit s3 filesystem to put, get, and delete, no more listing. reject s3 cleanup rules with wildcards
2025-07-23 13:51:41 +02:00
Ingo Oppermann
a08d8a0ca8
Extract prefix from glob pattern
2025-03-24 21:02:50 +01:00
Ingo Oppermann
519f0cfb2b
Fix buffer re-use, use swiss-map backend
2024-10-14 14:26:15 +02:00
Ingo Oppermann
92decc7111
Use global buffer pool where feasible
2024-10-14 10:51:35 +02:00
Ingo Oppermann
719449a4c8
Implement own byte buffer type
2024-10-10 16:35:39 +02:00
Ingo Oppermann
a581f1dbc2
User buffer pool where appropriate
2024-10-10 15:09:50 +02:00
Ingo Oppermann
f97943b275
Move content encoding in the beginning of the middleware chain, update dependencies
2024-10-09 14:25:42 +02:00
Ingo Oppermann
1e9f7f647a
Add -gc option
2024-10-09 10:29:33 +02:00
Ingo Oppermann
fe2cbd4f60
Add buffer pool for memfs
2024-10-08 14:27:23 +02:00
Ingo Oppermann
0a4118fa38
Reset buffer
2024-10-03 21:30:53 +02:00
Ingo Oppermann
b2a1909d79
Use byte array on stack for copying data, limit size hint
2024-10-03 15:01:00 +02:00
Ingo Oppermann
385628382c
Fix linter error
2024-09-05 14:10:16 +02:00
Ingo Oppermann
3756ce4977
Add AppendFileReader to filesystem, allows session logging with less I/O
2024-08-20 17:34:50 +02:00
Ingo Oppermann
1327fd6e21
Add memfs storage based on dolthub's swiss maps
2024-08-20 14:14:47 +02:00
Ingo Oppermann
46950372be
WIP: Optimize copy from io.Reader, allow to suggest file size
2024-07-24 15:40:28 +02:00
Ingo Oppermann
79791d190b
Optimize isDir function on memfs
2024-07-24 12:55:28 +02:00
Ingo Oppermann
7c3bc983a9
Rename lock for currentSize
2024-03-15 15:28:13 +01:00
Ingo Oppermann
91d07c6013
Replace memfs storage backend with a faster one
2024-03-15 14:48:32 +01:00
Ingo Oppermann
d14b0b9b13
Don't get always a new random string
2024-03-15 11:51:10 +01:00
Ingo Oppermann
a77edb8b78
Add concurrency benchmark for memfs
2024-03-14 11:19:16 +01:00
Ingo Oppermann
0e5c5f52c5
Add lock for accessing files map
2024-03-14 11:18:45 +01:00
Ingo Oppermann
dc11d33a97
Add benchmark for reading files
2023-08-03 09:38:36 +03:00
Ingo Oppermann
97a8e0f815
Use same permission mask
2023-07-19 17:41:02 +02:00
Ingo Oppermann
bdcf4bdeb4
Introduce ErrNotExist and ErrExist in io/fs package
2023-07-19 16:42:30 +02:00
Ingo Oppermann
c1b47036d6
Fix error if history file doesn't exist
2023-07-19 13:17:55 +02:00
Ingo Oppermann
ee45c85c29
Fix logging, add json tags, fix create directories before copy/rename a file
2023-06-20 16:14:24 +02:00
Ingo Oppermann
129058e633
Improve FS.List and FS.RemoveList functions, improve CleanupFS
...
The FS.List and FS.RemoveList are up to 42x faster by precompiling the
globbing pattern.
The CleanupFS from restreamer/fs is up to 32x faster in the benchmarks
and cleanup is now only every 5 seconds instead of every second.
2023-06-12 11:36:07 +02:00
Ingo Oppermann
a79cfa0c77
Merge branch 'clusteriam' into vod
2023-06-01 15:16:54 +02:00
Ingo Oppermann
2c5f32fb26
Remove bytes.Buffer pool for memfs in order to be less a memory hog
2023-04-06 21:43:18 +02:00
Ingo Oppermann
d2e5720fa2
Fix limiting part sizes for S3 upload
2023-04-06 12:06:50 +02:00
Ingo Oppermann
508d82d579
Fix high memory consumption when copying files to S3
2023-04-05 10:30:17 +02:00
Ingo Oppermann
83f94e1ef5
Fix concurrent map read and map write
2023-03-24 09:13:12 +01:00
Ingo Oppermann
a692f88b69
Add DELETE /api/v3/fs/:storage endpoint
...
Works similar to the GET endpoint. It has the same query parameter (without
"sort" and "order"). A glob pattern has to be provided. To select all
files, use the glob pattern "/**".
2023-03-17 17:05:56 +01:00
Ingo Oppermann
7e7aadc6cb
Allow to bulk delete files based on ListOptions
2023-03-17 15:56:15 +01:00
Ingo Oppermann
ecfbbe3857
Allow to list files by ranges of size and/or lastmod
...
The listing options are implemented by the query parameters size_min,
size_max, lastmod_start, and lastmod_end, or by the new ListOptions
type. size_min and size_max expect a number of bytes, lastmod_start
and lastmod_end expect a unix timestamp. All values are inclusive.
2023-03-17 15:15:20 +01:00
Ingo Oppermann
b76e6a9b2c
Add Range support for HTTP GET requests on a mounted filesystem
2023-03-08 11:50:59 +01:00
Ingo Oppermann
8755117e92
Add /api/v3/iam/user endpoints
2023-03-07 16:31:58 +01:00
Ingo Oppermann
a3ff16ef30
Add PUT /api/v3/fs endpoint for file operations
2023-03-03 14:26:17 +01:00
Ingo Oppermann
1c04961fc1
Fix tests
2023-02-21 12:57:33 +01:00
Ingo Oppermann
05a176370a
Fix missing filesystem metadata and middlewares
2023-02-14 16:16:35 +01:00
Ingo Oppermann
2a3288ffd0
Use abstract filesystem for stores
2023-02-01 16:09:20 +01:00
Ingo Oppermann
f519acfd71
Add S3 storage support
2023-01-31 14:45:58 +01:00
Ingo Oppermann
dfc81ac38f
Add ffmpeg migration tool, annotate process config with ffmpeg version constraint
2022-11-02 22:02:39 +01:00
Ingo Oppermann
f1d71c202b
Fix HLS streaming and cleanup on diskfs
2022-09-08 15:00:09 +02:00
