Ingo Oppermann
519f0cfb2b
Fix buffer re-use, use swiss-map backend
2024-10-14 14:26:15 +02:00
Ingo Oppermann
92decc7111
Use global buffer pool where feasible
2024-10-14 10:51:35 +02:00
Ingo Oppermann
719449a4c8
Implement own byte buffer type
2024-10-10 16:35:39 +02:00
Ingo Oppermann
a581f1dbc2
User buffer pool where appropriate
2024-10-10 15:09:50 +02:00
Ingo Oppermann
f97943b275
Move content encoding in the beginning of the middleware chain, update dependencies
2024-10-09 14:25:42 +02:00
Ingo Oppermann
1e9f7f647a
Add -gc option
2024-10-09 10:29:33 +02:00
Ingo Oppermann
fe2cbd4f60
Add buffer pool for memfs
2024-10-08 14:27:23 +02:00
Ingo Oppermann
0a4118fa38
Reset buffer
2024-10-03 21:30:53 +02:00
Ingo Oppermann
b2a1909d79
Use byte array on stack for copying data, limit size hint
2024-10-03 15:01:00 +02:00
Ingo Oppermann
385628382c
Fix linter error
2024-09-05 14:10:16 +02:00
Ingo Oppermann
3756ce4977
Add AppendFileReader to filesystem, allows session logging with less I/O
2024-08-20 17:34:50 +02:00
Ingo Oppermann
1327fd6e21
Add memfs storage based on dolthub's swiss maps
2024-08-20 14:14:47 +02:00
Ingo Oppermann
46950372be
WIP: Optimize copy from io.Reader, allow to suggest file size
2024-07-24 15:40:28 +02:00
Ingo Oppermann
79791d190b
Optimize isDir function on memfs
2024-07-24 12:55:28 +02:00
Ingo Oppermann
7c3bc983a9
Rename lock for currentSize
2024-03-15 15:28:13 +01:00
Ingo Oppermann
91d07c6013
Replace memfs storage backend with a faster one
2024-03-15 14:48:32 +01:00
Ingo Oppermann
d14b0b9b13
Don't get always a new random string
2024-03-15 11:51:10 +01:00
Ingo Oppermann
a77edb8b78
Add concurrency benchmark for memfs
2024-03-14 11:19:16 +01:00
Ingo Oppermann
0e5c5f52c5
Add lock for accessing files map
2024-03-14 11:18:45 +01:00
Ingo Oppermann
dc11d33a97
Add benchmark for reading files
2023-08-03 09:38:36 +03:00
Ingo Oppermann
97a8e0f815
Use same permission mask
2023-07-19 17:41:02 +02:00
Ingo Oppermann
bdcf4bdeb4
Introduce ErrNotExist and ErrExist in io/fs package
2023-07-19 16:42:30 +02:00
Ingo Oppermann
c1b47036d6
Fix error if history file doesn't exist
2023-07-19 13:17:55 +02:00
Ingo Oppermann
ee45c85c29
Fix logging, add json tags, fix create directories before copy/rename a file
2023-06-20 16:14:24 +02:00
Ingo Oppermann
129058e633
Improve FS.List and FS.RemoveList functions, improve CleanupFS
...
The FS.List and FS.RemoveList are up to 42x faster by precompiling the
globbing pattern.
The CleanupFS from restreamer/fs is up to 32x faster in the benchmarks
and cleanup is now only every 5 seconds instead of every second.
2023-06-12 11:36:07 +02:00
Ingo Oppermann
a79cfa0c77
Merge branch 'clusteriam' into vod
2023-06-01 15:16:54 +02:00
Ingo Oppermann
2c5f32fb26
Remove bytes.Buffer pool for memfs in order to be less a memory hog
2023-04-06 21:43:18 +02:00
Ingo Oppermann
d2e5720fa2
Fix limiting part sizes for S3 upload
2023-04-06 12:06:50 +02:00
Ingo Oppermann
508d82d579
Fix high memory consumption when copying files to S3
2023-04-05 10:30:17 +02:00
Ingo Oppermann
83f94e1ef5
Fix concurrent map read and map write
2023-03-24 09:13:12 +01:00
Ingo Oppermann
a692f88b69
Add DELETE /api/v3/fs/:storage endpoint
...
Works similar to the GET endpoint. It has the same query parameter (without
"sort" and "order"). A glob pattern has to be provided. To select all
files, use the glob pattern "/**".
2023-03-17 17:05:56 +01:00
Ingo Oppermann
7e7aadc6cb
Allow to bulk delete files based on ListOptions
2023-03-17 15:56:15 +01:00
Ingo Oppermann
ecfbbe3857
Allow to list files by ranges of size and/or lastmod
...
The listing options are implemented by the query parameters size_min,
size_max, lastmod_start, and lastmod_end, or by the new ListOptions
type. size_min and size_max expect a number of bytes, lastmod_start
and lastmod_end expect a unix timestamp. All values are inclusive.
2023-03-17 15:15:20 +01:00
Ingo Oppermann
b76e6a9b2c
Add Range support for HTTP GET requests on a mounted filesystem
2023-03-08 11:50:59 +01:00
Ingo Oppermann
8755117e92
Add /api/v3/iam/user endpoints
2023-03-07 16:31:58 +01:00
Ingo Oppermann
a3ff16ef30
Add PUT /api/v3/fs endpoint for file operations
2023-03-03 14:26:17 +01:00
Ingo Oppermann
1c04961fc1
Fix tests
2023-02-21 12:57:33 +01:00
Ingo Oppermann
05a176370a
Fix missing filesystem metadata and middlewares
2023-02-14 16:16:35 +01:00
Ingo Oppermann
2a3288ffd0
Use abstract filesystem for stores
2023-02-01 16:09:20 +01:00
Ingo Oppermann
f519acfd71
Add S3 storage support
2023-01-31 14:45:58 +01:00
Ingo Oppermann
dfc81ac38f
Add ffmpeg migration tool, annotate process config with ffmpeg version constraint
2022-11-02 22:02:39 +01:00
Ingo Oppermann
f1d71c202b
Fix HLS streaming and cleanup on diskfs
2022-09-08 15:00:09 +02:00
Ingo Oppermann
3e7b1751d5
Add process id and reference glob pattern matching
...
For the API endpoint /v3/process two new query parameter are introduced
in order to list only processes that match a pattern for the id and the
reference: idpattern and refpattern. The pattern is a glob pattern. If
patterns for both are given, the results will be intersected. If you use
other query parameters such as id or reference, they will be applied
after the result of the pattern matching.
2022-08-17 07:55:44 +03:00
Jan Stabenow
eb1cc37456
Add GoSRT & improvements (repo-merge)
...
Commits (Ingo Oppermann):
- Add experimental SRT connection stats and logs
- Hide /config/reload endpoint in reade-only mode
- Add SRT server
- Create v16 in go.mod
- Fix data races, tests, lint, and update dependencies
- Add trailing slash for routed directories (datarhei/restreamer#340 )
- Allow relative URLs in content in static routes
Co-Authored-By: Ingo Oppermann <57445+ioppermann@users.noreply.github.com>
2022-06-23 22:13:58 +02:00
Jan Stabenow
9746248c10
Add v16.8.0
2022-06-03 17:21:52 +02:00
Jan Stabenow
9c0b535199
Add v16.7.2
2022-05-13 19:26:45 +02:00
