docs: update README to reflect MongoDB storage architecture and migration system

This commit is contained in:
juancarmore 2025-11-18 10:59:09 +01:00
parent 0f237af827
commit f74b50d5c8
2 changed files with 89 additions and 23 deletions

View File

@ -50,7 +50,6 @@ Before starting, ensure you have the following installed:
curl -sSL https://get.livekit.io/cli | bash curl -sSL https://get.livekit.io/cli | bash
``` ```
## Getting Started ## Getting Started
Set up your local development environment by cloning the necessary repositories into a shared folder. This ensures the `openvidu-components-angular` library is available for development and linking. Set up your local development environment by cloning the necessary repositories into a shared folder. This ensures the `openvidu-components-angular` library is available for development and linking.
@ -82,7 +81,7 @@ cd openvidu-meet
Then, the application will be available at [http://localhost:6080](http://localhost:6080). Then, the application will be available at [http://localhost:6080](http://localhost:6080).
> **Note:** Livereload is also available at [http://localhost:5080](http://localhost:5080). > **Note:** Livereload is also available at [http://localhost:6081](http://localhost:6081).
## Development ## Development
@ -95,6 +94,7 @@ The recommended way to develop is using the integrated development mode that wat
``` ```
This command starts concurrent watchers for: This command starts concurrent watchers for:
- **openvidu-components-angular**: Core Angular components library - **openvidu-components-angular**: Core Angular components library
- **Typings**: Shared type definitions with automatic sync - **Typings**: Shared type definitions with automatic sync
- **Backend**: Node.js server with nodemon auto-restart - **Backend**: Node.js server with nodemon auto-restart
@ -103,6 +103,7 @@ This command starts concurrent watchers for:
> [!NOTE] > [!NOTE]
> The backend uses `backend/.env.development` for environment variables during development. Configure your LiveKit credentials there: > The backend uses `backend/.env.development` for environment variables during development. Configure your LiveKit credentials there:
>
> ```env > ```env
> LIVEKIT_URL=ws://localhost:7880 > LIVEKIT_URL=ws://localhost:7880
> LIVEKIT_API_KEY=your-api-key > LIVEKIT_API_KEY=your-api-key
@ -170,6 +171,7 @@ The `meet.sh` script supports flags to optimize CI/CD pipelines:
``` ```
**Available flags:** **Available flags:**
- `--skip-install`: Skip dependency installation - `--skip-install`: Skip dependency installation
- `--skip-build`: Skip build steps - `--skip-build`: Skip build steps
- `--skip-typings`: Skip typings build (use when already built) - `--skip-typings`: Skip typings build (use when already built)
@ -225,8 +227,9 @@ The test app will be available at [http://localhost:5080](http://localhost:5080)
``` ```
Documentation files will be generated in: Documentation files will be generated in:
- **Webcomponent**: `docs/webcomponent-*.md` (events, commands, attributes) - **Webcomponent**: `docs/webcomponent-*.md` (events, commands, attributes)
- **REST API**: `backend/public/openapi/public.html` - **REST API**: `meet-ce/backend/public/openapi/public.html`
If you specify an output directory, the documentation will be copied there. If you specify an output directory, the documentation will be copied there.
@ -289,8 +292,7 @@ openvidu-meet/
│ ├── src/ │ ├── src/
│ │ ├── api-key.ts │ │ ├── api-key.ts
│ │ ├── auth-config.ts │ │ ├── auth-config.ts
│ │ ├── participant.ts │ │ ├── room.ts
│ │ ├── event.model.ts
│ │ └── ... │ │ └── ...
│ └── package.json │ └── package.json
@ -302,9 +304,16 @@ openvidu-meet/
├── backend/ # Node.js/Express backend ├── backend/ # Node.js/Express backend
│ ├── src/ │ ├── src/
│ │ ├── config/ # Configuration files
│ │ ├── controllers/ # REST API controllers │ │ ├── controllers/ # REST API controllers
│ │ ├── services/ # Business logic │ │ ├── helpers/ # Helper functions
│ │ ├── middleware/ # Express middleware │ │ ├── middleware/ # Express middleware
│ │ ├── migrations/ # Database migration scripts
│ │ ├── models/ # Domain models
│ │ ├── repositories/ # Database interaction
│ │ ├── routes/ # API route definitions
│ │ ├── services/ # Business logic
│ │ ├── utils/ # Utility functions
│ │ └── environment.ts # Environment configuration │ │ └── environment.ts # Environment configuration
│ ├── openapi/ # OpenAPI specifications │ ├── openapi/ # OpenAPI specifications
│ └── public/ # Static files (includes built frontend) │ └── public/ # Static files (includes built frontend)
@ -412,6 +421,7 @@ Licensed under the Apache License, Version 2.0. See [LICENSE](LICENSE) for detai
- [OpenVidu Website](https://openvidu.io/) - [OpenVidu Website](https://openvidu.io/)
- [OpenVidu Meet](https://openvidu.io/latest/meet/) - [OpenVidu Meet](https://openvidu.io/latest/meet/)
--- ---
For questions and support, visit our [community forum](https://openvidu.discourse.group/). For questions and support, visit our [community forum](https://openvidu.discourse.group/).

View File

@ -23,9 +23,26 @@ pnpm install
pnpm run build:prod pnpm run build:prod
``` ```
## Storage Structure ## Storage Architecture
The OpenVidu Meet backend uses an S3 bucket to store all application data, including rooms, recordings, user information, and system config. The bucket follows a hierarchical structure organized as follows: The OpenVidu Meet backend uses **MongoDB** as its primary data storage system for all application data, including rooms, recordings, user information, API keys, and system configuration.
### MongoDB Collections
The application manages the following MongoDB collections:
- **`meetglobalconfigs`**: System-wide configuration (singleton collection)
- **`meetusers`**: User accounts with authentication and roles
- **`meetapikeys`**: API keys for authentication
- **`meetrooms`**: Room configurations and metadata
- **`meetrecordings`**: Recording metadata and access information
- **`meetmigrations`**: Migration tracking for data and schema migrations
Each document in these collections includes a `schemaVersion` field for schema evolution tracking (internal use only, not exposed via API).
### Legacy Storage (S3/ABS/GCS)
Prior versions of OpenVidu Meet used cloud object storage (S3, Azure Blob Storage, or Google Cloud Storage) for data persistence. The legacy storage structure followed this organization:
### Bucket Structure ### Bucket Structure
@ -109,6 +126,45 @@ Where:
This naming convention ensures uniqueness and provides traceability between the recording file, its metadata, and the originating room session. This naming convention ensures uniqueness and provides traceability between the recording file, its metadata, and the originating room session.
---
## Data Migration System
OpenVidu Meet includes a comprehensive migration system to handle data persistence changes and schema evolution.
### Legacy Storage to MongoDB Migration
On first startup, the application automatically migrates existing data from legacy storage (S3/Azure Blob Storage/Google Cloud Storage) to MongoDB. This migration:
- **Runs automatically** on application startup if legacy storage is configured
- **Is idempotent** - safe to run multiple times (skips already migrated data)
- **Preserves all data** - rooms, recordings, users, API keys, and global config
- **Tracks progress** in the `meetmigrations` collection
- **Is HA-safe** using distributed locks to prevent concurrent migrations
### MongoDB Schema Migration System
The application uses a schema versioning system to safely evolve MongoDB document structures over time. This system:
- **Runs automatically** at startup before accepting requests
- **Tracks schema versions** via the `schemaVersion` field in each document
- **Supports forward-only migrations** (v1 → v2 → v3)
- **Processes in batches** for efficiency with large collections
- **Is HA-safe** using distributed locks
- **Validates before execution** to ensure migration safety
Schema migrations handle scenarios like:
- Adding new required fields with default values
- Removing deprecated fields
- Renaming or restructuring fields
- Data type transformations
For detailed information about creating and managing schema migrations, see:
📖 **[Schema Migration Documentation](./src/migrations/README.md)**
---
## Recordings ## Recordings
The recording feature is based on the following key concepts: The recording feature is based on the following key concepts: