diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md new file mode 100644 index 000000000..44d4f99fd --- /dev/null +++ b/docs/ARCHITECTURE.md @@ -0,0 +1,1016 @@ +# Trilium Notes - Technical Architecture Documentation + +> **Version:** 0.99.3 +> **Last Updated:** November 2025 +> **Maintainer:** TriliumNext Team + +## Table of Contents + +1. [Introduction](#introduction) +2. [High-Level Architecture](#high-level-architecture) +3. [Monorepo Structure](#monorepo-structure) +4. [Core Architecture Patterns](#core-architecture-patterns) +5. [Data Layer](#data-layer) +6. [Caching System](#caching-system) +7. [Frontend Architecture](#frontend-architecture) +8. [Backend Architecture](#backend-architecture) +9. [API Architecture](#api-architecture) +10. [Build System](#build-system) +11. [Testing Strategy](#testing-strategy) +12. [Security Architecture](#security-architecture) +13. [Related Documentation](#related-documentation) + +--- + +## Introduction + +Trilium Notes is a hierarchical note-taking application built as a TypeScript monorepo. It supports multiple deployment modes (desktop, server, mobile web) and features advanced capabilities including synchronization, scripting, encryption, and rich content editing. + +### Key Characteristics + +- **Monorepo Architecture**: Uses pnpm workspaces for dependency management +- **Multi-Platform**: Desktop (Electron), Server (Node.js/Express), and Mobile Web +- **TypeScript-First**: Strong typing throughout the codebase +- **Plugin-Based**: Extensible architecture for note types and UI components +- **Offline-First**: Full functionality without network connectivity +- **Synchronization-Ready**: Built-in sync protocol for multi-device usage + +### Technology Stack + +- **Runtime**: Node.js (backend), Browser/Electron (frontend) +- **Language**: TypeScript, JavaScript +- **Database**: SQLite (better-sqlite3) +- **Build Tools**: Vite, ESBuild, pnpm +- **UI Framework**: Custom widget-based system +- **Rich Text**: CKEditor 5 (customized) +- **Code Editing**: CodeMirror 6 +- **Desktop**: Electron +- **Server**: Express.js + +--- + +## High-Level Architecture + +Trilium follows a **client-server architecture** even in desktop mode, where Electron runs both the backend server and frontend client within the same process. + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Frontend │ +│ ┌────────────┐ ┌────────────┐ ┌────────────┐ │ +│ │ Widgets │ │ Froca │ │ UI │ │ +│ │ System │ │ Cache │ │ Services │ │ +│ └────────────┘ └────────────┘ └────────────┘ │ +│ │ │ +│ WebSocket / REST API │ +│ │ │ +└─────────────────────────┼────────────────────────────────────┘ + │ +┌─────────────────────────┼────────────────────────────────────┐ +│ Backend Server │ +│ ┌────────────┐ ┌────────────┐ ┌────────────┐ │ +│ │ Express │ │ Becca │ │ Script │ │ +│ │ Routes │ │ Cache │ │ Engine │ │ +│ └────────────┘ └────────────┘ └────────────┘ │ +│ │ │ +│ ┌────┴─────┐ │ +│ │ SQLite │ │ +│ │ Database │ │ +│ └──────────┘ │ +└─────────────────────────────────────────────────────────────┘ +``` + +### Deployment Modes + +1. **Desktop Application** + - Electron wrapper running both frontend and backend + - Local SQLite database + - Full offline functionality + - Cross-platform (Windows, macOS, Linux) + +2. **Server Installation** + - Node.js server exposing web interface + - Multi-user capable + - Can sync with desktop clients + - Docker deployment supported + +3. **Mobile Web** + - Optimized responsive interface + - Accessed via browser + - Requires server installation + +--- + +## Monorepo Structure + +Trilium uses **pnpm workspaces** to manage its monorepo structure, with apps and packages clearly separated. + +``` +trilium/ +├── apps/ # Runnable applications +│ ├── client/ # Frontend application (shared by server & desktop) +│ ├── server/ # Node.js server with web interface +│ ├── desktop/ # Electron desktop application +│ ├── web-clipper/ # Browser extension for web content capture +│ ├── db-compare/ # Database comparison tool +│ ├── dump-db/ # Database export tool +│ ├── edit-docs/ # Documentation editing tool +│ ├── build-docs/ # Documentation build tool +│ └── website/ # Marketing website +│ +├── packages/ # Shared libraries +│ ├── commons/ # Shared interfaces and utilities +│ ├── ckeditor5/ # Custom rich text editor +│ ├── codemirror/ # Code editor customizations +│ ├── highlightjs/ # Syntax highlighting +│ ├── ckeditor5-admonition/ # CKEditor plugin: admonitions +│ ├── ckeditor5-footnotes/ # CKEditor plugin: footnotes +│ ├── ckeditor5-keyboard-marker/# CKEditor plugin: keyboard shortcuts +│ ├── ckeditor5-math/ # CKEditor plugin: math equations +│ ├── ckeditor5-mermaid/ # CKEditor plugin: diagrams +│ ├── express-partial-content/ # HTTP partial content middleware +│ ├── share-theme/ # Shared note theme +│ ├── splitjs/ # Split pane library +│ └── turndown-plugin-gfm/ # Markdown conversion +│ +├── docs/ # Documentation +├── scripts/ # Build and utility scripts +└── patches/ # Package patches (via pnpm) +``` + +### Package Dependencies + +The monorepo uses workspace protocol (`workspace:*`) for internal dependencies: + +``` +desktop → client → commons +server → client → commons +client → ckeditor5, codemirror, highlightjs +ckeditor5 → ckeditor5-* plugins +``` + +--- + +## Core Architecture Patterns + +### Three-Layer Cache System + +Trilium implements a sophisticated **three-tier caching system** to optimize performance and enable offline functionality: + +#### 1. Becca (Backend Cache) + +Located at: `apps/server/src/becca/` + +```typescript +// Becca caches all entities in memory +class Becca { + notes: Record + branches: Record + attributes: Record + attachments: Record + // ... other entity collections +} +``` + +**Responsibilities:** +- Server-side entity cache +- Maintains complete note tree in memory +- Handles entity relationships and integrity +- Provides fast lookups without database queries +- Manages entity lifecycle (create, update, delete) + +**Key Files:** +- `becca.ts` - Main cache instance +- `becca_loader.ts` - Loads entities from database +- `becca_service.ts` - Cache management operations +- `entities/` - Entity classes (BNote, BBranch, etc.) + +#### 2. Froca (Frontend Cache) + +Located at: `apps/client/src/services/froca.ts` + +```typescript +// Froca is a read-only mirror of backend data +class Froca { + notes: Record + branches: Record + attributes: Record + // ... other entity collections +} +``` + +**Responsibilities:** +- Frontend read-only cache +- Lazy loading of note tree +- Minimizes API calls +- Enables fast UI rendering +- Synchronizes with backend via WebSocket + +**Loading Strategy:** +- Initial load: root notes and immediate children +- Lazy load: notes loaded when accessed +- When note is loaded, all parent and child branches load +- Deleted entities tracked via missing branches + +#### 3. Shaca (Share Cache) + +Located at: `apps/server/src/share/` + +**Responsibilities:** +- Optimized cache for shared/published notes +- Handles public note access without authentication +- Performance-optimized for high-traffic scenarios +- Separate from main Becca to isolate concerns + +### Entity System + +Trilium's data model is based on five core entities: + +``` +┌──────────────────────────────────────────────────────────┐ +│ Note Tree │ +│ │ +│ ┌─────────┐ │ +│ │ Note │ │ +│ │ (BNote) │ │ +│ └────┬────┘ │ +│ │ │ +│ │ linked by │ +│ ▼ │ +│ ┌──────────┐ ┌─────────────┐ │ +│ │ Branch │◄────────│ Attribute │ │ +│ │(BBranch) │ │ (BAttribute)│ │ +│ └──────────┘ └─────────────┘ │ +│ │ │ +│ │ creates │ +│ ▼ │ +│ ┌──────────┐ ┌─────────────┐ │ +│ │ Revision │ │ Attachment │ │ +│ │(BRevision│ │(BAttachment)│ │ +│ └──────────┘ └─────────────┘ │ +│ │ +└──────────────────────────────────────────────────────────┘ +``` + +#### Entity Definitions + +**1. BNote** (`apps/server/src/becca/entities/bnote.ts`) +- Represents a note with title, content, and metadata +- Type can be: text, code, file, image, canvas, mermaid, etc. +- Contains content via blob reference +- Can be protected (encrypted) +- Has creation and modification timestamps + +**2. BBranch** (`apps/server/src/becca/entities/bbranch.ts`) +- Represents parent-child relationship between notes +- Enables note cloning (multiple parents) +- Contains positioning information +- Has optional prefix for customization +- Tracks expansion state in tree + +**3. BAttribute** (`apps/server/src/becca/entities/battribute.ts`) +- Key-value metadata attached to notes +- Two types: labels (tags) and relations (links) +- Can be inheritable to child notes +- Used for search, organization, and scripting +- Supports promoted attributes (displayed prominently) + +**4. BRevision** (`apps/server/src/becca/entities/brevision.ts`) +- Stores historical versions of note content +- Automatic versioning on edits +- Retains title, type, and content +- Enables note history browsing and restoration + +**5. BAttachment** (`apps/server/src/becca/entities/battachment.ts`) +- File attachments linked to notes +- Has owner (note), role, and mime type +- Content stored in blobs +- Can be protected (encrypted) + +**6. BBlob** (`apps/server/src/becca/entities/bblob.ts`) +- Binary large object storage +- Stores actual note content and attachments +- Referenced by notes, revisions, and attachments +- Supports encryption for protected content + +### Widget-Based UI + +The frontend uses a **widget system** for modular, reusable UI components. + +Located at: `apps/client/src/widgets/` + +```typescript +// Widget Hierarchy +BasicWidget +├── NoteContextAwareWidget (responds to note changes) +│ ├── RightPanelWidget (displayed in right sidebar) +│ └── Type-specific widgets +├── Container widgets (tabs, ribbons, etc.) +└── Specialized widgets (search, calendar, etc.) +``` + +**Base Classes:** + +1. **BasicWidget** (`basic_widget.ts`) + - Base class for all UI components + - Lifecycle: construction → rendering → events → destruction + - Handles DOM manipulation + - Event subscription management + - Child widget management + +2. **NoteContextAwareWidget** (`note_context_aware_widget.ts`) + - Extends BasicWidget + - Automatically updates when active note changes + - Accesses current note context + - Used for note-dependent UI + +3. **RightPanelWidget** + - Widgets displayed in right sidebar + - Collapsible sections + - Context-specific tools and information + +**Type-Specific Widgets:** + +Located at: `apps/client/src/widgets/type_widgets/` + +Each note type has a dedicated widget: +- `text_type_widget.ts` - CKEditor integration +- `code_type_widget.ts` - CodeMirror integration +- `file_type_widget.ts` - File preview and download +- `image_type_widget.ts` - Image display and editing +- `canvas_type_widget.ts` - Excalidraw integration +- `mermaid_type_widget.ts` - Diagram rendering +- And more... + +--- + +## Data Layer + +### Database Schema + +Trilium uses **SQLite** as its database engine, managed via `better-sqlite3`. + +Schema location: `apps/server/src/assets/db/schema.sql` + +**Core Tables:** + +```sql +-- Notes: Core content storage +notes ( + noteId, title, isProtected, type, mime, + blobId, isDeleted, dateCreated, dateModified +) + +-- Branches: Tree relationships +branches ( + branchId, noteId, parentNoteId, notePosition, + prefix, isExpanded, isDeleted +) + +-- Attributes: Metadata +attributes ( + attributeId, noteId, type, name, value, + position, isInheritable, isDeleted +) + +-- Revisions: Version history +revisions ( + revisionId, noteId, type, mime, title, + blobId, utcDateLastEdited +) + +-- Attachments: File attachments +attachments ( + attachmentId, ownerId, role, mime, title, + blobId, isProtected, isDeleted +) + +-- Blobs: Binary content +blobs ( + blobId, content, dateModified +) + +-- Options: Application settings +options ( + name, value, isSynced +) + +-- Entity Changes: Sync tracking +entity_changes ( + entityName, entityId, hash, changeId, + isSynced, utcDateChanged +) +``` + +### Data Access Patterns + +**Direct SQL:** +```typescript +// apps/server/src/services/sql.ts +sql.getRows("SELECT * FROM notes WHERE type = ?", ['text']) +sql.execute("UPDATE notes SET title = ? WHERE noteId = ?", [title, noteId]) +``` + +**Through Becca:** +```typescript +// Recommended approach - uses cache +const note = becca.getNote('noteId') +note.title = 'New Title' +note.save() +``` + +**Through Froca (Frontend):** +```typescript +// Read-only access +const note = froca.getNote('noteId') +console.log(note.title) +``` + +### Database Migrations + +Migration system: `apps/server/src/migrations/` + +- Sequential numbered files (e.g., `XXXX_migration_name.sql`) +- Automatic execution on version upgrade +- Schema version tracked in options table +- Both SQL and JavaScript migrations supported + +--- + +## Caching System + +### Cache Initialization + +**Backend (Becca):** +```typescript +// On server startup +await becca_loader.load() // Loads all entities into memory +becca.loaded = true +``` + +**Frontend (Froca):** +```typescript +// On app initialization +await froca.loadInitialTree() // Loads root and visible notes +// Lazy load on demand +const note = await froca.getNote(noteId) // Triggers load if not cached +``` + +### Cache Invalidation + +**Server-Side:** +- Entities automatically update cache on save +- WebSocket broadcasts changes to all clients +- Synchronization updates trigger cache refresh + +**Client-Side:** +- WebSocket listeners update Froca +- Manual reload via `froca.loadSubTree(noteId)` +- Full reload on protected session changes + +### Cache Consistency + +**Entity Change Tracking:** +```typescript +// Every entity modification tracked +entity_changes ( + entityName: 'notes', + entityId: 'note123', + hash: 'abc...', + changeId: 'change456', + utcDateChanged: '2025-11-02...' +) +``` + +**Sync Protocol:** +1. Client requests changes since last sync +2. Server returns entity_changes records +3. Client applies changes to Froca +4. Client sends local changes to server +5. Server updates Becca and database + +--- + +## Frontend Architecture + +### Application Entry Point + +**Desktop:** `apps/client/src/desktop.ts` +**Web:** `apps/client/src/index.ts` + +### Service Layer + +Located at: `apps/client/src/services/` + +Key services: +- `froca.ts` - Frontend cache +- `server.ts` - API communication +- `ws.ts` - WebSocket connection +- `tree_service.ts` - Note tree management +- `note_context.ts` - Active note tracking +- `protected_session.ts` - Encryption key management +- `link.ts` - Note linking and navigation +- `export.ts` - Note export functionality + +### UI Components + +**Main Layout:** +``` +┌──────────────────────────────────────────────────────┐ +│ Title Bar │ +├──────────┬────────────────────────┬──────────────────┤ +│ │ │ │ +│ Note │ Note Detail │ Right Panel │ +│ Tree │ Editor │ (Info, Links) │ +│ │ │ │ +│ │ │ │ +├──────────┴────────────────────────┴──────────────────┤ +│ Status Bar │ +└──────────────────────────────────────────────────────┘ +``` + +**Component Locations:** +- `widgets/containers/` - Layout containers +- `widgets/buttons/` - Toolbar buttons +- `widgets/dialogs/` - Modal dialogs +- `widgets/ribbon_widgets/` - Tab widgets +- `widgets/type_widgets/` - Note type editors + +### Event System + +**Application Events:** +```typescript +// Subscribe to events +appContext.addBeforeUnloadListener(() => { + // Cleanup before page unload +}) + +// Trigger events +appContext.trigger('noteTreeLoaded') +``` + +**Note Context Events:** +```typescript +// NoteContextAwareWidget automatically receives: +- noteSwitched() +- noteChanged() +- refresh() +``` + +### State Management + +Trilium uses **custom state management** rather than Redux/MobX: + +- `note_context.ts` - Active note and context +- `froca.ts` - Entity cache +- Component local state +- URL parameters for shareable state + +--- + +## Backend Architecture + +### Application Entry Point + +Location: `apps/server/src/main.ts` + +**Startup Sequence:** +1. Load configuration +2. Initialize database +3. Run migrations +4. Load Becca cache +5. Start Express server +6. Initialize WebSocket +7. Start scheduled tasks + +### Service Layer + +Located at: `apps/server/src/services/` + +**Core Services:** + +- **Notes Management** + - `notes.ts` - CRUD operations + - `note_contents.ts` - Content handling + - `note_types.ts` - Type-specific logic + - `cloning.ts` - Note cloning/multi-parent + +- **Tree Operations** + - `tree.ts` - Tree structure management + - `branches.ts` - Branch operations + - `consistency_checks.ts` - Tree integrity + +- **Search** + - `search/search.ts` - Main search engine + - `search/expressions/` - Search expression parsing + - `search/services/` - Search utilities + +- **Sync** + - `sync.ts` - Synchronization protocol + - `sync_update.ts` - Update handling + - `sync_mutex.ts` - Concurrency control + +- **Scripting** + - `backend_script_api.ts` - Backend script API + - `script_context.ts` - Script execution context + +- **Import/Export** + - `import/` - Various import formats + - `export/` - Export to different formats + - `zip.ts` - Archive handling + +- **Security** + - `encryption.ts` - Note encryption + - `protected_session.ts` - Session management + - `password.ts` - Password handling + +### Route Structure + +Located at: `apps/server/src/routes/` + +``` +routes/ +├── index.ts # Route registration +├── api/ # REST API endpoints +│ ├── notes.ts +│ ├── branches.ts +│ ├── attributes.ts +│ ├── search.ts +│ ├── login.ts +│ └── ... +└── custom/ # Special endpoints + ├── setup.ts + ├── share.ts + └── ... +``` + +**API Endpoint Pattern:** +```typescript +router.get('/api/notes/:noteId', (req, res) => { + const noteId = req.params.noteId + const note = becca.getNote(noteId) + res.json(note.getPojoWithContent()) +}) +``` + +### Middleware + +Key middleware components: +- `auth.ts` - Authentication +- `csrf.ts` - CSRF protection +- `request_context.ts` - Request-scoped data +- `error_handling.ts` - Error responses + +--- + +## API Architecture + +### Internal API + +**REST Endpoints** (`/api/*`) + +Used by the frontend for all operations: + +**Note Operations:** +- `GET /api/notes/:noteId` - Get note +- `POST /api/notes/:noteId/content` - Update content +- `PUT /api/notes/:noteId` - Update metadata +- `DELETE /api/notes/:noteId` - Delete note + +**Tree Operations:** +- `GET /api/tree` - Get note tree +- `POST /api/branches` - Create branch +- `PUT /api/branches/:branchId` - Update branch +- `DELETE /api/branches/:branchId` - Delete branch + +**Search:** +- `GET /api/search?query=...` - Search notes +- `GET /api/search-note/:noteId` - Execute search note + +### ETAPI (External API) + +Located at: `apps/server/src/etapi/` + +**Purpose:** Third-party integrations and automation + +**Authentication:** Token-based (ETAPI tokens) + +**OpenAPI Spec:** Auto-generated + +**Key Endpoints:** +- `/etapi/notes` - Note CRUD +- `/etapi/branches` - Branch management +- `/etapi/attributes` - Attribute operations +- `/etapi/attachments` - Attachment handling + +**Example:** +```bash +curl -H "Authorization: YOUR_TOKEN" \ + https://trilium.example.com/etapi/notes/noteId +``` + +### WebSocket API + +Located at: `apps/server/src/services/ws.ts` + +**Purpose:** Real-time updates and synchronization + +**Protocol:** WebSocket (Socket.IO-like custom protocol) + +**Message Types:** +- `sync` - Synchronization request +- `entity-change` - Entity update notification +- `refresh-tree` - Tree structure changed +- `open-note` - Open note in UI + +**Client Subscribe:** +```typescript +ws.subscribe('entity-change', (data) => { + froca.processEntityChange(data) +}) +``` + +--- + +## Build System + +### Package Manager: pnpm + +**Why pnpm:** +- Fast, disk-efficient +- Strict dependency isolation +- Native monorepo support via workspaces +- Patch package support + +**Workspace Configuration:** +```yaml +# pnpm-workspace.yaml +packages: + - 'apps/*' + - 'packages/*' +``` + +### Build Tools + +**Vite** (Development & Production) +- Fast HMR for development +- Optimized production builds +- Asset handling +- Plugin ecosystem + +**ESBuild** (TypeScript compilation) +- Fast TypeScript transpilation +- Bundling support +- Minification + +**TypeScript** +- Project references for monorepo +- Strict type checking +- Shared `tsconfig.base.json` + +### Build Scripts + +**Root `package.json` scripts:** +```json +{ + "server:start": "pnpm run --filter server dev", + "server:build": "pnpm run --filter server build", + "client:build": "pnpm run --filter client build", + "desktop:build": "pnpm run --filter desktop build", + "test:all": "pnpm test:parallel && pnpm test:sequential" +} +``` + +### Build Process + +**Development:** +```bash +pnpm install # Install dependencies +pnpm server:start # Start dev server (port 8080) +# or +pnpm desktop:start # Start Electron dev +``` + +**Production (Server):** +```bash +pnpm server:build # Build server + client +node apps/server/dist/main.js +``` + +**Production (Desktop):** +```bash +pnpm desktop:build # Build Electron app +# Creates distributable in apps/desktop/out/make/ +``` + +**Docker:** +```bash +docker build -t trilium . +docker run -p 8080:8080 trilium +``` + +### Asset Pipeline + +**Client Assets:** +- Entry: `apps/client/src/index.html` +- Bundled by Vite +- Output: `apps/client/dist/` + +**Server Static:** +- Serves client assets in production +- Public directory: `apps/server/public/` + +**Desktop:** +- Packages client assets +- Electron main process: `apps/desktop/src/main.ts` +- Electron renderer: loads client app + +--- + +## Testing Strategy + +### Test Organization + +**Parallel Tests** (can run simultaneously): +- Client tests +- Package tests +- E2E tests (isolated databases) + +**Sequential Tests** (shared resources): +- Server tests (shared database) +- CKEditor plugin tests + +### Test Frameworks + +- **Vitest** - Unit and integration tests +- **Playwright** - E2E tests +- **Happy-DOM** - DOM testing environment + +### Running Tests + +```bash +pnpm test:all # All tests +pnpm test:parallel # Fast parallel tests +pnpm test:sequential # Sequential tests only +pnpm coverage # With coverage reports +``` + +### Test Locations + +``` +apps/ +├── server/ +│ └── src/**/*.spec.ts # Server tests +├── client/ +│ └── src/**/*.spec.ts # Client tests +└── server-e2e/ + └── tests/**/*.spec.ts # E2E tests +``` + +### E2E Testing + +**Server E2E:** +- Tests full REST API +- Tests WebSocket functionality +- Tests sync protocol + +**Desktop E2E:** +- Playwright with Electron +- Tests full desktop app +- Screenshot comparison + +--- + +## Security Architecture + +### Encryption System + +**Per-Note Encryption:** +- Notes can be individually protected +- AES-256 encryption +- Password-derived encryption key (PBKDF2) +- Separate protected session management + +**Protected Session:** +- Time-limited access to protected notes +- Automatic timeout +- Re-authentication required +- Frontend: `protected_session.ts` +- Backend: `protected_session.ts` + +### Authentication + +**Password Auth:** +- PBKDF2 key derivation +- Salt per installation +- Hash verification + +**OpenID Connect:** +- External identity provider support +- OAuth 2.0 flow +- Configurable providers + +**TOTP (2FA):** +- Time-based one-time passwords +- QR code setup +- Backup codes + +### Authorization + +**Single-User Model:** +- Desktop: single user (owner) +- Server: single user per installation + +**Share Notes:** +- Public access without authentication +- Separate Shaca cache +- Read-only access + +### CSRF Protection + +**CSRF Tokens:** +- Required for state-changing operations +- Token in header or cookie +- Validation middleware + +### Input Sanitization + +**XSS Prevention:** +- DOMPurify for HTML sanitization +- CKEditor content filtering +- CSP headers + +**SQL Injection:** +- Parameterized queries only +- Better-sqlite3 prepared statements +- No string concatenation in SQL + +### Dependency Security + +**Vulnerability Scanning:** +- Renovate bot for updates +- npm audit integration +- Override vulnerable sub-dependencies + +--- + +## Related Documentation + +### User Documentation +- [User Guide](User%20Guide/User%20Guide/) - End-user features and usage +- [Installation Guide](User%20Guide/User%20Guide/Installation%20&%20Setup/) +- [Basic Concepts](User%20Guide/User%20Guide/Basic%20Concepts%20and%20Features/) + +### Developer Documentation +- [Developer Guide](Developer%20Guide/Developer%20Guide/) - Development setup +- [Environment Setup](Developer%20Guide/Developer%20Guide/Environment%20Setup.md) +- [Project Structure](Developer%20Guide/Developer%20Guide/Project%20Structure.md) +- [Adding Note Types](Developer%20Guide/Developer%20Guide/Development%20and%20architecture/Adding%20a%20new%20note%20type/) +- [Database Schema](Developer%20Guide/Developer%20Guide/Development%20and%20architecture/Database/) + +### API Documentation +- [Script API](Script%20API/) - User scripting API +- [ETAPI Documentation](https://triliumnext.github.io/Docs/Wiki/etapi) - External API + +### Additional Resources +- [CLAUDE.md](../CLAUDE.md) - AI assistant guidance +- [README.md](../README.md) - Project overview +- [SECURITY.md](../SECURITY.md) - Security policy + +--- + +## Appendices + +### Glossary + +- **Becca**: Backend Cache - server-side entity cache +- **Froca**: Frontend Cache - client-side entity mirror +- **Shaca**: Share Cache - cache for public shared notes +- **ETAPI**: External API for third-party integrations +- **Protected Note**: Encrypted note requiring password +- **Clone**: Note with multiple parent branches +- **Branch**: Parent-child relationship between notes +- **Attribute**: Metadata (label or relation) attached to note +- **Blob**: Binary large object containing note content + +### File Naming Conventions + +- `BEntity` - Backend entity (e.g., BNote, BBranch) +- `FEntity` - Frontend entity (e.g., FNote, FBranch) +- `*_widget.ts` - Widget classes +- `*_service.ts` - Service modules +- `*.spec.ts` - Test files +- `XXXX_*.sql` - Migration files + +### Architecture Decision Records + +For historical context on major architectural decisions, see: +- Migration to TypeScript monorepo +- Adoption of pnpm workspaces +- CKEditor 5 upgrade +- Entity change tracking system + +--- + +**Document Maintainer:** TriliumNext Team +**Last Review:** November 2025 +**Next Review:** When major architectural changes occur diff --git a/docs/DATABASE.md b/docs/DATABASE.md new file mode 100644 index 000000000..d02452f21 --- /dev/null +++ b/docs/DATABASE.md @@ -0,0 +1,736 @@ +# Trilium Database Architecture + +> **Related:** [ARCHITECTURE.md](ARCHITECTURE.md) | [Database Schema](Developer%20Guide/Developer%20Guide/Development%20and%20architecture/Database/) + +## Overview + +Trilium uses **SQLite** as its embedded database engine, providing a reliable, file-based storage system that requires no separate database server. The database stores all notes, their relationships, metadata, and configuration. + +## Database File + +**Location:** +- Desktop: `~/.local/share/trilium-data/document.db` (Linux/Mac) or `%APPDATA%/trilium-data/document.db` (Windows) +- Server: Configured via `TRILIUM_DATA_DIR` environment variable +- Docker: Mounted volume at `/home/node/trilium-data/` + +**Characteristics:** +- Single-file database +- Embedded (no server required) +- ACID compliant +- Cross-platform +- Supports up to 281 TB database size +- Efficient for 100k+ notes + +## Database Driver + +**Library:** `better-sqlite3` + +**Why better-sqlite3:** +- Native performance (C++ bindings) +- Synchronous API (simpler code) +- Prepared statements +- Transaction support +- Type safety + +**Usage:** +```typescript +// apps/server/src/services/sql.ts +import Database from 'better-sqlite3' + +const db = new Database('document.db') +const stmt = db.prepare('SELECT * FROM notes WHERE noteId = ?') +const note = stmt.get(noteId) +``` + +## Schema Overview + +Schema location: `apps/server/src/assets/db/schema.sql` + +**Entity Tables:** +- `notes` - Core note data +- `branches` - Tree relationships +- `attributes` - Metadata (labels/relations) +- `revisions` - Version history +- `attachments` - File attachments +- `blobs` - Binary content storage + +**System Tables:** +- `options` - Application configuration +- `entity_changes` - Change tracking for sync +- `recent_notes` - Recently accessed notes +- `etapi_tokens` - API authentication tokens +- `user_data` - User credentials +- `sessions` - Web session storage + +## Entity Tables + +### Notes Table + +```sql +CREATE TABLE notes ( + noteId TEXT NOT NULL PRIMARY KEY, + title TEXT NOT NULL DEFAULT "note", + isProtected INT NOT NULL DEFAULT 0, + type TEXT NOT NULL DEFAULT 'text', + mime TEXT NOT NULL DEFAULT 'text/html', + blobId TEXT DEFAULT NULL, + isDeleted INT NOT NULL DEFAULT 0, + deleteId TEXT DEFAULT NULL, + dateCreated TEXT NOT NULL, + dateModified TEXT NOT NULL, + utcDateCreated TEXT NOT NULL, + utcDateModified TEXT NOT NULL +); + +-- Indexes for performance +CREATE INDEX IDX_notes_title ON notes (title); +CREATE INDEX IDX_notes_type ON notes (type); +CREATE INDEX IDX_notes_dateCreated ON notes (dateCreated); +CREATE INDEX IDX_notes_dateModified ON notes (dateModified); +CREATE INDEX IDX_notes_utcDateModified ON notes (utcDateModified); +CREATE INDEX IDX_notes_blobId ON notes (blobId); +``` + +**Field Descriptions:** + +| Field | Type | Description | +|-------|------|-------------| +| `noteId` | TEXT | Unique identifier (UUID or custom) | +| `title` | TEXT | Note title (displayed in tree) | +| `isProtected` | INT | 1 if encrypted, 0 if not | +| `type` | TEXT | Note type: text, code, file, image, etc. | +| `mime` | TEXT | MIME type: text/html, application/json, etc. | +| `blobId` | TEXT | Reference to content in blobs table | +| `isDeleted` | INT | Soft delete flag | +| `deleteId` | TEXT | Unique delete operation ID | +| `dateCreated` | TEXT | Creation date (local timezone) | +| `dateModified` | TEXT | Last modified (local timezone) | +| `utcDateCreated` | TEXT | Creation date (UTC) | +| `utcDateModified` | TEXT | Last modified (UTC) | + +**Note Types:** +- `text` - Rich text with HTML +- `code` - Source code +- `file` - Binary file +- `image` - Image file +- `search` - Saved search +- `render` - Custom HTML rendering +- `relation-map` - Relationship diagram +- `canvas` - Excalidraw drawing +- `mermaid` - Mermaid diagram +- `book` - Container for documentation +- `web-view` - Embedded web page +- `mindmap` - Mind map +- `geomap` - Geographical map + +### Branches Table + +```sql +CREATE TABLE branches ( + branchId TEXT NOT NULL PRIMARY KEY, + noteId TEXT NOT NULL, + parentNoteId TEXT NOT NULL, + notePosition INTEGER NOT NULL, + prefix TEXT, + isExpanded INTEGER NOT NULL DEFAULT 0, + isDeleted INTEGER NOT NULL DEFAULT 0, + deleteId TEXT DEFAULT NULL, + utcDateModified TEXT NOT NULL +); + +-- Indexes +CREATE INDEX IDX_branches_noteId_parentNoteId ON branches (noteId, parentNoteId); +CREATE INDEX IDX_branches_parentNoteId ON branches (parentNoteId); +``` + +**Field Descriptions:** + +| Field | Type | Description | +|-------|------|-------------| +| `branchId` | TEXT | Unique identifier for this branch | +| `noteId` | TEXT | Child note ID | +| `parentNoteId` | TEXT | Parent note ID | +| `notePosition` | INT | Sort order among siblings | +| `prefix` | TEXT | Optional prefix text (e.g., "Chapter 1:") | +| `isExpanded` | INT | Tree expansion state | +| `isDeleted` | INT | Soft delete flag | +| `deleteId` | TEXT | Delete operation ID | +| `utcDateModified` | TEXT | Last modified (UTC) | + +**Key Concepts:** +- **Cloning:** A note can have multiple branches (multiple parents) +- **Position:** Siblings ordered by `notePosition` +- **Prefix:** Display text before note title in tree +- **Soft Delete:** Allows sync before permanent deletion + +### Attributes Table + +```sql +CREATE TABLE attributes ( + attributeId TEXT NOT NULL PRIMARY KEY, + noteId TEXT NOT NULL, + type TEXT NOT NULL, + name TEXT NOT NULL, + value TEXT DEFAULT '' NOT NULL, + position INT DEFAULT 0 NOT NULL, + utcDateModified TEXT NOT NULL, + isDeleted INT NOT NULL, + deleteId TEXT DEFAULT NULL, + isInheritable INT DEFAULT 0 NULL +); + +-- Indexes +CREATE INDEX IDX_attributes_name_value ON attributes (name, value); +CREATE INDEX IDX_attributes_noteId ON attributes (noteId); +CREATE INDEX IDX_attributes_value ON attributes (value); +``` + +**Field Descriptions:** + +| Field | Type | Description | +|-------|------|-------------| +| `attributeId` | TEXT | Unique identifier | +| `noteId` | TEXT | Note this attribute belongs to | +| `type` | TEXT | 'label' or 'relation' | +| `name` | TEXT | Attribute name | +| `value` | TEXT | Attribute value (text for labels, noteId for relations) | +| `position` | INT | Display order | +| `utcDateModified` | TEXT | Last modified (UTC) | +| `isDeleted` | INT | Soft delete flag | +| `deleteId` | TEXT | Delete operation ID | +| `isInheritable` | INT | Inherited by child notes | + +**Attribute Types:** + +**Labels** (key-value pairs): +```sql +-- Example: #priority=high +INSERT INTO attributes (attributeId, noteId, type, name, value) +VALUES ('attr1', 'note123', 'label', 'priority', 'high') +``` + +**Relations** (links to other notes): +```sql +-- Example: ~author=[[noteId]] +INSERT INTO attributes (attributeId, noteId, type, name, value) +VALUES ('attr2', 'note123', 'relation', 'author', 'author-note-id') +``` + +**Special Attributes:** +- `#run=frontendStartup` - Execute script on frontend load +- `#run=backendStartup` - Execute script on backend load +- `#customWidget` - Custom widget implementation +- `#iconClass` - Custom tree icon +- `#cssClass` - CSS class for note +- `#sorted` - Auto-sort children +- `#hideChildrenOverview` - Don't show child list + +### Revisions Table + +```sql +CREATE TABLE revisions ( + revisionId TEXT NOT NULL PRIMARY KEY, + noteId TEXT NOT NULL, + type TEXT DEFAULT '' NOT NULL, + mime TEXT DEFAULT '' NOT NULL, + title TEXT NOT NULL, + isProtected INT NOT NULL DEFAULT 0, + blobId TEXT DEFAULT NULL, + utcDateLastEdited TEXT NOT NULL, + utcDateCreated TEXT NOT NULL, + utcDateModified TEXT NOT NULL, + dateLastEdited TEXT NOT NULL, + dateCreated TEXT NOT NULL +); + +-- Indexes +CREATE INDEX IDX_revisions_noteId ON revisions (noteId); +CREATE INDEX IDX_revisions_utcDateCreated ON revisions (utcDateCreated); +CREATE INDEX IDX_revisions_utcDateLastEdited ON revisions (utcDateLastEdited); +CREATE INDEX IDX_revisions_blobId ON revisions (blobId); +``` + +**Revision Strategy:** +- Automatic revision created on note modification +- Configurable interval (default: daily max) +- Stores complete note snapshot +- Allows reverting to previous versions +- Can be disabled with `#disableVersioning` + +### Attachments Table + +```sql +CREATE TABLE attachments ( + attachmentId TEXT NOT NULL PRIMARY KEY, + ownerId TEXT NOT NULL, + role TEXT NOT NULL, + mime TEXT NOT NULL, + title TEXT NOT NULL, + isProtected INT NOT NULL DEFAULT 0, + position INT DEFAULT 0 NOT NULL, + blobId TEXT DEFAULT NULL, + dateModified TEXT NOT NULL, + utcDateModified TEXT NOT NULL, + utcDateScheduledForErasureSince TEXT DEFAULT NULL, + isDeleted INT NOT NULL, + deleteId TEXT DEFAULT NULL +); + +-- Indexes +CREATE INDEX IDX_attachments_ownerId_role ON attachments (ownerId, role); +CREATE INDEX IDX_attachments_blobId ON attachments (blobId); +``` + +**Attachment Roles:** +- `file` - Regular file attachment +- `image` - Image file +- `cover-image` - Note cover image +- Custom roles for specific purposes + +### Blobs Table + +```sql +CREATE TABLE blobs ( + blobId TEXT NOT NULL PRIMARY KEY, + content TEXT NULL DEFAULT NULL, + dateModified TEXT NOT NULL, + utcDateModified TEXT NOT NULL +); +``` + +**Blob Usage:** +- Stores actual content (text or binary) +- Referenced by notes, revisions, attachments +- Deduplication via hash-based blobId +- TEXT type stores both text and binary (base64) + +**Content Types:** +- **Text notes:** HTML content +- **Code notes:** Plain text source code +- **Binary notes:** Base64 encoded data +- **Protected notes:** Encrypted content + +## System Tables + +### Options Table + +```sql +CREATE TABLE options ( + name TEXT NOT NULL PRIMARY KEY, + value TEXT NOT NULL, + isSynced INTEGER DEFAULT 0 NOT NULL, + utcDateModified TEXT NOT NULL +); +``` + +**Key Options:** +- `documentId` - Unique installation ID +- `dbVersion` - Schema version +- `syncVersion` - Sync protocol version +- `passwordVerificationHash` - Password verification +- `encryptedDataKey` - Encryption key (encrypted) +- `theme` - UI theme +- Various feature flags and settings + +**Synced Options:** +- `isSynced = 1` - Synced across devices +- `isSynced = 0` - Local to this installation + +### Entity Changes Table + +```sql +CREATE TABLE entity_changes ( + id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, + entityName TEXT NOT NULL, + entityId TEXT NOT NULL, + hash TEXT NOT NULL, + isErased INT NOT NULL, + changeId TEXT NOT NULL, + componentId TEXT NOT NULL, + instanceId TEXT NOT NULL, + isSynced INTEGER NOT NULL, + utcDateChanged TEXT NOT NULL +); + +-- Indexes +CREATE UNIQUE INDEX IDX_entityChanges_entityName_entityId + ON entity_changes (entityName, entityId); +CREATE INDEX IDX_entity_changes_changeId ON entity_changes (changeId); +``` + +**Purpose:** Track all entity modifications for synchronization + +**Entity Types:** +- `notes` +- `branches` +- `attributes` +- `revisions` +- `attachments` +- `options` +- `etapi_tokens` + +### Recent Notes Table + +```sql +CREATE TABLE recent_notes ( + noteId TEXT NOT NULL PRIMARY KEY, + notePath TEXT NOT NULL, + utcDateCreated TEXT NOT NULL +); +``` + +**Purpose:** Track recently accessed notes for quick access + +### Sessions Table + +```sql +CREATE TABLE sessions ( + sid TEXT PRIMARY KEY, + sess TEXT NOT NULL, + expired TEXT NOT NULL +); +``` + +**Purpose:** HTTP session storage for web interface + +### User Data Table + +```sql +CREATE TABLE user_data ( + tmpID INT PRIMARY KEY, + username TEXT, + email TEXT, + userIDEncryptedDataKey TEXT, + userIDVerificationHash TEXT, + salt TEXT, + derivedKey TEXT, + isSetup TEXT DEFAULT "false" +); +``` + +**Purpose:** Store user authentication credentials + +### ETAPI Tokens Table + +```sql +CREATE TABLE etapi_tokens ( + etapiTokenId TEXT PRIMARY KEY NOT NULL, + name TEXT NOT NULL, + tokenHash TEXT NOT NULL, + utcDateCreated TEXT NOT NULL, + utcDateModified TEXT NOT NULL, + isDeleted INT NOT NULL DEFAULT 0 +); +``` + +**Purpose:** API token authentication for external access + +## Data Relationships + +``` + ┌──────────────┐ + │ Notes │ + └───┬──────────┘ + │ + ┌───────────┼───────────┐ + │ │ │ + ▼ ▼ ▼ + ┌────────┐ ┌──────────┐ ┌───────────┐ + │Branches│ │Attributes│ │Attachments│ + └────────┘ └──────────┘ └─────┬─────┘ + │ │ + │ │ + │ ┌──────────┐ │ + └──────▶│ Blobs │◀────────┘ + └──────────┘ + ▲ + │ + ┌────┴─────┐ + │Revisions │ + └──────────┘ +``` + +**Relationships:** +- Notes ↔ Branches (many-to-many via noteId) +- Notes → Attributes (one-to-many) +- Notes → Blobs (one-to-one) +- Notes → Revisions (one-to-many) +- Notes → Attachments (one-to-many) +- Attachments → Blobs (one-to-one) +- Revisions → Blobs (one-to-one) + +## Database Access Patterns + +### Direct SQL Access + +**Location:** `apps/server/src/services/sql.ts` + +```typescript +// Execute query (returns rows) +const notes = sql.getRows('SELECT * FROM notes WHERE type = ?', ['text']) + +// Execute query (returns single row) +const note = sql.getRow('SELECT * FROM notes WHERE noteId = ?', [noteId]) + +// Execute statement (no return) +sql.execute('UPDATE notes SET title = ? WHERE noteId = ?', [title, noteId]) + +// Insert +sql.insert('notes', { + noteId: 'new-note-id', + title: 'New Note', + type: 'text', + // ... +}) + +// Transactions +sql.transactional(() => { + sql.execute('UPDATE ...') + sql.execute('INSERT ...') +}) +``` + +### Entity-Based Access (Recommended) + +**Via Becca Cache:** + +```typescript +// Get entity from cache +const note = becca.getNote(noteId) + +// Modify and save +note.title = 'Updated Title' +note.save() // Writes to database + +// Create new +const newNote = becca.createNote({ + parentNoteId: 'root', + title: 'New Note', + type: 'text', + content: 'Hello World' +}) + +// Delete +note.markAsDeleted() +``` + +## Database Migrations + +**Location:** `apps/server/src/migrations/` + +**Migration Files:** +- Format: `XXXX_migration_name.sql` or `XXXX_migration_name.js` +- Executed in numerical order +- Version tracked in `options.dbVersion` + +**SQL Migration Example:** +```sql +-- 0280_add_new_column.sql +ALTER TABLE notes ADD COLUMN newField TEXT DEFAULT NULL; + +UPDATE options SET value = '280' WHERE name = 'dbVersion'; +``` + +**JavaScript Migration Example:** +```javascript +// 0285_complex_migration.js +module.exports = () => { + const notes = sql.getRows('SELECT * FROM notes WHERE type = ?', ['old-type']) + + for (const note of notes) { + sql.execute('UPDATE notes SET type = ? WHERE noteId = ?', + ['new-type', note.noteId]) + } +} +``` + +**Migration Process:** +1. Server checks `dbVersion` on startup +2. Compares with latest migration number +3. Executes pending migrations in order +4. Updates `dbVersion` after each +5. Restarts if migrations ran + +## Database Maintenance + +### Backup + +**Full Backup:** +```bash +# Copy database file +cp document.db document.db.backup + +# Or use Trilium's backup feature +# Settings → Backup +``` + +**Automatic Backups:** +- Daily backup (configurable) +- Stored in `backup/` directory +- Retention policy (keep last N backups) + +### Vacuum + +**Purpose:** Reclaim unused space, defragment + +```sql +VACUUM; +``` + +**When to vacuum:** +- After deleting many notes +- Database file size larger than expected +- Performance degradation + +### Integrity Check + +```sql +PRAGMA integrity_check; +``` + +**Result:** "ok" or list of errors + +### Consistency Checks + +**Built-in Consistency Checks:** + +Location: `apps/server/src/services/consistency_checks.ts` + +- Orphaned branches +- Missing parent notes +- Circular dependencies +- Invalid entity references +- Blob reference integrity + +**Run Checks:** +```typescript +// Via API +POST /api/consistency-check + +// Or from backend script +api.runConsistencyChecks() +``` + +## Performance Optimization + +### Indexes + +**Existing Indexes:** +- `notes.title` - Fast title searches +- `notes.type` - Filter by type +- `notes.dateCreated/Modified` - Time-based queries +- `branches.noteId_parentNoteId` - Tree navigation +- `attributes.name_value` - Attribute searches + +**Query Optimization:** +```sql +-- Use indexed columns in WHERE clause +SELECT * FROM notes WHERE type = 'text' -- Uses index + +-- Avoid functions on indexed columns +SELECT * FROM notes WHERE LOWER(title) = 'test' -- No index + +-- Better +SELECT * FROM notes WHERE title = 'Test' -- Uses index +``` + +### Connection Settings + +```typescript +// apps/server/src/services/sql.ts +const db = new Database('document.db', { + // Enable WAL mode for better concurrency + verbose: console.log +}) + +db.pragma('journal_mode = WAL') +db.pragma('synchronous = NORMAL') +db.pragma('cache_size = -64000') // 64MB cache +db.pragma('temp_store = MEMORY') +``` + +**WAL Mode Benefits:** +- Better concurrency (readers don't block writers) +- Faster commits +- More robust + +### Query Performance + +**Use EXPLAIN QUERY PLAN:** +```sql +EXPLAIN QUERY PLAN +SELECT * FROM notes +WHERE type = 'text' + AND dateCreated > '2025-01-01' +``` + +**Analyze slow queries:** +- Check index usage +- Avoid SELECT * +- Use prepared statements +- Batch operations in transactions + +## Database Size Management + +**Typical Sizes:** +- 1,000 notes: ~5-10 MB +- 10,000 notes: ~50-100 MB +- 100,000 notes: ~500 MB - 1 GB + +**Size Reduction Strategies:** + +1. **Delete old revisions** +2. **Remove large attachments** +3. **Vacuum database** +4. **Compact blobs** +5. **Archive old notes** + +**Blob Deduplication:** +- Blobs identified by content hash +- Identical content shares one blob +- Automatic deduplication on insert + +## Security Considerations + +### Protected Notes Encryption + +**Encryption Process:** +```typescript +// Encrypt blob content +const encryptedContent = encrypt(content, dataKey) +blob.content = encryptedContent + +// Store encrypted +sql.insert('blobs', { blobId, content: encryptedContent }) +``` + +**Encryption Details:** +- Algorithm: AES-256-CBC +- Key derivation: PBKDF2 (10,000 iterations) +- Per-note encryption +- Master key encrypted with user password + +### SQL Injection Prevention + +**Always use parameterized queries:** +```typescript +// GOOD - Safe from SQL injection +sql.execute('SELECT * FROM notes WHERE title = ?', [userInput]) + +// BAD - Vulnerable to SQL injection +sql.execute(`SELECT * FROM notes WHERE title = '${userInput}'`) +``` + +### Database File Protection + +**File Permissions:** +- Owner read/write only +- No group/other access +- Located in user-specific directory + +--- + +**See Also:** +- [ARCHITECTURE.md](ARCHITECTURE.md) - Overall architecture +- [Database Schema Files](Developer%20Guide/Developer%20Guide/Development%20and%20architecture/Database/) +- [Migration Scripts](../apps/server/src/migrations/) diff --git a/docs/README.md b/docs/README.md index be65024ee..ea2fdd378 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,4 +1,17 @@ -# Trilium Notes +# Trilium Notes Documentation + +## 📚 Technical Documentation + +**NEW:** Comprehensive technical and architectural documentation is now available! + +- **[Technical Documentation Index](TECHNICAL_DOCUMENTATION.md)** - Complete index to all technical docs +- **[Architecture Overview](ARCHITECTURE.md)** - System design and core patterns +- **[Database Architecture](DATABASE.md)** - Complete database documentation +- **[Synchronization](SYNCHRONIZATION.md)** - Sync protocol and implementation +- **[Scripting System](SCRIPTING.md)** - User scripting guide and API +- **[Security Architecture](SECURITY_ARCHITECTURE.md)** - Security implementation details + +## 📖 User Documentation Please see the [main documentation](index.md) or visit one of our translated versions: @@ -9,4 +22,11 @@ Please see the [main documentation](index.md) or visit one of our translated ver - [简体中文](README-ZH_CN.md) - [繁體中文](README-ZH_TW.md) +## 🔧 Developer Documentation + +- [Developer Guide](Developer%20Guide/Developer%20Guide/) - Development environment and contribution guide +- [Script API](Script%20API/) - Complete scripting API reference + +## 🔗 Additional Resources + For the full application README, please visit our [GitHub repository](https://github.com/triliumnext/trilium). \ No newline at end of file diff --git a/docs/SCRIPTING.md b/docs/SCRIPTING.md new file mode 100644 index 000000000..49e548c50 --- /dev/null +++ b/docs/SCRIPTING.md @@ -0,0 +1,734 @@ +# Trilium Scripting System + +> **Related:** [ARCHITECTURE.md](ARCHITECTURE.md) | [Script API Documentation](Script%20API/) + +## Overview + +Trilium features a **powerful scripting system** that allows users to extend and customize the application without modifying source code. Scripts are written in JavaScript and can execute both in the **frontend (browser)** and **backend (Node.js)** contexts. + +## Script Types + +### Frontend Scripts + +**Location:** Attached to notes with `#run=frontendStartup` attribute + +**Execution Context:** Browser environment + +**Access:** +- Trilium Frontend API +- Browser APIs (DOM, localStorage, etc.) +- Froca (frontend cache) +- UI widgets +- No direct file system access + +**Lifecycle:** +- `frontendStartup` - Run once when Trilium loads +- `frontendReload` - Run on every note context change + +**Example:** +```javascript +// Attach to note with #run=frontendStartup +const api = window.api + +// Add custom button to toolbar +api.addButtonToToolbar({ + title: 'My Button', + icon: 'star', + action: () => { + api.showMessage('Hello from frontend!') + } +}) +``` + +### Backend Scripts + +**Location:** Attached to notes with `#run=backendStartup` attribute + +**Execution Context:** Node.js server environment + +**Access:** +- Trilium Backend API +- Node.js APIs (fs, http, etc.) +- Becca (backend cache) +- Database (SQL) +- External libraries (via require) + +**Lifecycle:** +- `backendStartup` - Run once when server starts +- Event handlers (custom events) + +**Example:** +```javascript +// Attach to note with #run=backendStartup +const api = require('@triliumnext/api') + +// Listen for note creation +api.dayjs // Example: access dayjs library + +api.onNoteCreated((note) => { + if (note.title.includes('TODO')) { + note.setLabel('priority', 'high') + } +}) +``` + +### Render Scripts + +**Location:** Attached to notes with `#customWidget` or similar attributes + +**Purpose:** Custom note rendering/widgets + +**Example:** +```javascript +// Custom widget for a note +class MyWidget extends api.NoteContextAwareWidget { + doRender() { + this.$widget = $('
') + .text('Custom widget content') + return this.$widget + } +} + +module.exports = MyWidget +``` + +## Script API + +### Frontend API + +**Location:** `apps/client/src/services/frontend_script_api.ts` + +**Global Access:** `window.api` + +**Key Methods:** + +```typescript +// Note Operations +api.getNote(noteId) // Get note object +api.getBranch(branchId) // Get branch object +api.getActiveNote() // Currently displayed note +api.openNote(noteId, activateNote) // Open note in UI + +// UI Operations +api.showMessage(message) // Show toast notification +api.showDialog() // Show modal dialog +api.confirm(message) // Show confirmation dialog +api.prompt(message, defaultValue) // Show input prompt + +// Tree Operations +api.getTree() // Get note tree structure +api.expandTree(noteId) // Expand tree branch +api.collapseTree(noteId) // Collapse tree branch + +// Search +api.searchForNotes(searchQuery) // Search notes +api.searchForNote(searchQuery) // Get single note + +// Navigation +api.openTabWithNote(noteId) // Open note in new tab +api.closeActiveTab() // Close current tab +api.activateNote(noteId) // Switch to note + +// Attributes +api.getAttribute(noteId, type, name) // Get attribute +api.getAttributes(noteId, type, name) // Get all matching attributes + +// Custom Widgets +api.addButtonToToolbar(def) // Add toolbar button +api.addCustomWidget(def) // Add custom widget + +// Events +api.runOnNoteOpened(callback) // Note opened event +api.runOnNoteContentChange(callback) // Content changed event + +// Utilities +api.dayjs // Date/time library +api.formatDate(date) // Format date +api.log(message) // Console log +``` + +### Backend API + +**Location:** `apps/server/src/services/backend_script_api.ts` + +**Access:** `require('@triliumnext/api')` or global `api` + +**Key Methods:** + +```typescript +// Note Operations +api.getNote(noteId) // Get note from Becca +api.getNoteWithContent(noteId) // Get note with content +api.createNote(parentNoteId, title) // Create new note +api.deleteNote(noteId) // Delete note + +// Branch Operations +api.getBranch(branchId) // Get branch +api.createBranch(noteId, parentNoteId) // Create branch (clone) + +// Attribute Operations +api.getAttribute(noteId, type, name) // Get attribute +api.createAttribute(noteId, type, name, value) // Create attribute + +// Database Access +api.sql.getRow(query, params) // Execute SQL query (single row) +api.sql.getRows(query, params) // Execute SQL query (multiple rows) +api.sql.execute(query, params) // Execute SQL statement + +// Events +api.onNoteCreated(callback) // Note created event +api.onNoteUpdated(callback) // Note updated event +api.onNoteDeleted(callback) // Note deleted event +api.onAttributeCreated(callback) // Attribute created event + +// Search +api.searchForNotes(searchQuery) // Search notes + +// Date/Time +api.dayjs // Date/time library +api.now() // Current date/time + +// Logging +api.log(message) // Log message +api.error(message) // Log error + +// External Communication +api.axios // HTTP client library + +// Utilities +api.backup.backupNow() // Trigger backup +api.export.exportSubtree(noteId) // Export notes +``` + +## Script Attributes + +### Execute Attributes + +- `#run=frontendStartup` - Execute on frontend startup +- `#run=backendStartup` - Execute on backend startup +- `#run=hourly` - Execute every hour +- `#run=daily` - Execute daily + +### Widget Attributes + +- `#customWidget` - Custom note widget +- `#widget` - Standard widget integration + +### Other Attributes + +- `#disableVersioning` - Disable automatic versioning for this note +- `#hideChildrenOverview` - Hide children in overview +- `#iconClass` - Custom icon for note + +## Entity Classes + +### Frontend Entities + +**FNote** (`apps/client/src/entities/fnote.ts`) + +```typescript +class FNote { + noteId: string + title: string + type: string + mime: string + + // Relationships + getParentNotes(): FNote[] + getChildNotes(): FNote[] + getBranches(): FBranch[] + + // Attributes + getAttribute(type, name): FAttribute + getAttributes(type?, name?): FAttribute[] + hasLabel(name): boolean + getLabelValue(name): string + + // Content + getContent(): Promise + + // Navigation + open(): void +} +``` + +**FBranch** + +```typescript +class FBranch { + branchId: string + noteId: string + parentNoteId: string + prefix: string + notePosition: number + + getNote(): FNote + getParentNote(): FNote +} +``` + +**FAttribute** + +```typescript +class FAttribute { + attributeId: string + noteId: string + type: 'label' | 'relation' + name: string + value: string + + getNote(): FNote + getTargetNote(): FNote // For relations +} +``` + +### Backend Entities + +**BNote** (`apps/server/src/becca/entities/bnote.ts`) + +```typescript +class BNote { + noteId: string + title: string + type: string + mime: string + isProtected: boolean + + // Content + getContent(): string | Buffer + setContent(content: string | Buffer): void + + // Relationships + getParentNotes(): BNote[] + getChildNotes(): BNote[] + getBranches(): BBranch[] + + // Attributes + getAttribute(type, name): BAttribute + getAttributes(type?, name?): BAttribute[] + setLabel(name, value): BAttribute + setRelation(name, targetNoteId): BAttribute + hasLabel(name): boolean + getLabelValue(name): string + + // Operations + save(): void + markAsDeleted(): void +} +``` + +**BBranch** + +```typescript +class BBranch { + branchId: string + noteId: string + parentNoteId: string + prefix: string + notePosition: number + + getNote(): BNote + getParentNote(): BNote + save(): void +} +``` + +**BAttribute** + +```typescript +class BAttribute { + attributeId: string + noteId: string + type: 'label' | 'relation' + name: string + value: string + + getNote(): BNote + getTargetNote(): BNote // For relations + save(): void +} +``` + +## Script Examples + +### Frontend Examples + +**1. Custom Toolbar Button** + +```javascript +// #run=frontendStartup +api.addButtonToToolbar({ + title: 'Export to PDF', + icon: 'file-export', + action: async () => { + const note = api.getActiveNote() + if (note) { + await api.runOnBackend('exportToPdf', [note.noteId]) + api.showMessage('Export started') + } + } +}) +``` + +**2. Auto-Save Reminder** + +```javascript +// #run=frontendStartup +let saveTimer +api.runOnNoteContentChange(() => { + clearTimeout(saveTimer) + saveTimer = setTimeout(() => { + api.showMessage('Remember to save your work!') + }, 300000) // 5 minutes +}) +``` + +**3. Note Statistics Widget** + +```javascript +// #customWidget +class StatsWidget extends api.NoteContextAwareWidget { + doRender() { + this.$widget = $('
') + return this.$widget + } + + async refreshWithNote(note) { + const content = await note.getContent() + const words = content.split(/\s+/).length + const chars = content.length + + this.$widget.html(` +
Words: ${words}
+
Characters: ${chars}
+ `) + } +} + +module.exports = StatsWidget +``` + +### Backend Examples + +**1. Auto-Tagging on Note Creation** + +```javascript +// #run=backendStartup +api.onNoteCreated((note) => { + // Auto-tag TODO notes + if (note.title.includes('TODO')) { + note.setLabel('type', 'todo') + note.setLabel('priority', 'normal') + } + + // Auto-tag meeting notes by date + if (note.title.match(/Meeting \d{4}-\d{2}-\d{2}/)) { + note.setLabel('type', 'meeting') + const dateMatch = note.title.match(/(\d{4}-\d{2}-\d{2})/) + if (dateMatch) { + note.setLabel('date', dateMatch[1]) + } + } +}) +``` + +**2. Daily Backup Reminder** + +```javascript +// #run=daily +const todayNote = api.getTodayNote() +todayNote.setLabel('backupDone', 'false') + +// Create reminder note +api.createNote(todayNote.noteId, '🔔 Backup Reminder', { + content: 'Remember to verify today\'s backup!', + type: 'text' +}) +``` + +**3. External API Integration** + +```javascript +// #run=backendStartup +api.onNoteCreated(async (note) => { + // Sync new notes to external service + if (note.hasLabel('sync-external')) { + try { + await api.axios.post('https://external-api.com/sync', { + noteId: note.noteId, + title: note.title, + content: note.getContent() + }) + note.setLabel('lastSync', api.dayjs().format()) + } catch (error) { + api.log('Sync failed: ' + error.message) + } + } +}) +``` + +**4. Database Cleanup** + +```javascript +// #run=weekly +// Clean up old revisions +const cutoffDate = api.dayjs().subtract(90, 'days').format() + +const oldRevisions = api.sql.getRows(` + SELECT revisionId FROM revisions + WHERE utcDateCreated < ? +`, [cutoffDate]) + +api.log(`Deleting ${oldRevisions.length} old revisions`) + +for (const row of oldRevisions) { + api.sql.execute('DELETE FROM revisions WHERE revisionId = ?', [row.revisionId]) +} +``` + +## Script Storage + +**Storage Location:** Scripts are stored as regular notes + +**Identifying Scripts:** +- Have `#run` attribute or `#customWidget` attribute +- Type is typically `code` with MIME `application/javascript` + +**Script Note Structure:** +``` +📁 Scripts (folder note) +├── 📜 Frontend Scripts +│ ├── Custom Toolbar Button (#run=frontendStartup) +│ └── Statistics Widget (#customWidget) +└── 📜 Backend Scripts + ├── Auto-Tagger (#run=backendStartup) + └── Daily Backup (#run=daily) +``` + +## Script Execution + +### Frontend Execution + +**Timing:** +1. Trilium frontend loads +2. Froca cache initializes +3. Script notes with `#run=frontendStartup` are found +4. Scripts execute in dependency order + +**Isolation:** +- Each script runs in separate context +- Shared `window.api` object +- Can access global window object + +### Backend Execution + +**Timing:** +1. Server starts +2. Becca cache loads +3. Script notes with `#run=backendStartup` are found +4. Scripts execute in dependency order + +**Isolation:** +- Each script is a separate module +- Can require Node.js modules +- Shared `api` global + +### Error Handling + +**Frontend:** +```javascript +try { + // Script code +} catch (error) { + api.showError('Script error: ' + error.message) + console.error(error) +} +``` + +**Backend:** +```javascript +try { + // Script code +} catch (error) { + api.log('Script error: ' + error.message) + console.error(error) +} +``` + +## Security Considerations + +### Frontend Scripts + +**Risks:** +- Can access all notes via Froca +- Can manipulate DOM +- Can make API calls +- Limited by browser security model + +**Mitigations:** +- User must trust scripts they add +- Scripts run with user privileges +- No access to file system + +### Backend Scripts + +**Risks:** +- Full Node.js access +- Can execute system commands +- Can access file system +- Can make network requests + +**Mitigations:** +- Scripts are user-created (trusted) +- Single-user model (no privilege escalation) +- Review scripts before adding `#run` attribute + +### Best Practices + +1. **Review script code** before adding execution attributes +2. **Use specific attributes** rather than wildcard searches +3. **Avoid eval()** and dynamic code execution +4. **Validate inputs** in scripts +5. **Handle errors** gracefully +6. **Log important actions** for audit trail + +## Performance Considerations + +### Optimization Tips + +**1. Cache Results:** +```javascript +// Bad: Re-query on every call +function getConfig() { + return api.getNote('config').getContent() +} + +// Good: Cache the result +let cachedConfig +function getConfig() { + if (!cachedConfig) { + cachedConfig = api.getNote('config').getContent() + } + return cachedConfig +} +``` + +**2. Use Efficient Queries:** +```javascript +// Bad: Load all notes and filter +const todos = api.searchForNotes('#type=todo') + +// Good: Use specific search +const todos = api.searchForNotes('#type=todo #status=pending') +``` + +**3. Batch Operations:** +```javascript +// Bad: Save after each change +notes.forEach(note => { + note.title = 'Updated' + note.save() +}) + +// Good: Batch changes +notes.forEach(note => { + note.title = 'Updated' +}) +// Save happens in batch +``` + +**4. Debounce Event Handlers:** +```javascript +let timeout +api.runOnNoteContentChange(() => { + clearTimeout(timeout) + timeout = setTimeout(() => { + // Process change + }, 500) +}) +``` + +## Debugging Scripts + +### Frontend Debugging + +**Browser DevTools:** +```javascript +console.log('Debug info:', data) +debugger // Breakpoint +``` + +**Trilium Log:** +```javascript +api.log('Script executed') +``` + +### Backend Debugging + +**Console Output:** +```javascript +console.log('Backend debug:', data) +api.log('Script log message') +``` + +**Inspect Becca:** +```javascript +api.log('Note count:', Object.keys(api.becca.notes).length) +``` + +## Advanced Topics + +### Custom Note Types + +Scripts can implement custom note type handlers: + +```javascript +// Register custom type +api.registerNoteType({ + type: 'mytype', + mime: 'application/x-mytype', + renderNote: (note) => { + // Custom rendering + } +}) +``` + +### External Libraries + +**Frontend:** +```javascript +// Load external library +const myLib = await import('https://cdn.example.com/lib.js') +``` + +**Backend:** +```javascript +// Use Node.js require +const fs = require('fs') +const axios = require('axios') +``` + +### State Persistence + +**Frontend:** +```javascript +// Use localStorage +localStorage.setItem('myScript:data', JSON.stringify(data)) +const data = JSON.parse(localStorage.getItem('myScript:data')) +``` + +**Backend:** +```javascript +// Store in special note +const stateNote = api.getNote('script-state-note') +stateNote.setContent(JSON.stringify(data)) + +const data = JSON.parse(stateNote.getContent()) +``` + +--- + +**See Also:** +- [Script API Documentation](Script%20API/) - Complete API reference +- [Advanced Showcases](https://triliumnext.github.io/Docs/Wiki/advanced-showcases) - Example scripts +- [ARCHITECTURE.md](ARCHITECTURE.md) - Overall architecture diff --git a/docs/SECURITY_ARCHITECTURE.md b/docs/SECURITY_ARCHITECTURE.md new file mode 100644 index 000000000..27993deac --- /dev/null +++ b/docs/SECURITY_ARCHITECTURE.md @@ -0,0 +1,834 @@ +# Trilium Security Architecture + +> **Related:** [ARCHITECTURE.md](ARCHITECTURE.md) | [SECURITY.md](../SECURITY.md) + +## Overview + +Trilium implements a **defense-in-depth security model** with multiple layers of protection for user data. The security architecture covers authentication, authorization, encryption, input sanitization, and secure communication. + +## Security Principles + +1. **Data Privacy**: User data is protected at rest and in transit +2. **Encryption**: Per-note encryption for sensitive content +3. **Authentication**: Multiple authentication methods supported +4. **Authorization**: Single-user model with granular protected sessions +5. **Input Validation**: All user input sanitized +6. **Secure Defaults**: Security features enabled by default +7. **Transparency**: Open source allows security audits + +## Threat Model + +### Threats Considered + +1. **Unauthorized Access** + - Physical access to device + - Network eavesdropping + - Stolen credentials + - Session hijacking + +2. **Data Exfiltration** + - Malicious scripts + - XSS attacks + - SQL injection + - CSRF attacks + +3. **Data Corruption** + - Malicious modifications + - Database tampering + - Sync conflicts + +4. **Privacy Leaks** + - Unencrypted backups + - Search indexing + - Temporary files + - Memory dumps + +### Out of Scope + +- Nation-state attackers +- Zero-day vulnerabilities in dependencies +- Hardware vulnerabilities (Spectre, Meltdown) +- Physical access with unlimited time +- Quantum computing attacks + +## Authentication + +### Password Authentication + +**Implementation:** `apps/server/src/services/password.ts` + +**Password Storage:** +```typescript +// Password is never stored directly +const salt = crypto.randomBytes(32) +const derivedKey = crypto.pbkdf2Sync(password, salt, 10000, 32, 'sha256') +const verificationHash = crypto.createHash('sha256') + .update(derivedKey) + .digest('hex') + +// Store only salt and verification hash +sql.insert('user_data', { + salt: salt.toString('hex'), + derivedKey: derivedKey.toString('hex') // Used for encryption +}) + +sql.insert('options', { + name: 'passwordVerificationHash', + value: verificationHash +}) +``` + +**Password Requirements:** +- Minimum length: 4 characters (configurable) +- No maximum length +- All characters allowed +- Can be changed by user + +**Login Process:** +```typescript +// 1. User submits password +POST /api/login/password +Body: { password: "user-password" } + +// 2. Server derives key +const derivedKey = crypto.pbkdf2Sync(password, salt, 10000, 32, 'sha256') + +// 3. Verify against stored hash +const verificationHash = crypto.createHash('sha256') + .update(derivedKey) + .digest('hex') + +if (verificationHash === storedHash) { + // 4. Create session + req.session.loggedIn = true + req.session.regenerate() +} +``` + +### TOTP (Two-Factor Authentication) + +**Implementation:** `apps/server/src/routes/api/login.ts` + +**Setup Process:** +```typescript +// 1. Generate secret +const secret = speakeasy.generateSecret({ + name: `Trilium (${username})`, + length: 32 +}) + +// 2. Store encrypted secret +const encryptedSecret = encrypt(secret.base32, dataKey) +sql.insert('options', { + name: 'totpSecret', + value: encryptedSecret +}) + +// 3. Generate QR code +const qrCodeUrl = secret.otpauth_url +``` + +**Verification:** +```typescript +// User submits TOTP token +POST /api/login/totp +Body: { token: "123456" } + +// Verify token +const secret = decrypt(encryptedSecret, dataKey) +const verified = speakeasy.totp.verify({ + secret: secret, + encoding: 'base32', + token: token, + window: 1 // Allow 1 time step tolerance +}) +``` + +### OpenID Connect + +**Implementation:** `apps/server/src/routes/api/login.ts` + +**Supported Providers:** +- Any OpenID Connect compatible provider +- Google, GitHub, Auth0, etc. + +**Flow:** +```typescript +// 1. Redirect to provider +GET /api/login/openid + +// 2. Provider redirects back with code +GET /api/login/openid/callback?code=... + +// 3. Exchange code for tokens +const tokens = await openidClient.callback(redirectUri, req.query) + +// 4. Verify ID token +const claims = tokens.claims() + +// 5. Create session +req.session.loggedIn = true +``` + +### Session Management + +**Session Storage:** SQLite database (sessions table) + +**Session Configuration:** +```typescript +app.use(session({ + secret: sessionSecret, + resave: false, + saveUninitialized: false, + rolling: true, + cookie: { + maxAge: 7 * 24 * 60 * 60 * 1000, // 7 days + httpOnly: true, + secure: isHttps, + sameSite: 'lax' + }, + store: new SqliteStore({ + db: db, + table: 'sessions' + }) +})) +``` + +**Session Invalidation:** +- Automatic timeout after inactivity +- Manual logout clears session +- Server restart invalidates all sessions (optional) + +## Authorization + +### Single-User Model + +**Desktop:** +- Single user (owner of device) +- No multi-user support +- Full access to all notes + +**Server:** +- Single user per installation +- Authentication required for all operations +- No user roles or permissions + +### Protected Sessions + +**Purpose:** Temporary access to encrypted (protected) notes + +**Implementation:** `apps/server/src/services/protected_session.ts` + +**Workflow:** +```typescript +// 1. User enters password for protected notes +POST /api/protected-session/enter +Body: { password: "protected-password" } + +// 2. Derive encryption key +const protectedDataKey = deriveKey(password) + +// 3. Verify password (decrypt known encrypted value) +const decrypted = decrypt(testValue, protectedDataKey) +if (decrypted === expectedValue) { + // 4. Store in memory (not in session) + protectedSessionHolder.setProtectedDataKey(protectedDataKey) + + // 5. Set timeout + setTimeout(() => { + protectedSessionHolder.clearProtectedDataKey() + }, timeout) +} +``` + +**Protected Session Timeout:** +- Default: 10 minutes (configurable) +- Extends on activity +- Cleared on browser close +- Separate from main session + +### API Authorization + +**Internal API:** +- Requires authenticated session +- CSRF token validation +- Same-origin policy + +**ETAPI (External API):** +- Token-based authentication +- No session required +- Rate limiting + +## Encryption + +### Note Encryption + +**Encryption Algorithm:** AES-256-CBC + +**Key Hierarchy:** +``` +User Password + ↓ (PBKDF2) +Data Key (for protected notes) + ↓ (AES-256) +Protected Note Content +``` + +**Encryption Process:** +```typescript +// 1. Generate IV (initialization vector) +const iv = crypto.randomBytes(16) + +// 2. Encrypt content +const cipher = crypto.createCipheriv('aes-256-cbc', dataKey, iv) +let encrypted = cipher.update(content, 'utf8', 'base64') +encrypted += cipher.final('base64') + +// 3. Prepend IV to encrypted content +const encryptedBlob = iv.toString('base64') + ':' + encrypted + +// 4. Store in database +sql.insert('blobs', { + blobId: blobId, + content: encryptedBlob +}) +``` + +**Decryption Process:** +```typescript +// 1. Split IV and encrypted content +const [ivBase64, encryptedData] = encryptedBlob.split(':') +const iv = Buffer.from(ivBase64, 'base64') + +// 2. Decrypt +const decipher = crypto.createDecipheriv('aes-256-cbc', dataKey, iv) +let decrypted = decipher.update(encryptedData, 'base64', 'utf8') +decrypted += decipher.final('utf8') + +return decrypted +``` + +**Protected Note Metadata:** +- Title is NOT encrypted (for tree display) +- Type and MIME are NOT encrypted +- Content IS encrypted +- Attributes CAN be encrypted (optional) + +### Data Key Management + +**Master Data Key:** +```typescript +// Generated once during setup +const dataKey = crypto.randomBytes(32) // 256 bits + +// Encrypted with derived key from user password +const derivedKey = crypto.pbkdf2Sync(password, salt, 10000, 32, 'sha256') +const encryptedDataKey = encrypt(dataKey, derivedKey) + +// Stored in database +sql.insert('options', { + name: 'encryptedDataKey', + value: encryptedDataKey.toString('hex') +}) +``` + +**Key Rotation:** +- Not currently supported +- Requires re-encrypting all protected notes +- Planned for future version + +### Transport Encryption + +**HTTPS:** +- Required for server installations (recommended) +- TLS 1.2+ only +- Strong cipher suites preferred +- Certificate validation enabled + +**Desktop:** +- Local communication (no network) +- No HTTPS required + +### Backup Encryption + +**Database Backups:** +- Protected notes remain encrypted in backup +- Backup file should be protected separately +- Consider encrypting backup storage location + +## Input Sanitization + +### XSS Prevention + +**HTML Sanitization:** + +Location: `apps/client/src/services/dompurify.ts` + +```typescript +import DOMPurify from 'dompurify' + +// Configure DOMPurify +DOMPurify.setConfig({ + ALLOWED_TAGS: ['b', 'i', 'em', 'strong', 'a', 'p', 'br', 'div', ...], + ALLOWED_ATTR: ['href', 'title', 'class', 'id', ...], + ALLOW_DATA_ATTR: false +}) + +// Sanitize HTML before rendering +const cleanHtml = DOMPurify.sanitize(userHtml) +``` + +**CKEditor Configuration:** +```typescript +// apps/client/src/widgets/type_widgets/text_type_widget.ts +ClassicEditor.create(element, { + // Restrict allowed content + htmlSupport: { + allow: [ + { name: /./, attributes: true, classes: true, styles: true } + ], + disallow: [ + { name: 'script' }, + { name: 'iframe', attributes: /^(?!src$).*/ } + ] + } +}) +``` + +**Content Security Policy:** +```typescript +// apps/server/src/main.ts +app.use((req, res, next) => { + res.setHeader('Content-Security-Policy', + "default-src 'self'; " + + "script-src 'self' 'unsafe-inline' 'unsafe-eval'; " + + "style-src 'self' 'unsafe-inline'; " + + "img-src 'self' data: blob:;" + ) + next() +}) +``` + +### SQL Injection Prevention + +**Parameterized Queries:** +```typescript +// GOOD - Safe from SQL injection +const notes = sql.getRows( + 'SELECT * FROM notes WHERE title = ?', + [userInput] +) + +// BAD - Vulnerable to SQL injection +const notes = sql.getRows( + `SELECT * FROM notes WHERE title = '${userInput}'` +) +``` + +**ORM Usage:** +```typescript +// Entity-based access prevents SQL injection +const note = becca.getNote(noteId) +note.title = userInput // Sanitized by entity +note.save() // Parameterized query +``` + +### CSRF Prevention + +**CSRF Token Validation:** + +Location: `apps/server/src/routes/middleware/csrf.ts` + +```typescript +// Generate CSRF token +const csrfToken = crypto.randomBytes(32).toString('hex') +req.session.csrfToken = csrfToken + +// Validate on state-changing requests +app.use((req, res, next) => { + if (['POST', 'PUT', 'DELETE'].includes(req.method)) { + const token = req.headers['x-csrf-token'] + if (token !== req.session.csrfToken) { + return res.status(403).json({ error: 'CSRF token mismatch' }) + } + } + next() +}) +``` + +**Client-Side:** +```typescript +// apps/client/src/services/server.ts +const csrfToken = getCsrfToken() + +fetch('/api/notes', { + method: 'POST', + headers: { + 'X-CSRF-Token': csrfToken, + 'Content-Type': 'application/json' + }, + body: JSON.stringify(data) +}) +``` + +### File Upload Validation + +**Validation:** +```typescript +// apps/server/src/routes/api/attachments.ts +const allowedMimeTypes = [ + 'image/jpeg', + 'image/png', + 'application/pdf', + // ... +] + +if (!allowedMimeTypes.includes(file.mimetype)) { + throw new Error('File type not allowed') +} + +// Validate file size +const maxSize = 100 * 1024 * 1024 // 100 MB +if (file.size > maxSize) { + throw new Error('File too large') +} + +// Sanitize filename +const sanitizedFilename = path.basename(file.originalname) + .replace(/[^a-z0-9.-]/gi, '_') +``` + +## Network Security + +### HTTPS Configuration + +**Server Setup:** +```typescript +// apps/server/src/main.ts +const httpsOptions = { + key: fs.readFileSync('server.key'), + cert: fs.readFileSync('server.cert') +} + +https.createServer(httpsOptions, app).listen(443) +``` + +**Certificate Validation:** +- Require valid certificates in production +- Self-signed certificates allowed for development +- Certificate pinning not implemented + +### Secure Headers + +```typescript +// apps/server/src/main.ts +app.use((req, res, next) => { + // Prevent clickjacking + res.setHeader('X-Frame-Options', 'SAMEORIGIN') + + // Prevent MIME sniffing + res.setHeader('X-Content-Type-Options', 'nosniff') + + // XSS protection + res.setHeader('X-XSS-Protection', '1; mode=block') + + // Referrer policy + res.setHeader('Referrer-Policy', 'same-origin') + + // HTTPS upgrade + if (req.secure) { + res.setHeader('Strict-Transport-Security', 'max-age=31536000') + } + + next() +}) +``` + +### Rate Limiting + +**API Rate Limiting:** +```typescript +// apps/server/src/routes/middleware/rate_limit.ts +const rateLimit = require('express-rate-limit') + +const apiLimiter = rateLimit({ + windowMs: 15 * 60 * 1000, // 15 minutes + max: 1000, // Limit each IP to 1000 requests per window + message: 'Too many requests from this IP' +}) + +app.use('/api/', apiLimiter) +``` + +**Login Rate Limiting:** +```typescript +const loginLimiter = rateLimit({ + windowMs: 15 * 60 * 1000, + max: 5, // 5 failed attempts + skipSuccessfulRequests: true +}) + +app.post('/api/login/password', loginLimiter, loginHandler) +``` + +## Data Security + +### Secure Data Deletion + +**Soft Delete:** +```typescript +// Mark as deleted (sync first) +note.isDeleted = 1 +note.deleteId = generateUUID() +note.save() + +// Entity change tracked for sync +addEntityChange('notes', noteId, note) +``` + +**Hard Delete (Erase):** +```typescript +// After sync completed +sql.execute('DELETE FROM notes WHERE noteId = ?', [noteId]) +sql.execute('DELETE FROM branches WHERE noteId = ?', [noteId]) +sql.execute('DELETE FROM attributes WHERE noteId = ?', [noteId]) + +// Mark entity change as erased +sql.execute('UPDATE entity_changes SET isErased = 1 WHERE entityId = ?', [noteId]) +``` + +**Blob Cleanup:** +```typescript +// Find orphaned blobs (not referenced by any note/revision/attachment) +const orphanedBlobs = sql.getRows(` + SELECT blobId FROM blobs + WHERE blobId NOT IN (SELECT blobId FROM notes WHERE blobId IS NOT NULL) + AND blobId NOT IN (SELECT blobId FROM revisions WHERE blobId IS NOT NULL) + AND blobId NOT IN (SELECT blobId FROM attachments WHERE blobId IS NOT NULL) +`) + +// Delete orphaned blobs +for (const blob of orphanedBlobs) { + sql.execute('DELETE FROM blobs WHERE blobId = ?', [blob.blobId]) +} +``` + +### Memory Security + +**Protected Data in Memory:** +- Protected data keys stored in memory only +- Cleared on timeout +- Not written to disk +- Not in session storage + +**Memory Cleanup:** +```typescript +// Clear sensitive data +const clearSensitiveData = () => { + protectedDataKey = null + + // Force garbage collection if available + if (global.gc) { + global.gc() + } +} +``` + +### Temporary Files + +**Secure Temporary Files:** +```typescript +const tempDir = os.tmpdir() +const tempFile = path.join(tempDir, `trilium-${crypto.randomBytes(16).toString('hex')}`) + +// Write temp file +fs.writeFileSync(tempFile, data, { mode: 0o600 }) // Owner read/write only + +// Clean up after use +fs.unlinkSync(tempFile) +``` + +## Dependency Security + +### Vulnerability Scanning + +**Tools:** +- `npm audit` - Check for known vulnerabilities +- Renovate bot - Automatic dependency updates +- GitHub Dependabot alerts + +**Process:** +```bash +# Check for vulnerabilities +npm audit + +# Fix automatically +npm audit fix + +# Manual review for breaking changes +npm audit fix --force +``` + +### Dependency Pinning + +**package.json:** +```json +{ + "dependencies": { + "express": "4.18.2", // Exact version + "better-sqlite3": "^9.2.2" // Compatible versions + } +} +``` + +**pnpm Overrides:** +```json +{ + "pnpm": { + "overrides": { + "lodash@<4.17.21": ">=4.17.21", // Force minimum version + "axios@<0.21.2": ">=0.21.2" + } + } +} +``` + +### Patch Management + +**pnpm Patches:** +```bash +# Create patch +pnpm patch @ckeditor/ckeditor5 + +# Edit files in temporary directory +# ... + +# Generate patch file +pnpm patch-commit /tmp/ckeditor5-patch + +# Patch applied automatically on install +``` + +## Security Best Practices + +### For Users + +1. **Strong Passwords** + - Use unique password for Trilium + - Enable TOTP 2FA + - Protect password manager + +2. **Protected Notes** + - Use for sensitive information + - Set reasonable session timeout + - Don't leave sessions unattended + +3. **Backups** + - Regular backups to secure location + - Encrypt backup storage + - Test backup restoration + +4. **Server Setup** + - Use HTTPS only + - Keep software updated + - Firewall configuration + - Use reverse proxy (nginx, Caddy) + +5. **Scripts** + - Review scripts before using + - Be cautious with external scripts + - Understand script permissions + +### For Developers + +1. **Code Review** + - Review all security-related changes + - Test authentication/authorization changes + - Validate input sanitization + +2. **Testing** + - Write security tests + - Test edge cases + - Penetration testing + +3. **Dependencies** + - Regular updates + - Audit new dependencies + - Monitor security advisories + +4. **Secrets** + - No secrets in source code + - Use environment variables + - Secure key generation + +## Security Auditing + +### Logs + +**Security Events Logged:** +- Login attempts (success/failure) +- Protected session access +- Password changes +- ETAPI token usage +- Failed CSRF validations + +**Log Location:** +- Desktop: Console output +- Server: Log files or stdout + +### Monitoring + +**Metrics to Monitor:** +- Failed login attempts +- API error rates +- Unusual database changes +- Large exports/imports + +## Incident Response + +### Security Issue Reporting + +**Process:** +1. Email security@triliumnext.com +2. Include vulnerability details +3. Provide reproduction steps +4. Allow reasonable disclosure time + +**Response:** +1. Acknowledge within 48 hours +2. Investigate and validate +3. Develop fix +4. Coordinate disclosure +5. Release patch + +### Breach Response + +**If Compromised:** +1. Change password immediately +2. Review recent activity +3. Check for unauthorized changes +4. Restore from backup if needed +5. Update security settings + +## Future Security Enhancements + +**Planned:** +- Hardware security key support (U2F/WebAuthn) +- End-to-end encryption for sync +- Zero-knowledge architecture option +- Encryption key rotation +- Audit log enhancements +- Per-note access controls + +**Under Consideration:** +- Multi-user support with permissions +- Blockchain-based sync verification +- Homomorphic encryption for search +- Quantum-resistant encryption + +--- + +**See Also:** +- [SECURITY.md](../SECURITY.md) - Security policy +- [ARCHITECTURE.md](ARCHITECTURE.md) - Overall architecture +- [Protected Notes Guide](https://triliumnext.github.io/Docs/Wiki/protected-notes) diff --git a/docs/SYNCHRONIZATION.md b/docs/SYNCHRONIZATION.md new file mode 100644 index 000000000..2f39eee40 --- /dev/null +++ b/docs/SYNCHRONIZATION.md @@ -0,0 +1,583 @@ +# Trilium Synchronization Architecture + +> **Related:** [ARCHITECTURE.md](ARCHITECTURE.md) | [User Guide: Synchronization](https://triliumnext.github.io/Docs/Wiki/synchronization) + +## Overview + +Trilium implements a sophisticated **bidirectional synchronization system** that allows users to sync their note databases across multiple devices (desktop clients and server instances). The sync protocol is designed to handle: + +- Concurrent modifications across devices +- Conflict resolution +- Partial sync (only changed entities) +- Protected note synchronization +- Efficient bandwidth usage + +## Sync Architecture + +``` +┌─────────────┐ ┌─────────────┐ +│ Desktop 1 │ │ Desktop 2 │ +│ (Client) │ │ (Client) │ +└──────┬──────┘ └──────┬──────┘ + │ │ + │ WebSocket/HTTP │ + │ │ + ▼ ▼ +┌────────────────────────────────────────────────┐ +│ Sync Server │ +│ ┌──────────────────────────────────────┐ │ +│ │ Sync Service │ │ +│ │ - Entity Change Management │ │ +│ │ - Conflict Resolution │ │ +│ │ - Version Tracking │ │ +│ └──────────────────────────────────────┘ │ +│ │ │ +│ ┌──────┴───────┐ │ +│ │ Database │ │ +│ │ (entity_changes)│ │ +│ └──────────────┘ │ +└────────────────────────────────────────────────┘ +``` + +## Core Concepts + +### Entity Changes + +Every modification to any entity (note, branch, attribute, etc.) creates an **entity change** record: + +```sql +entity_changes ( + id, -- Auto-increment ID + entityName, -- 'notes', 'branches', 'attributes', etc. + entityId, -- ID of the changed entity + hash, -- Content hash for integrity + isErased, -- If entity was erased (deleted permanently) + changeId, -- Unique change identifier + componentId, -- Installation identifier + instanceId, -- Process instance identifier + isSynced, -- Whether synced to server + utcDateChanged -- When change occurred +) +``` + +**Key Properties:** +- **changeId**: Globally unique identifier (UUID) for the change +- **componentId**: Unique per Trilium installation (persists across restarts) +- **instanceId**: Unique per process (changes on restart) +- **hash**: SHA-256 hash of entity data for integrity verification + +### Sync Versions + +Each Trilium installation tracks: +- **Local sync version**: Highest change ID seen locally +- **Server sync version**: Highest change ID on server +- **Entity versions**: Last sync version for each entity type + +### Change Tracking + +**When an entity is modified:** + +```typescript +// apps/server/src/services/entity_changes.ts +function addEntityChange(entityName, entityId, entity) { + const hash = calculateHash(entity) + const changeId = generateUUID() + + sql.insert('entity_changes', { + entityName, + entityId, + hash, + changeId, + componentId: config.componentId, + instanceId: config.instanceId, + isSynced: 0, + utcDateChanged: now() + }) +} +``` + +**Entity modification triggers:** +- Note content update +- Note metadata change +- Branch creation/deletion/reorder +- Attribute addition/removal +- Options modification + +## Sync Protocol + +### Sync Handshake + +**Step 1: Client Initiates Sync** + +```typescript +// Client sends current sync version +POST /api/sync/check +{ + "sourceId": "client-component-id", + "maxChangeId": 12345 +} +``` + +**Step 2: Server Responds with Status** + +```typescript +// Server checks for changes +Response: +{ + "entityChanges": 567, // Changes on server + "maxChangeId": 12890, // Server's max change ID + "outstandingPushCount": 23 // Client changes not yet synced +} +``` + +**Step 3: Decision** + +- If `entityChanges > 0`: Pull changes from server +- If `outstandingPushCount > 0`: Push changes to server +- Both can happen in sequence + +### Pull Sync (Server → Client) + +**Client Requests Changes:** + +```typescript +POST /api/sync/pull +{ + "sourceId": "client-component-id", + "lastSyncedChangeId": 12345 +} +``` + +**Server Responds:** + +```typescript +Response: +{ + "notes": [ + { noteId: "abc", title: "New Note", ... } + ], + "branches": [...], + "attributes": [...], + "revisions": [...], + "attachments": [...], + "entityChanges": [ + { entityName: "notes", entityId: "abc", changeId: "...", ... } + ], + "maxChangeId": 12890 +} +``` + +**Client Processing:** + +1. Apply entity changes to local database +2. Update Froca cache +3. Update local sync version +4. Trigger UI refresh + +### Push Sync (Client → Server) + +**Client Sends Changes:** + +```typescript +POST /api/sync/push +{ + "sourceId": "client-component-id", + "entities": [ + { + "entity": { + "noteId": "xyz", + "title": "Modified Note", + ... + }, + "entityChange": { + "changeId": "change-uuid", + "entityName": "notes", + ... + } + } + ] +} +``` + +**Server Processing:** + +1. Validate changes +2. Check for conflicts +3. Apply changes to database +4. Update Becca cache +5. Mark as synced +6. Broadcast to other connected clients via WebSocket + +**Conflict Detection:** + +```typescript +// Check if entity was modified on server since client's last sync +const serverEntity = becca.getNote(noteId) +const serverLastModified = serverEntity.utcDateModified + +if (serverLastModified > clientSyncVersion) { + // CONFLICT! + resolveConflict(serverEntity, clientEntity) +} +``` + +## Conflict Resolution + +### Conflict Types + +**1. Content Conflict** +- Both client and server modified same note content +- **Resolution**: Last-write-wins based on `utcDateModified` + +**2. Structure Conflict** +- Branch moved/deleted on one side, modified on other +- **Resolution**: Tombstone records, reconciliation + +**3. Attribute Conflict** +- Same attribute modified differently +- **Resolution**: Last-write-wins + +### Conflict Resolution Strategy + +**Last-Write-Wins:** +```typescript +if (clientEntity.utcDateModified > serverEntity.utcDateModified) { + // Client wins, apply client changes + applyClientChange(clientEntity) +} else { + // Server wins, reject client change + // Client will pull server version on next sync +} +``` + +**Tombstone Records:** +- Deleted entities leave tombstone in `entity_changes` +- Prevents re-sync of deleted items +- `isErased = 1` for permanent deletions + +### Protected Notes Sync + +**Challenge:** Encrypted content can't be synced without password + +**Solution:** + +1. **Protected session required**: User must unlock protected notes +2. **Encrypted sync**: Content synced in encrypted form +3. **Hash verification**: Integrity checked without decryption +4. **Lazy decryption**: Only decrypt when accessed + +**Sync Flow:** + +```typescript +// Client side +if (note.isProtected && !protectedSessionHolder.isProtectedSessionAvailable()) { + // Skip protected notes if session not active + continue +} + +// Server side +if (note.isProtected) { + // Sync encrypted blob + // Don't decrypt for sync + syncEncryptedBlob(note.blobId) +} +``` + +## Sync States + +### Connection States + +- **Connected**: WebSocket connection active +- **Disconnected**: No connection to sync server +- **Syncing**: Actively transferring data +- **Conflict**: Sync paused due to conflict + +### Entity Sync States + +Each entity can be in: +- **Synced**: In sync with server +- **Pending**: Local changes not yet pushed +- **Conflict**: Conflicting changes detected + +### UI Indicators + +```typescript +// apps/client/src/widgets/sync_status.ts +class SyncStatusWidget { + showSyncStatus() { + if (isConnected && allSynced) { + showIcon('synced') + } else if (isSyncing) { + showIcon('syncing-spinner') + } else if (hasConflicts) { + showIcon('conflict-warning') + } else { + showIcon('not-synced') + } + } +} +``` + +## Performance Optimizations + +### Incremental Sync + +Only entities changed since last sync are transferred: + +```sql +SELECT * FROM entity_changes +WHERE id > :lastSyncedChangeId +ORDER BY id ASC +LIMIT 1000 +``` + +### Batch Processing + +Changes sent in batches to reduce round trips: + +```typescript +const BATCH_SIZE = 1000 +const changes = getUnsyncedChanges(BATCH_SIZE) +await syncBatch(changes) +``` + +### Hash-Based Change Detection + +```typescript +// Only sync if hash differs +const localHash = calculateHash(localEntity) +const serverHash = getServerHash(entityId) + +if (localHash !== serverHash) { + syncEntity(localEntity) +} +``` + +### Compression + +Large payloads compressed before transmission: + +```typescript +// Server sends compressed response +res.setHeader('Content-Encoding', 'gzip') +res.send(gzip(syncData)) +``` + +## Error Handling + +### Network Errors + +**Retry Strategy:** +```typescript +const RETRY_DELAYS = [1000, 2000, 5000, 10000, 30000] + +async function syncWithRetry(attempt = 0) { + try { + await performSync() + } catch (error) { + if (attempt < RETRY_DELAYS.length) { + setTimeout(() => { + syncWithRetry(attempt + 1) + }, RETRY_DELAYS[attempt]) + } + } +} +``` + +### Sync Integrity Checks + +**Hash Verification:** +```typescript +// Verify entity hash matches +const calculatedHash = calculateHash(entity) +const receivedHash = entityChange.hash + +if (calculatedHash !== receivedHash) { + throw new Error('Hash mismatch - data corruption detected') +} +``` + +**Consistency Checks:** +- Orphaned branches detection +- Missing parent notes +- Invalid entity references +- Circular dependencies + +## Sync Server Configuration + +### Server Setup + +**Required Options:** +```javascript +{ + "syncServerHost": "https://sync.example.com", + "syncServerTimeout": 60000, + "syncProxy": "" // Optional HTTP proxy +} +``` + +**Authentication:** +- Username/password or +- Sync token (generated on server) + +### Client Setup + +**Desktop Client:** +```javascript +// Settings → Sync +{ + "syncServerHost": "https://sync.example.com", + "username": "user@example.com", + "password": "********" +} +``` + +**Test Connection:** +```typescript +POST /api/sync/test +Response: { "success": true } +``` + +## Sync API Endpoints + +Located at: `apps/server/src/routes/api/sync.ts` + +**Endpoints:** + +- `POST /api/sync/check` - Check sync status +- `POST /api/sync/pull` - Pull changes from server +- `POST /api/sync/push` - Push changes to server +- `POST /api/sync/finished` - Mark sync complete +- `POST /api/sync/test` - Test connection +- `GET /api/sync/stats` - Sync statistics + +## WebSocket Sync Updates + +Real-time sync via WebSocket: + +```typescript +// Server broadcasts change to all connected clients +ws.broadcast('entity-change', { + entityName: 'notes', + entityId: 'abc123', + changeId: 'change-uuid', + sourceId: 'originating-component-id' +}) + +// Client receives and applies +ws.on('entity-change', (data) => { + if (data.sourceId !== myComponentId) { + froca.processEntityChange(data) + } +}) +``` + +## Sync Scheduling + +### Automatic Sync + +**Desktop:** +- Sync on startup +- Periodic sync (configurable interval, default: 60s) +- Sync before shutdown + +**Server:** +- Sync on entity modification +- WebSocket push to connected clients + +### Manual Sync + +User can trigger: +- Full sync +- Sync now +- Sync specific subtree + +## Troubleshooting + +### Common Issues + +**Sync stuck:** +```sql +-- Reset sync state +UPDATE entity_changes SET isSynced = 0; +DELETE FROM options WHERE name LIKE 'sync%'; +``` + +**Hash mismatch:** +- Data corruption detected +- Re-sync from backup +- Check database integrity + +**Conflict loop:** +- Manual intervention required +- Export conflicting notes +- Choose winning version +- Re-sync + +### Sync Diagnostics + +**Check sync status:** +```typescript +GET /api/sync/stats +Response: { + "unsyncedChanges": 0, + "lastSyncDate": "2025-11-02T12:00:00Z", + "syncVersion": 12890 +} +``` + +**Entity change log:** +```sql +SELECT * FROM entity_changes +WHERE isSynced = 0 +ORDER BY id DESC; +``` + +## Security Considerations + +### Encrypted Sync + +- Protected notes synced encrypted +- No plain text over network +- Server cannot read protected content + +### Authentication + +- Username/password over HTTPS only +- Sync tokens for token-based auth +- Session cookies with CSRF protection + +### Authorization + +- Users can only sync their own data +- No cross-user sync support +- Sync server validates ownership + +## Performance Metrics + +**Typical Sync Performance:** +- 1000 changes: ~2-5 seconds +- 10000 changes: ~20-50 seconds +- Initial full sync (100k notes): ~5-10 minutes + +**Factors:** +- Network latency +- Database size +- Number of protected notes +- Attachment sizes + +## Future Improvements + +**Planned Enhancements:** +- Differential sync (binary diff) +- Peer-to-peer sync (no central server) +- Multi-server sync +- Partial sync (subtree only) +- Sync over Tor/I2P + +--- + +**See Also:** +- [ARCHITECTURE.md](ARCHITECTURE.md) - Overall architecture +- [Sync User Guide](https://triliumnext.github.io/Docs/Wiki/synchronization) +- [Sync API Source](../apps/server/src/routes/api/sync.ts) diff --git a/docs/TECHNICAL_DOCUMENTATION.md b/docs/TECHNICAL_DOCUMENTATION.md new file mode 100644 index 000000000..445df0530 --- /dev/null +++ b/docs/TECHNICAL_DOCUMENTATION.md @@ -0,0 +1,423 @@ +# Trilium Notes - Technical Documentation Index + +Welcome to the comprehensive technical and architectural documentation for Trilium Notes. This index provides quick access to all technical documentation resources. + +## 📚 Core Architecture Documentation + +### [ARCHITECTURE.md](ARCHITECTURE.md) +**Main technical architecture document** covering the complete system design. + +**Topics Covered:** +- High-level architecture overview +- Monorepo structure and organization +- Core architecture patterns (Becca, Froca, Shaca) +- Entity system and data model +- Widget-based UI architecture +- Frontend and backend architecture +- API architecture (Internal, ETAPI, WebSocket) +- Build system and tooling +- Testing strategy +- Security overview + +**Audience:** Developers, architects, contributors + +--- + +### [DATABASE.md](DATABASE.md) +**Complete database architecture and schema documentation.** + +**Topics Covered:** +- SQLite database structure +- Entity tables (notes, branches, attributes, revisions, attachments, blobs) +- System tables (options, entity_changes, sessions) +- Data relationships and integrity +- Database access patterns +- Migrations and versioning +- Performance optimization +- Backup and maintenance +- Security considerations + +**Audience:** Backend developers, database administrators + +--- + +### [SYNCHRONIZATION.md](SYNCHRONIZATION.md) +**Detailed synchronization protocol and implementation.** + +**Topics Covered:** +- Sync architecture overview +- Entity change tracking +- Sync protocol (handshake, pull, push) +- Conflict resolution strategies +- Protected notes synchronization +- Performance optimizations +- Error handling and retry logic +- Sync server configuration +- WebSocket real-time updates +- Troubleshooting guide + +**Audience:** Advanced users, sync server administrators, contributors + +--- + +### [SCRIPTING.md](SCRIPTING.md) +**Comprehensive guide to the Trilium scripting system.** + +**Topics Covered:** +- Script types (frontend, backend, render) +- Frontend API reference +- Backend API reference +- Entity classes (FNote, BNote, etc.) +- Script examples and patterns +- Script storage and execution +- Security considerations +- Performance optimization +- Debugging techniques +- Advanced topics + +**Audience:** Power users, script developers, plugin creators + +--- + +### [SECURITY_ARCHITECTURE.md](SECURITY_ARCHITECTURE.md) +**In-depth security architecture and implementation.** + +**Topics Covered:** +- Security principles and threat model +- Authentication methods (password, TOTP, OpenID) +- Session management +- Authorization and protected sessions +- Encryption (notes, transport, backups) +- Input sanitization (XSS, SQL injection, CSRF) +- Network security (HTTPS, headers, rate limiting) +- Data security and secure deletion +- Dependency security +- Security best practices +- Incident response + +**Audience:** Security engineers, administrators, auditors + +--- + +## 🔧 Developer Documentation + +### [Developer Guide](Developer%20Guide/Developer%20Guide/) +Collection of developer-focused documentation for contributing to Trilium. + +**Key Documents:** +- [Environment Setup](Developer%20Guide/Developer%20Guide/Environment%20Setup.md) - Setting up development environment +- [Project Structure](Developer%20Guide/Developer%20Guide/Project%20Structure.md) - Monorepo organization +- [Development and Architecture](Developer%20Guide/Developer%20Guide/Development%20and%20architecture/) - Various development topics + +**Topics Include:** +- Local development setup +- Building and deployment +- Adding new note types +- Database schema details +- Internationalization +- Icons and UI customization +- Docker development +- Troubleshooting + +**Audience:** Contributors, developers + +--- + +## 📖 User Documentation + +### [User Guide](User%20Guide/User%20Guide/) +Comprehensive end-user documentation for using Trilium. + +**Key Sections:** +- Installation & Setup +- Basic Concepts and Features +- Note Types +- Advanced Usage +- Synchronization +- Import/Export + +**Audience:** End users, administrators + +--- + +### [Script API](Script%20API/) +Complete API reference for user scripting. + +**Coverage:** +- Frontend API methods +- Backend API methods +- Entity properties and methods +- Event handlers +- Utility functions + +**Audience:** Script developers, power users + +--- + +## 🚀 Quick Start Guides + +### For Users +1. [Installation Guide](User%20Guide/User%20Guide/Installation%20&%20Setup/) - Get Trilium running +2. [Basic Concepts](User%20Guide/User%20Guide/Basic%20Concepts%20and%20Features/) - Learn the fundamentals +3. [Scripting Guide](SCRIPTING.md) - Extend Trilium with scripts + +### For Developers +1. [Environment Setup](Developer%20Guide/Developer%20Guide/Environment%20Setup.md) - Setup development environment +2. [Architecture Overview](ARCHITECTURE.md) - Understand the system +3. [Contributing Guide](../README.md#-contribute) - Start contributing + +### For Administrators +1. [Server Installation](User%20Guide/User%20Guide/Installation%20&%20Setup/Server%20Installation.md) - Deploy Trilium server +2. [Synchronization Setup](SYNCHRONIZATION.md) - Configure sync +3. [Security Best Practices](SECURITY_ARCHITECTURE.md#security-best-practices) - Secure your installation + +--- + +## 🔍 Documentation by Topic + +### Architecture & Design +- [Overall Architecture](ARCHITECTURE.md) +- [Monorepo Structure](ARCHITECTURE.md#monorepo-structure) +- [Three-Layer Cache System](ARCHITECTURE.md#three-layer-cache-system) +- [Entity System](ARCHITECTURE.md#entity-system) +- [Widget-Based UI](ARCHITECTURE.md#widget-based-ui) + +### Data & Storage +- [Database Architecture](DATABASE.md) +- [Entity Tables](DATABASE.md#entity-tables) +- [Data Relationships](DATABASE.md#data-relationships) +- [Blob Storage](DATABASE.md#blobs-table) +- [Database Migrations](DATABASE.md#database-migrations) + +### Synchronization +- [Sync Architecture](SYNCHRONIZATION.md#sync-architecture) +- [Sync Protocol](SYNCHRONIZATION.md#sync-protocol) +- [Conflict Resolution](SYNCHRONIZATION.md#conflict-resolution) +- [Protected Notes Sync](SYNCHRONIZATION.md#protected-notes-sync) +- [WebSocket Sync](SYNCHRONIZATION.md#websocket-sync-updates) + +### Security +- [Authentication](SECURITY_ARCHITECTURE.md#authentication) +- [Encryption](SECURITY_ARCHITECTURE.md#encryption) +- [Input Sanitization](SECURITY_ARCHITECTURE.md#input-sanitization) +- [Network Security](SECURITY_ARCHITECTURE.md#network-security) +- [Security Best Practices](SECURITY_ARCHITECTURE.md#security-best-practices) + +### Scripting & Extensibility +- [Script Types](SCRIPTING.md#script-types) +- [Frontend API](SCRIPTING.md#frontend-api) +- [Backend API](SCRIPTING.md#backend-api) +- [Script Examples](SCRIPTING.md#script-examples) +- [Custom Widgets](SCRIPTING.md#render-scripts) + +### Frontend +- [Client Architecture](ARCHITECTURE.md#frontend-architecture) +- [Widget System](ARCHITECTURE.md#widget-based-ui) +- [Event System](ARCHITECTURE.md#event-system) +- [Froca Cache](ARCHITECTURE.md#2-froca-frontend-cache) +- [UI Components](ARCHITECTURE.md#ui-components) + +### Backend +- [Server Architecture](ARCHITECTURE.md#backend-architecture) +- [Service Layer](ARCHITECTURE.md#service-layer) +- [Route Structure](ARCHITECTURE.md#route-structure) +- [Becca Cache](ARCHITECTURE.md#1-becca-backend-cache) +- [Middleware](ARCHITECTURE.md#middleware) + +### Build & Deploy +- [Build System](ARCHITECTURE.md#build-system) +- [Package Manager](ARCHITECTURE.md#package-manager-pnpm) +- [Build Tools](ARCHITECTURE.md#build-tools) +- [Docker](Developer%20Guide/Developer%20Guide/Development%20and%20architecture/Docker.md) +- [Deployment](Developer%20Guide/Developer%20Guide/Building%20and%20deployment/) + +### Testing +- [Testing Strategy](ARCHITECTURE.md#testing-strategy) +- [Test Organization](ARCHITECTURE.md#test-organization) +- [E2E Testing](ARCHITECTURE.md#e2e-testing) + +--- + +## 📋 Reference Documentation + +### File Locations +``` +trilium/ +├── apps/ +│ ├── client/ # Frontend application +│ ├── server/ # Backend server +│ ├── desktop/ # Electron app +│ └── ... +├── packages/ +│ ├── commons/ # Shared code +│ ├── ckeditor5/ # Rich text editor +│ └── ... +├── docs/ +│ ├── ARCHITECTURE.md # Main architecture doc +│ ├── DATABASE.md # Database documentation +│ ├── SYNCHRONIZATION.md # Sync documentation +│ ├── SCRIPTING.md # Scripting guide +│ ├── SECURITY_ARCHITECTURE.md # Security documentation +│ ├── Developer Guide/ # Developer docs +│ ├── User Guide/ # User docs +│ └── Script API/ # API reference +└── ... +``` + +### Key Source Files +- **Backend Entry:** `apps/server/src/main.ts` +- **Frontend Entry:** `apps/client/src/desktop.ts` / `apps/client/src/index.ts` +- **Becca Cache:** `apps/server/src/becca/becca.ts` +- **Froca Cache:** `apps/client/src/services/froca.ts` +- **Database Schema:** `apps/server/src/assets/db/schema.sql` +- **Backend API:** `apps/server/src/services/backend_script_api.ts` +- **Frontend API:** `apps/client/src/services/frontend_script_api.ts` + +### Important Directories +- **Entities:** `apps/server/src/becca/entities/` +- **Widgets:** `apps/client/src/widgets/` +- **Services:** `apps/server/src/services/` +- **Routes:** `apps/server/src/routes/` +- **Migrations:** `apps/server/src/migrations/` +- **Tests:** Various `*.spec.ts` files throughout + +--- + +## 🎯 Common Tasks + +### Understanding the Codebase +1. Read [ARCHITECTURE.md](ARCHITECTURE.md) for overview +2. Explore [Monorepo Structure](ARCHITECTURE.md#monorepo-structure) +3. Review [Entity System](ARCHITECTURE.md#entity-system) +4. Check [Key Files](ARCHITECTURE.md#key-files-for-understanding-architecture) + +### Adding Features +1. Review relevant architecture documentation +2. Check [Developer Guide](Developer%20Guide/Developer%20Guide/) +3. Follow existing patterns in codebase +4. Write tests +5. Update documentation + +### Debugging Issues +1. Check [Troubleshooting](Developer%20Guide/Developer%20Guide/Troubleshooting/) +2. Review [Database](DATABASE.md) for data issues +3. Check [Synchronization](SYNCHRONIZATION.md) for sync issues +4. Review [Security](SECURITY_ARCHITECTURE.md) for auth issues + +### Performance Optimization +1. [Database Performance](DATABASE.md#performance-optimization) +2. [Cache Optimization](ARCHITECTURE.md#caching-system) +3. [Build Optimization](ARCHITECTURE.md#build-system) +4. [Script Performance](SCRIPTING.md#performance-considerations) + +--- + +## 🔗 External Resources + +### Official Links +- **Website:** https://triliumnotes.org +- **Documentation:** https://docs.triliumnotes.org +- **GitHub:** https://github.com/TriliumNext/Trilium +- **Discussions:** https://github.com/TriliumNext/Trilium/discussions +- **Matrix Chat:** https://matrix.to/#/#triliumnext:matrix.org + +### Community Resources +- **Awesome Trilium:** https://github.com/Nriver/awesome-trilium +- **TriliumRocks:** https://trilium.rocks/ +- **Wiki:** https://triliumnext.github.io/Docs/Wiki/ + +### Related Projects +- **TriliumDroid:** https://github.com/FliegendeWurst/TriliumDroid +- **Web Clipper:** Included in main repository + +--- + +## 📝 Documentation Conventions + +### Document Structure +- Overview section +- Table of contents +- Main content with headings +- Code examples where relevant +- "See Also" references + +### Code Examples +```typescript +// TypeScript examples with comments +const example = 'value' +``` + +```sql +-- SQL examples with formatting +SELECT * FROM notes WHERE noteId = ? +``` + +### Cross-References +- Use relative links: `[text](path/to/file.md)` +- Reference sections: `[text](file.md#section)` +- External links: Full URLs + +### Maintenance +- Review on major releases +- Update for architectural changes +- Add examples for new features +- Keep API references current + +--- + +## 🤝 Contributing to Documentation + +### What to Document +- New features and APIs +- Architecture changes +- Migration guides +- Performance tips +- Security considerations + +### How to Contribute +1. Edit markdown files in `docs/` +2. Follow existing structure and style +3. Include code examples +4. Test links and formatting +5. Submit pull request + +### Documentation Standards +- Clear, concise language +- Complete code examples +- Proper markdown formatting +- Cross-references to related docs +- Updated version numbers + +--- + +## 📅 Version Information + +- **Documentation Version:** 0.99.3 +- **Last Updated:** November 2025 +- **Trilium Version:** 0.99.3+ +- **Next Review:** When major architectural changes occur + +--- + +## 💡 Getting Help + +### For Users +- [User Guide](User%20Guide/User%20Guide/) +- [GitHub Discussions](https://github.com/TriliumNext/Trilium/discussions) +- [Matrix Chat](https://matrix.to/#/#triliumnext:matrix.org) + +### For Developers +- [Developer Guide](Developer%20Guide/Developer%20Guide/) +- [Architecture Docs](ARCHITECTURE.md) +- [GitHub Issues](https://github.com/TriliumNext/Trilium/issues) + +### For Contributors +- [Contributing Guidelines](../README.md#-contribute) +- [Code of Conduct](../CODE_OF_CONDUCT) +- [Developer Setup](Developer%20Guide/Developer%20Guide/Environment%20Setup.md) + +--- + +**Maintained by:** TriliumNext Team +**License:** AGPL-3.0-only +**Repository:** https://github.com/TriliumNext/Trilium