15 Commits

Author SHA1 Message Date
perfectra1n
30da95d75a feat(search): update fulltext search and add stress test improvements
- Modified note_content_fulltext.ts for enhanced search capabilities
- Updated becca_mocking.ts for better test support
- Improved stress-test-populate.ts script
2025-11-16 14:20:46 -08:00
perf3ct
052e28ab1b
feat(search): if the search is empty, return all notes
Some checks are pending
Checks / main (push) Waiting to run
2025-11-04 11:59:41 -08:00
perf3ct
10988095c2
feat(search): get the correct comparison and rice out the fts5 search
Some checks failed
Checks / main (push) Has been cancelled
2025-10-27 14:37:44 -07:00
perf3ct
d074841885
Revert "feat(search): try to get fts search to work in large environments"
This reverts commit 053f722cb8bcdd8c68af252704cb52fa8df0a5f1.
2025-09-02 19:24:50 +00:00
perf3ct
0afb8a11c8
Revert "feat(search): try to deal with huge dbs, might need to squash later"
This reverts commit 37d0136c500897536e57eb561c881c1bc6d15890.
2025-09-02 19:24:46 +00:00
perf3ct
37d0136c50
feat(search): try to deal with huge dbs, might need to squash later 2025-09-01 04:33:10 +00:00
perf3ct
053f722cb8
feat(search): try to get fts search to work in large environments 2025-08-31 03:15:29 +00:00
perf3ct
1db4971da6
feat(search): implement FST5 w/ sqlite for faster and better searching
feat(search): don't limit the number of blobs to put in virtual tables

fix(search): improve FTS triggers to handle all SQL operations correctly

The root cause of FTS index issues during import was that database triggers
weren't properly handling all SQL operations, particularly upsert operations
(INSERT ... ON CONFLICT ... DO UPDATE) that are commonly used during imports.

Key improvements:
- Fixed INSERT trigger to handle INSERT OR REPLACE operations
- Updated UPDATE trigger to fire on ANY change (not just specific columns)
- Improved blob triggers to use INSERT OR REPLACE for atomic updates
- Added proper handling for notes created before their blobs (import scenario)
- Added triggers for protection state changes
- All triggers now use LEFT JOIN to handle missing blobs gracefully

This ensures the FTS index stays synchronized even when:
- Entity events are disabled during import
- Notes are re-imported (upsert operations)
- Blobs are deduplicated across notes
- Notes are created before their content blobs

The solution works entirely at the database level through triggers,
removing the need for application-level workarounds.

fix(search): consolidate FTS trigger fixes into migration 234

- Merged improved trigger logic from migration 235 into 234
- Deleted unnecessary migration 235 since DB version is still 234
- Ensures triggers handle all SQL operations (INSERT OR REPLACE, upserts)
- Fixes FTS indexing for imported notes by handling missing blobs
- Schema.sql and migration 234 now have identical trigger implementations
2025-08-30 20:39:40 +00:00
perf3ct
057040af06
feat(quick_search): limit the size of the Notes to search through, to 2MB 2025-08-03 19:49:43 +00:00
perf3ct
6c79be881d
feat(search): allow for search through very large notes 2025-08-03 01:44:55 +00:00
perf3ct
8094259c78
feat(search): implement edit_distances (misspelling tolerances) into fulltext search 2025-08-02 23:59:21 +00:00
Elian Doran
b2af043110
chore(nx/server): move source code 2025-04-22 17:16:41 +03:00
Elian Doran
62dbcc0a2e
chore(nx): move all monorepo-style in subfolder for processing 2025-04-22 10:06:06 +03:00
Elian Doran
a59954eb17
chore(monorepo/server): rows import 2025-04-18 12:34:27 +03:00
Elian Doran
5058891af8
chore(monorepo/server): move server-side source code 2025-04-18 12:09:56 +03:00