feat(llm): try to improve tool calling, part 4

apps/server/src/services/llm/constants/llm_prompt_constants.ts

@@ -187,18 +187,45 @@ When responding:
 
         // Tool instructions for Anthropic Claude
         TOOL_INSTRUCTIONS: `<instructions>
-When using tools to search for information, follow these requirements:
+You are an interactive assistant specializing in thorough information retrieval and analysis. Your primary goal is to help users by utilizing available tools comprehensively and systematically.
 
-1. ALWAYS TRY MULTIPLE SEARCH APPROACHES before concluding information isn't available
+CRITICAL TOOL USAGE MANDATES:
-2. YOU MUST PERFORM AT LEAST 3 DIFFERENT SEARCHES with varied parameters before giving up
+
-3. If a search returns no results:
+1. **IMMEDIATE TOOL REACTION**: After receiving ANY tool result, you MUST analyze it thoroughly and determine if additional tools are needed. NEVER stop after a single tool execution unless you have completely fulfilled the user's request.
-   - Try broader terms (e.g., "Kubernetes" instead of "Kubernetes deployment")
+
-   - Use synonyms (e.g., "meeting" instead of "conference")
+2. **AUTOMATIC CONTINUATION**: When you receive tool results, ALWAYS consider them as part of an ongoing investigation. Use the information to:
-   - Remove specific qualifiers (e.g., "report" instead of "Q3 financial report")
+   - Plan additional searches if more information is needed
-   - Try different search tools (vector_search for conceptual matches, keyword_search for exact matches)
+   - Cross-reference results with different search approaches
-4. NEVER tell the user "there are no notes about X" until you've tried multiple search variations
+   - Verify findings with alternative tools
-5. EXPLAIN your search strategy when adjusting parameters (e.g., "I'll try a broader search for...")
+   - Build upon partial results to construct a complete answer
-6. When searches fail, AUTOMATICALLY try different approaches rather than asking the user what to do
+
+3. **MANDATORY MULTI-TOOL SEQUENCES**: 
+   - ALWAYS perform at least 2-3 different tool calls for any information request
+   - Chain tools together: use results from one tool to inform parameters for the next
+   - If initial searches return partial results, IMMEDIATELY run complementary searches
+   - Never accept empty or minimal results without trying alternative approaches
+
+4. **SEARCH STRATEGY REQUIREMENTS**:
+   - Try broader terms if specific searches fail (e.g., "Kubernetes" instead of "Kubernetes deployment")
+   - Use synonyms and alternative terminology (e.g., "meeting" vs "conference" vs "discussion")
+   - Remove qualifiers progressively (e.g., "Q3 financial report" → "financial report" → "report")
+   - Combine different search tools for comprehensive coverage
+
+5. **NEVER GIVE UP EASILY**: 
+   - NEVER tell the user "there are no notes about X" until you've tried at least 3 different search variations
+   - If search_notes fails, try keyword_search and vice versa
+   - Automatically try different approaches rather than asking the user what to do next
+   - Use read_note if you find relevant note IDs in search results
+
+6. **CONTINUATION SIGNALS**: When you receive tool results, phrases like "Based on these results, I'll now..." or "Let me search for additional information..." indicate you understand this is ongoing work requiring further analysis.
+
+7. **COMPREHENSIVE ANALYSIS**: After using tools, always:
+   - Synthesize information from multiple sources
+   - Identify gaps that require additional searches
+   - Cross-reference findings for completeness
+   - Provide thorough, well-researched responses
+
+Remember: Tool usage is iterative and cumulative. Each tool result should inform your next action, leading to comprehensive assistance.
 </instructions>`,
 
         ACKNOWLEDGMENT: "I understand. I'll follow those instructions.",
@@ -222,18 +249,41 @@ Be concise and informative in your responses.
 </system_prompt>`,
 
         // Tool instructions for OpenAI models
-        TOOL_INSTRUCTIONS: `When using tools to search for information, you must follow these requirements:
+        TOOL_INSTRUCTIONS: `You are an interactive assistant specializing in comprehensive information retrieval and analysis. Your goal is systematic and thorough tool usage.
 
-1. ALWAYS TRY MULTIPLE SEARCH APPROACHES before concluding information isn't available
+MANDATORY TOOL USAGE PROTOCOLS:
-2. YOU MUST PERFORM AT LEAST 3 DIFFERENT SEARCHES with varied parameters before giving up
+
-3. If a search returns no results:
+1. **CONTINUOUS TOOL ENGAGEMENT**: After receiving ANY tool result, immediately analyze it and determine what additional tools are needed. Never stop after a single tool execution.
-   - Try broader terms (e.g., "Kubernetes" instead of "Kubernetes deployment")
+
-   - Use synonyms (e.g., "meeting" instead of "conference")
+2. **ITERATIVE INVESTIGATION**: Treat every tool result as part of an ongoing investigation:
-   - Remove specific qualifiers (e.g., "report" instead of "Q3 financial report")
+   - Use results to plan follow-up searches
-   - Try different search tools (vector_search for conceptual matches, keyword_search for exact matches)
+   - Cross-reference findings with alternative approaches  
-4. NEVER tell the user "there are no notes about X" until you've tried multiple search variations
+   - Build upon partial results systematically
-5. EXPLAIN your search strategy when adjusting parameters (e.g., "I'll try a broader search for...")
+   - Chain tools together for comprehensive coverage
-6. When searches fail, AUTOMATICALLY try different approaches rather than asking the user what to do`
+
+3. **MULTI-TOOL REQUIREMENTS**:
+   - Always perform 2-3 different tool calls minimum for information requests
+   - Use results from one tool to inform parameters for the next tool
+   - If searches return partial results, immediately run complementary searches
+   - Never accept empty results without trying alternative approaches
+
+4. **SEARCH ESCALATION STRATEGY**:
+   - Progress from specific to broader terms (e.g., "Kubernetes deployment" → "Kubernetes" → "containers")
+   - Try synonyms and variations (e.g., "meeting" → "conference" → "discussion")
+   - Remove qualifiers systematically (e.g., "Q3 2024 report" → "financial report" → "report")
+   - Combine different search tools for maximum coverage
+
+5. **PERSISTENCE REQUIREMENTS**:
+   - Never tell the user "no information found" until trying at least 3 different search variations
+   - If search_notes fails, immediately try keyword_search
+   - Automatically pivot to alternative approaches without asking the user
+   - Use read_note when search results include relevant note IDs
+
+6. **CONTINUATION INDICATORS**: Use phrases like "Based on these results, I'll now search for..." to signal ongoing analysis and additional tool usage.
+
+7. **COMPREHENSIVE SYNTHESIS**: After tool usage, always synthesize findings, identify information gaps, and provide thoroughly researched responses.
+
+Remember: Each tool result should drive further investigation until the user's request is completely fulfilled.`
     },
 
     OLLAMA: {
@@ -248,18 +298,44 @@ Based on this information, please answer: <query>${query}</query>`,
         // Tool instructions for Ollama
         TOOL_INSTRUCTIONS: `
 CRITICAL INSTRUCTIONS FOR TOOL USAGE:
-1. YOU MUST TRY MULTIPLE TOOLS AND SEARCH VARIATIONS before concluding information isn't available
+
-2. ALWAYS PERFORM AT LEAST 3 DIFFERENT SEARCHES with different parameters before giving up on finding information
+MANDATORY TOOL CONTINUATION PROTOCOL:
-3. If a search returns no results, IMMEDIATELY TRY ANOTHER SEARCH with different parameters:
+- After receiving ANY tool result, you MUST immediately analyze it and continue with additional tools
-   - Use broader terms: If "Kubernetes deployment" fails, try just "Kubernetes" or "container orchestration"
+- NEVER stop after a single tool execution - always plan follow-up actions
-   - Try synonyms: If "meeting notes" fails, try "conference", "discussion", or "conversation"
+- Treat each tool result as a step in an ongoing investigation, not a final answer
-   - Remove specific qualifiers: If "quarterly financial report 2024" fails, try just "financial report"
+
-   - Try semantic variations: If keyword_search fails, use vector_search which finds conceptually related content
+AGGRESSIVE TOOL CHAINING REQUIREMENTS:
-4. CHAIN TOOLS TOGETHER: Use the results of one tool to inform parameters for the next tool
+1. ALWAYS PERFORM MULTIPLE TOOL SEQUENCES: Minimum 2-3 tools per request, even if first tool succeeds
-5. NEVER respond with "there are no notes about X" until you've tried at least 3 different search variations
+2. IMMEDIATE FOLLOW-UP: When you get tool results, instantly decide what additional tools to use:
-6. DO NOT ask the user what to do next when searches fail - AUTOMATICALLY try different approaches
+   - If search finds results → use read_note to get full content
-7. ALWAYS EXPLAIN what you're doing: "I didn't find results for X, so I'm now searching for Y instead"
+   - If search is partial → try broader/alternative search terms
-8. If all reasonable search variations fail (minimum 3 attempts), THEN you may inform the user that the information might not be in their notes`
+   - If results seem incomplete → cross-reference with different tools
+3. CHAIN TOOLS SYSTEMATICALLY: Use results from one tool to inform parameters for the next tool
+4. AUTOMATIC ESCALATION: If search returns no results, IMMEDIATELY try another search with different parameters:
+   - Broader terms: "Kubernetes deployment" → "Kubernetes" → "container orchestration"
+   - Synonyms: "meeting notes" → "conference" → "discussion" → "conversation"
+   - Remove qualifiers: "quarterly financial report 2024" → "financial report" → "report"
+   - Different tools: If keyword_search fails, use search_notes for semantic matching
+
+PERSISTENCE MANDATES:
+5. NEVER respond with "there are no notes about X" until trying at least 3-4 different search variations
+6. DO NOT ask the user what to do next - AUTOMATICALLY try different approaches
+7. ALWAYS EXPLAIN your continuation strategy: "I found some results, now I'll search for additional details..."
+8. If tool results are empty/minimal, IMMEDIATELY pivot to alternative approaches
+
+CONTINUATION SIGNALS:
+- Use phrases like "Based on these results, I'll now..." to show ongoing work
+- "Let me search for additional information..."
+- "I'll cross-reference this with..."
+- These phrases signal you understand this is continuing work requiring more analysis
+
+COMPREHENSIVE COVERAGE:
+9. Synthesize information from multiple tool calls before responding
+10. Identify gaps in information and use additional tools to fill them
+11. Only provide final answers after exhausting relevant tool combinations
+12. If all reasonable variations fail (minimum 3-4 attempts), THEN inform user that information might not be available
+
+Remember: Tool usage is ITERATIVE and CONTINUOUS. Each result drives the next action until complete information is gathered.`
     },
 
     // Common prompts across providers

apps/server/src/services/llm/pipeline/stages/message_preparation_stage.ts

@@ -42,11 +42,29 @@ export class MessagePreparationStage extends BasePipelineStage<MessagePreparatio
         // If tools are enabled, enhance system prompt with tools guidance
         if (toolsEnabled) {
             const toolCount = toolRegistry.getAllTools().length;
-            const toolsPrompt = `You have access to ${toolCount} tools to help you respond. When you need information that might be in the user's notes, use the search_notes tool to find relevant content or the read_note tool to read a specific note by ID. Use tools when specific information is required rather than making assumptions.`;
+            const toolsPrompt = `You have access to ${toolCount} tools to help you respond. 
+
+CRITICAL: You are designed for CONTINUOUS TOOL USAGE and ITERATIVE INVESTIGATION. When you receive tool results, this is NOT the end of your analysis - it's the beginning of deeper investigation.
+
+MANDATORY APPROACH:
+- After ANY tool execution, immediately analyze the results and plan follow-up actions
+- Use multiple tools in sequence to build comprehensive responses
+- Chain tools together systematically - use results from one tool to inform the next
+- When you find partial information, immediately search for additional details
+- Cross-reference findings with alternative search approaches
+- Never stop after a single tool unless you have completely fulfilled the request
+
+TOOL CHAINING EXAMPLES:
+- If search_notes finds relevant note IDs → immediately use read_note to get full content
+- If initial search returns partial results → try broader terms or alternative keywords
+- If one search tool fails → immediately try a different search tool
+- Use the information from each tool to inform better parameters for subsequent tools
+
+Remember: Tool usage should be continuous and iterative until you have thoroughly investigated the user's request.`;
 
             // Add tools guidance to system prompt
             finalSystemPrompt = finalSystemPrompt + '\n\n' + toolsPrompt;
-            log.info(`Enhanced system prompt with tools guidance: ${toolCount} tools available`);
+            log.info(`Enhanced system prompt with aggressive tool chaining guidance: ${toolCount} tools available`);
         }
 
         // Format messages using provider-specific approach

apps/server/src/services/llm/pipeline/stages/tool_calling_stage.ts

@@ -450,10 +450,13 @@ export class ToolCallingStage extends BasePipelineStage<ToolExecutionInput, { re
                 log.info(`Empty result detected for tool ${name}. Will add suggestion to try different parameters.`);
             }
 
-            // Add enhancement for empty results
+            // Add enhancement for empty results and continuation signals
             let enhancedContent = resultContent;
             if (isEmptyResult && !resultContent.startsWith('Error:')) {
-                enhancedContent = `${resultContent}\n\nNOTE: This tool returned no useful results with the provided parameters. Consider trying again with different parameters such as broader search terms, different filters, or alternative approaches.`;
+                enhancedContent = `${resultContent}\n\nCONTINUATION REQUIRED: This tool returned no useful results with the provided parameters. You MUST immediately try additional searches with broader terms, alternative keywords, or different tools. Do not stop here - continue your investigation.`;
+            } else if (!resultContent.startsWith('Error:')) {
+                // Add continuation signal for successful results too
+                enhancedContent = `${resultContent}\n\nCONTINUATION: Analyze these results and determine if additional tools are needed to provide a comprehensive response. Consider follow-up searches, reading specific notes, or cross-referencing with other tools.`;
             }
 
             // Add a new message for the tool result
@@ -483,31 +486,34 @@ export class ToolCallingStage extends BasePipelineStage<ToolExecutionInput, { re
         log.info(`Follow-up needed: ${needsFollowUp}`);
         log.info(`Reasoning: ${hasToolResults ? 'Has tool results to process' : 'No tool results'} ${hasErrors ? ', contains errors' : ''} ${hasEmptyResults ? ', contains empty results' : ''}`);
 
-        // Add aggressive system message for continued tool usage
+        // Add system message to ensure continuation - always add this for any tool results
         if (needsFollowUp) {
-            log.info('Adding enhanced system message to encourage continued tool usage');
+            log.info('Adding system message to ensure LLM continues with additional tools');
 
             let directiveMessage = '';
 
+            // Handle empty results with aggressive re-search instructions
             if (hasEmptyResults) {
-                directiveMessage = `IMPORTANT: No results found with your search. You MUST continue searching with different approaches:
+                directiveMessage = `CRITICAL: Empty results detected. YOU MUST NOT GIVE UP. IMMEDIATELY try alternative approaches:
-1. Use discover_tools to find alternative search methods
+- Use broader search terms (remove specific qualifiers)
-2. Try broader search terms or synonyms
+- Try synonyms and related concepts
-3. Use different search tools (search_notes, keyword_search_notes, attribute_search)
+- Use different search tools (if keyword_search failed, try search_notes)
-4. Search for related concepts instead of specific terms
+- Remove date/time constraints and try general terms
-5. Use read_note on any noteIds you've found previously
+DO NOT respond to the user until you've tried at least 2-3 additional search variations.`;
-
-CRITICAL: Continue executing tools to find information. Do NOT ask the user for guidance yet - exhaust all search options first.`;
             } else {
-                directiveMessage = `EXCELLENT! You found ${toolResultMessages.length} results. Now you MUST continue with these actions:
+                // For successful results, provide comprehensive follow-up instructions
-1. Use read_note with ALL noteId values to get full content
+                directiveMessage = `TOOL ANALYSIS CONTINUATION: You have received tool results above. This is NOT the end of your investigation. You must:
-2. After reading notes, use search_notes or keyword_search_notes to find related information
-3. Use attribute_search to find notes with similar tags/labels
-4. Use note_summarization on long notes
-5. Use content_extraction to pull specific information
-6. Use relationship tool to find connected notes
 
-REMEMBER: Execute multiple tools in sequence to gather comprehensive information. The user expects thorough analysis using 10-20+ tool calls. Continue executing tools!`;
+1. Analyze the results thoroughly
+2. Determine if additional information is needed
+3. Use follow-up tools to gather comprehensive information
+4. If search results included note IDs, use read_note to get full content
+5. Cross-reference findings with alternative search approaches
+6. Continue investigation until you can provide a complete, well-researched response
+
+REMEMBER: Execute multiple tools in sequence to build comprehensive answers. The user expects thorough investigation with systematic tool usage.
+
+Continue your systematic investigation now.`;
             }
 
             updatedMessages.push({