Introduction
Optimizing AI app performance is crucial for ensuring that applications run efficiently and provide accurate, timely responses. This document outlines best practices for optimizing AI app performance in Glean, with examples to illustrate each point.
1. Clear and Specific Instructions
Providing clear and specific instructions to the AI app is essential for obtaining accurate responses. Instructions should guide the app on how to respond in terms of tone, length, and format.
Example:
For a competitive intelligence app:
-
Instruction: "You are a competitive intelligence assistant specializing in analyzing competitors to Acme in the generative AI space. Focus on comparing products and highlighting similarities, differences, advantages, and disadvantages. Use a tabular format for comparisons."
2. Use of Knowledge Sources
Selecting appropriate knowledge sources is critical. Ensure that the app references relevant and up-to-date documents. Avoid using sources with ambiguous or outdated information.
Example:
-
Knowledge Sources: Choose specific folders or documents in Google Drive, Confluence etc. that contain the most relevant and accurate information for the app's purpose.
3. Response Instructions
Response instructions should be concise and direct. They should specify how the app should structure its responses and what actions to take if it cannot complete a task.
Example:
For an IT help desk app:
-
Instruction: "Always greet the user with 'Hello'. Respond concisely. If a response involves multiple steps, provide step-by-step instructions in a bulleted list. If there is no clear answer, ask the user if they'd like to open a ticket."
4. Testing and Debugging
Regularly test the AI app in different scenarios to ensure it performs well. Use test channels in Slack or other platforms to verify functionality and accuracy before broader deployment.
Example:
-
Testing: Publish the app to a test Slack channel and simulate various user queries to check the app's responses. Adjust instructions and knowledge sources based on the test results.
5. Handling Hallucinations
AI apps may sometimes generate responses based on incorrect or irrelevant information. To mitigate this, provide instructions to use only the specified knowledge sources and avoid generating false responses.
Example:
-
Instruction: "Only respond with direct quotations from documents in the specified knowledge sources and provide citations. Do not generate information that is not found in the source documents."
6. Iterative Improvement
Interacting with the AI is an iterative process. Continuously refine prompts and instructions based on user feedback and performance metrics.
Example:
-
Feedback Loop: Use thumbs up/down feedback buttons in Glean to gather user feedback on responses. Analyze the feedback to identify areas for improvement and update the app's instructions accordingly.
7. Customization and Personalization
Tailor the AI app to meet specific business needs by customizing its behavior and personality. Use detailed instructions to guide the app's responses.
Example:
For a sales assistant app:
-
Instruction: "When asked to compare two products, always respond in tabular form. Include a column that highlights why Acme is superior to the competitor. Offer to compose an email to a prospective customer with this information."
8. Monitoring and Analytics
Monitor the app's performance using analytics tools. Track metrics such as response accuracy, user engagement, and feedback to ensure the app meets performance standards.
Example:
-
Analytics: Use Glean's built-in analytics to track the app's usage and performance. Identify trends and areas for improvement based on the data collected.
Conclusion
By following these best practices, you can optimize the performance of AI apps in Glean, ensuring they provide accurate, efficient, and valuable responses to users. Regular testing, clear instructions, and continuous improvement are key to maintaining high performance.