Optimization Hub
Overview
The Trustwise Optimization Hub offers a streamlined way to set up and run scans without needing a Blueprint in advance. It enables users to configure and execute safety, alignment, and performance scans on Large Language Models (LLMs), delivering detailed insights to improve AI models. Once a scan is run, a Blueprint will automatically be created with the chosen configurations for future use. This guide explains the key concepts, criteria, and parameters used within the Optimization Hub for effective AI project analysis and compliance.
For additional details on safety, alignment, and performance metrics, refer to the Metrics Information page.
Running a Scan in the Optimization Hub
Follow these steps to set up and run a scan:
-
Access the Optimization Hub
Open the Optimization Hub. -
Select or Upload Documents
In the Data section, choose an existing document from those you have previously uploaded or upload a new one to provide context for the scan.
See Data Upload Instructions for details. -
Choose an Embedding Model
Once a document is selected, the embedding model will be preselected based on the model originally used to embed that document.
See Embedding Model Registration Instructions for details on registering embedding models. -
(Optional) Choose a Rerank Model
If desired, select a rerank model to improve retrieval accuracy. Otherwise, leave this field blank.
See Reranker Model Registration Instructions for details. -
Configure Controls
Pick the controls to apply to your scan:- Safety
- Alignment
- Cost
- Carbon
-
Apply External Policies
Select the external policies relevant to your project:- UK FCA
- NIST AI RMF
- EU AI Act (Coming Soon)
-
Select Control Maps
Specify the data sources used for cost and carbon analysis:- Cost Maps: Defaults to the Trustwise cost map.
- Carbon Maps: Defaults to the Trustwise Energy and Emissions Map.
Note: If Cost or Carbon controls are not selected, you do not have to select a corresponsing Control Map either.
-
Set Metric Thresholds
Define evaluation thresholds for your selected controls. Adjust values to align with project goals. -
Select Infrastructure
- Provider: AWS, GCP, or Azure
- Processor / Instance Type / Region: Choose based on your workload needs.
-
Specify Industry Context
Select the industry category:- Banking
- Insurance
- Healthcare
- Industrial
- TMT
- Others
-
Define Workload Type
Choose the workload that best represents your use case:- Content Creation
- Knowledge Extraction
- Text Summarization
- Sentiment Detection
- User Recommendations
- Language Conversion
- Personalization
-
Select LLMs
Pick up to three registered LLMs (providers include OpenAI, Together AI, Hugging Face, NVIDIA, Azure, Gemini).The first model selected becomes the baseline for synthetic query generation.
See LLM Registration Instructions for details.
-
Provide Queries
- Custom Queries: Manually enter your queries.
- Synthetic Queries: Enable auto-generation using your document, embedding model, and baseline LLM.
-
Submit and Run
Review your selections, then click Submit → Proceed to start the scan.
A Blueprint is automatically created or updated, and you can track results in the Launchpad or Command Center.
Key Concepts
Controls
The Optimization Hub lets you select Controls that define essential parameters for LLM optimization:
- Safety: Ensures ethical and safe operation, minimizing harmful or unintended outputs.
- Alignment: Aligns model behavior with desired outcomes and project goals.
- Cost: Monitors and controls financial costs associated with workloads, promoting efficiency.
- Carbon: Tracks and optimizes the environmental impact (carbon footprint) of your workload.
Choosing the appropriate controls helps you balance safety, alignment, costs, and environmental sustainability.
Policies
Policies in the Optimization Hub provide guidelines and standards to ensure your AI project aligns with internal goals and complies with external regulations. Users can select policies to automatically monitor adherence to ethical, financial, and environmental standards, integrating policy checks directly into model scans for ongoing compliance and optimization.
Internal Policies
Internal Policies allow you to apply in-house standards to your scans, ensuring that they meet specific organizational goals. Currently, the TW-Energy-Emission-Map policy is available, focusing on energy consumption and emissions management, with more policies to be added soon.
External Policies
External Policies ensure that your project complies with relevant regulatory standards:
- UK FCA: Aligns with the UK's Financial Conduct Authority guidelines, ensuring compliance with financial regulations, consumer protection, and best practices.
- NIST AI RMF: Adheres to the National Institute of Standards and Technology AI Risk Management Framework, focusing on ethical and responsible AI use.
- EU AI Act (Coming Soon): Aligns with the EU's Artificial Intelligence Act, which establishes a legal framework for AI safety, transparency, and human rights protection within the EU.
Industry-Specific Optimization
The Optimization Hub also allows users to tailor scans based on industry type:
- Banking: For financial services, focusing on fraud detection, customer support, loan approvals, and transaction analysis.
- Insurance: Optimized for risk assessment, claims processing, and underwriting in the insurance sector.
- Healthcare: Tailored for patient data management, medical diagnosis support, EHR analysis, and medical coding.
- Industrial: Suited for manufacturing, supply chain optimization, predictive maintenance, and automation.
- TMT (Technology, Media, and Telecommunications): For digital media, telecommunications, and data analysis within these sectors.
- Others: For domains not covered by predefined categories, allowing customization to align with specific industry needs.
Troubleshooting and Support
For any issues or questions during setup or execution, refer to the help documentation or contact support at help@trustwise.ai.