Microsoft Foundry Adds AI Model Customization

- Direct Preference Optimization (DPO) is a more recent fine-tuning method that is simpler and more computationally efficient than Reinforcement Learning from Human Feedback (RLHF). Instead of training a separate reward model, DPO directly optimizes the language model using a dataset of paired responses, where one is marked as "preferred" and the other as "not preferred." This approach has been used to optimize well-known open-source models like Llama 3 and Phi-3. - The new Foundry SDK is designed to streamline the development of custom AI applications by providing a unified interface for accessing models, orchestrating agents, running evaluations, and enabling application tracing. It includes a "Project client" for Foundry-native operations and an "OpenAI-compatible client" for functionalities that build on OpenAI's patterns, allowing developers to work within their preferred environment. - For enterprise use cases in analytics and business intelligence, custom copilots can be trained to understand specific business contexts and data schemas. This enables them to assist with generating complex SQL queries from natural language, summarizing key insights from dashboards, and even automating the creation of reports, which can significantly speed up data exploration and decision-making for business stakeholders. - In comparison to other cloud platforms, Microsoft Foundry is positioned as an integrated environment with strong enterprise governance and deep connections to the broader Microsoft ecosystem, including Microsoft 365 and Power BI. While AWS SageMaker offers extensive customization and a wide array of tools, it can have a steeper learning curve. Google's Vertex AI is known for its powerful AutoML capabilities and seamless integration with Google's data analytics services. - The development workflow for custom training on platforms like Vertex AI often involves containerizing the training application code with its dependencies. This allows for greater portability and simplifies the process of scaling experiments and retraining models in a production environment. Vertex AI provides pre-built containers for popular frameworks like TensorFlow and PyTorch, as well as the option to use custom containers. - Real-world applications of custom AI copilots have shown significant productivity gains. For example, some organizations have reported saving several hours per employee per week by automating tasks like document preparation, claims processing, and summarizing meeting notes. In the business intelligence space, AI assistants are being used to translate plain language questions into SQL code, allowing managers to self-serve data analysis without deep technical expertise. - When fine-tuning models with DPO, it can be beneficial to first perform Supervised Fine-Tuning (SFT) on a dataset of preferred responses. This initial SFT step helps the model learn the basic structure and style of the desired output, creating a stronger foundation for the subsequent DPO phase to refine the model's behavior based on preference pairs. - The architecture for a custom copilot in an analytics setting often involves a multi-agent system. This can include a central orchestrator that routes tasks to specialized agents for data access, query generation, and action execution. This modular approach improves accuracy, maintainability, and scalability.