Presenters

Abstract: Large Language Model Operations or LLMOPs is an emerging discipline which encompasses the practices, techniques and tools used for the operational management of large language models in production environments. LLMOPs allows for the efficient deployment, monitoring and maintenance of large language models. LLMOPs is an extension to Machine Learning Operations or MLOPs. The operational requirements of MLOPs typically apply to LLMOPs as well, but there are challenges with training and deploying LLMs that require a unique approach to LLMOPs.

In this tutorial we provide a comprehensive guide for anyone looking to leverage LLMs in real-world applications. We cover key concepts, including model selection, fine-tuning, security and scalability through practical examples and hands-on exercises. We also address best practices for monitoring, error handling, and continuous integration/continuous deployment (CI/CD) pipelines to ensure robust model performance. By the end of the tutorial, participants will be equipped with the knowledge and skills necessary to effectively operationalize LLMs through example walkthrough of real world applications like chatbots, question-answer apps, and dialogue summarization uses cases.

Duration: Half-Day

Target Audience: Data science practitioners from industry or academia, or any general audience who are interested in using and maintaining Large Language Models in a production environment. The audience can benefit from prior knowledge of Python programming language, a general idea of GenAI and an idea of MLOPs.

Description and Outline:

References:

[1] arXiv:2404.00903
[2] Kulkarni, Akshay & Shivananda, Adarsha & Kulkarni, Anoosh & Gudivada, (2023). LLMs for Enterprise and LLMOps. 10.1007/978-1-4842-9994-4_7.
[3] https://www.databricks.com/glossary/llmops
[4] https://www.ibm.com/topics/llmops