Running large language models at the enterprise level often means sending prompts and data to a managed service in the cloud, much like with consumer use cases.
This has worked in the past because it’s a convenient way for an enterprise to experiment with LLMs and how they could impact and improve the business, but once you start scaling up new tools utilizing these LLMs, the cloud-based model starts to show some cracks.
