Anaconda AI Navigator

The goal of AI Navigator is to bring state-of-the-art models to Anaconda’s users and to support diverse hardware configurations. The LLM repository contains Llama 3, Gemma, and Mistral models, just to name a few. In addition, four quantization levels are available for each model, to ensure compatibility across a wide range of devices.

After a user downloads a curated model to their PC, computations to support subsequent interaction with the model rely entirely on the user’s local computing resources. This is true whether the user interacts with the model through the chat agent or through the API server. No content of the interaction leaves a user’s machine, and the local model does not learn from the interaction. In fact, a user can interact with the local models through AI Navigator without any network access, once the models of interest have been downloaded.

The Anaconda team verifies that models are coming directly from their publishers and then builds quantizations (compressed versions of the model). Without AI Navigator, a user typically must rely on unsecured third parties or individuals to provide quantized models, because publishers tend to only publish full models, which can be very large and expensive to work with.

AI Navigator comes with an API server, which serves or runs the local LLM model chosen such that it can accept incoming API call requests and return responses. This lets developers incorporate intelligence into their agents and apps.

You can now easily replace OpenAI API calls with calls to the server in AI Navigator, as these calls are compatible with each other. From the API Server page, you can configure the local server’s address, port, and the model file you’d like to set up.

AI Navigator is currently available for Windows 11 and macOS 13+. Requirements for AI Navigator itself are minimal. Instead, requirements are determined by the model you want to use. Each model has two main requirements: (1) You must have enough disk space available to download and store the model. Most models fall between 2 and 10GB, but some models are quite robust, requiring up to 150GB of storage space. (2) You must have enough memory to run the model. AI Navigator uses a combination of the main system memory (RAM) and any available GPU memory (VRAM) to run the model. We recommend a minimum of 16GB of RAM to be able to run most models without issue, but some models require more memory to operate.

The embedded chatbot agent within AI Navigator is the most intuitive way to interact with an LLM thanks to its user-friendly interface. The LLMs you download to your own machine will power the chat, and all the computation is local, meaning that no proprietary or private data will leave your devices from the chat.

The intuitive user interface allows the users to conveniently select the their LLM of choice to power the chat, perform common GenAI tasks, regenerate a response, and check previous chat histories. Through the UI, more sophisticated users can also change the system prompt and configure key settings to shape the chat responses.

For AI app and agent developers,  once they achieve success with a particular model for their specific tasks and scenarios, they can load the LLM into the API server to build and test their own AI apps and agents.