How to deploy Aya Expanse 8B and 32B in the Cloud?
Aya Expanse is a groundbreaking release in language research, featuring advanced global communication capabilities. It combines a high-performing Command model series with insights gained from a year of Cohere For AI’s focused research efforts. This work spans areas like data optimization, language-specific tuning, safety refinement, and model integration. The outcome is a versatile and robust tool designed to support communication across 23 different languages.
Aya Expanse’s language range includes Arabic, Simplified and Traditional Chinese, Czech, Dutch, English, French, German, Greek, Hebrew, Hindi, Indonesian, Italian, Japanese, Korean, Persian, Polish, Portuguese, Romanian, Russian, Spanish, Turkish, Ukrainian, and Vietnamese.
Key Highlights
- 8-billion and 32-billion parameter multilingual communication models crafted by Cohere For AI
- Covers languages including Arabic, Simplified and Traditional Chinese, Czech, Dutch, English, French, German, Greek, Hebrew, Hindi, Indonesian, Italian, Japanese, Korean, Persian, Polish, Portuguese, Romanian, Russian, Spanish, Turkish, Ukrainian, and Vietnamese
- Built on Cohere’s Command series with extensive research focused on data management, language preference alignment, and safety adjustments
- Supports an extended text range of up to 128,000 characters
- Distributed under a CC-BY-NC license with supplementary guidelines
Model Details
Input: Models input text only.
Output: Models generate text only.
Model Architecture: Aya Expanse 8B is a forward-predictive language system that utilizes a refined transformer structure. Following its main training phase, it undergoes additional tuning through guided adjustments, preference-based refinement, and integration of various training models.
Prerequisites for Running Aya Expense 8B and 32B Model
Make sure you have the following:
- GPUs: 2xRTXA6000 (for smooth execution).
- Disk Space: 100 GB free.
- RAM: At least 100 GB.
- CPU: 48 Cores
Step-by-Step Process to deploy Aya Expense 8B and 32B Model in the Cloud
For the purpose of this tutorial, we will use a GPU-powered Virtual Machine offered by NodeShift; however, you can replicate the same steps with any other cloud provider of your choice. NodeShift provides the most affordable Virtual Machines at a scale that meets GDPR, SOC2, and ISO27001 requirements.
Step 1: Sign Up and Set Up a NodeShift Cloud Account
Visit the NodeShift Platform and create an account. Once you've signed up, log into your account.
Follow the account setup process and provide the necessary details and information.
Step 2: Create a GPU Node (Virtual Machine)
GPU Nodes are NodeShift's GPU Virtual Machines, on-demand resources equipped with diverse GPUs ranging from H100s to A100s. These GPU-powered VMs provide enhanced environmental control, allowing configuration adjustments for GPUs, CPUs, RAM, and Storage based on specific requirements.
Navigate to the menu on the left side. Select the GPU Nodes option, create a GPU Node in the Dashboard, click the Create GPU Node button, and create your first Virtual Machine deployment.
Step 3: Select a Model, Region, and Storage
In the "GPU Nodes" tab, select a GPU Model and Storage according to your needs and the geographical region where you want to launch your model.
We will use 2x RTX A6000 GPU for this tutorial to achieve the fastest performance. However, you can choose a more affordable GPU with less VRAM if that better suits your requirements.
Step 4: Select Authentication Method
There are two authentication methods available: Password and SSH Key. SSH keys are a more secure option. To create them, please refer to our official documentation.
Step 5: Choose an Image
Next, you will need to choose an image for your Virtual Machine. We will deploy Aya Expense 8B and 32B on an NVIDIA Cuda Virtual Machine. This proprietary, closed-source parallel computing platform will allow you to install Aya Expense 8B and 32B Model on your GPU Node.
After choosing the image, click the 'Create' button, and your Virtual Machine will be deployed.
Step 6: Virtual Machine Successfully Deployed
You will get visual confirmation that your node is up and running.
Step 7: Connect to GPUs using SSH
NodeShift GPUs can be connected to and controlled through a terminal using the SSH key provided during GPU creation.
Once your GPU Node deployment is successfully created and has reached the 'RUNNING' status, you can navigate to the page of your GPU Deployment Instance. Then, click the 'Connect' button in the top right corner.
Now open your terminal and paste the proxy SSH IP or direct SSH IP.
Next, if you want to check the GPU details, run the command below:
nvidia-smi
Step 8: Install Aya Expense 8B and 32B
After completing the steps above, it's time to download Aya Expense 8B and 32B from the Ollama website.
Website Link: https://ollama.com/library/aya-expanse
Then run the following command to install the Ollama:
curl -fsSL https://ollama.com/install.sh | sh
After the installation process is complete, run the following command to see a list of available commands:
ollama
Next, run the following command to host the Aya Expense 8B and 32B model so that it can be accessed and utilized efficiently.
ollama serve
Step 9: Select 8B Size of the Model
On website select the 8B size of the model, First we will Run 8B size:
Website Link: https://ollama.com/library/aya-expanse:8b
Step 10: Pull Aya Expense 8B Model
To pull the Aya Expense 8B Model, run the following command:
ollama pull aya-expanse:8b
Step 11: Run Aya Expense 8B Model
Now, you can run the model in the terminal using the following command and interact with your model:
ollama run aya-expanse:8b
Step 12: Select 32B Size of the Model
On website select the 32B size of the model, Next we will Run 32B size:
Website Link: https://ollama.com/library/aya-expanse:32b
Step 13: Pull Aya Expense 32B Model
To pull the Aya Expense 32B Model, run the following command:
ollama pull aya-expanse:32b
Step 14: Run Aya Expense 32B Model
Now, you can run the model in the terminal using the following command and interact with your model:
ollama run aya-expanse:32b
Conclusion
Aya Expense 8B and 32B is a groundbreaking open-source model from Cohere For AI that brings state-of-the-art AI capabilities to developers and researchers. Following this step-by-step guide, you can quickly deploy Aya Expense 8B and 32B on a GPU-powered Virtual Machine with NodeShift, harnessing its full potential. NodeShift provides an accessible, secure, affordable platform to run your AI models efficiently. It is an excellent choice for those experimenting with Aya Expense 8B & 32B and other cutting-edge AI models.
