Using Azure OpenAI for a custom chatbot
Finally I got access to Azure OpenAI so recreating my previous example to work with Azure OpenAI. It is generally similar and the differences are highlighted in this Azure doc. There are some minor LangChain and LlamaIndex changes too. You need to complete the prerequisites steps documented prior to proceeding. It may take a few days since you need to submit a form and someone in Microsoft has to approve.
The steps are as described previously (please see this link for more details):
- Create embeddings of custom data
- Chain these with Azure Chat Completions
Make sure to set following env variable or modify code to provide them directly. See the “Manage keys” and “Endpoint” as shown in the Azure portal.
export AZURE_API_KEY='abc...'
export AZURE_API_BASE='https://...'Embeddings
See build_index_azure.py where LlamaIndex invokes Azure and then saves the files. The source PDFs are under local-data and the index output gets stored in local-index-azure as vector files.
Note: The deployment referred is the “Deployment name” you will see after you have deployed a model. You can name it anything you want. You need to deploy 2 different models. At time of this writing, the recommended model text-embedding-ada-002 is used.
Here is a truncated view of an API call.
method=post path=https://<your-azure-ai-instance>.openai.azure.com//openai/deployments/text-embedding-ada-002/embeddings?api-version=2023-03-15-preview
api_version=2023-03-15-preview data='{"input": [[2964, 6234, 25, 220, 23, 198, 1213, 1292, 25, 829, 300, 1481, 9585, 16378, 271, 3923, ....]], "encoding_format": "base64"}'
response_code=200
body='{\n "object": "list",\n "data": [\n {\n "object": "embedding",\n "index": 0,\n "embedding": "lp..."\n }\n ],\n "model": "ada",\n "usage": {\n "prompt_tokens": 336,\n "total_tokens": 336\n }\n}\n'In the end I have an output like:
INFO:llama_index.token_counter.token_counter:> [build_index_from_nodes] Total embedding token usage: 56230 tokens
> [build_index_from_nodes] Total embedding token usage: 56230 tokensChatbot
I have a chat interface using Gradio. See local-index-chat-azure.py It loads the local index built earlier, and using LangChain combines with OpenAI to provide a conversational interface.
In your terminal you should see something like:
> Entering new AgentExecutor chain...
Thought: Do I need to use a tool? No
AI: Hello Rahul! Is there something you need assistance with?
> Finished chain.
> Entering new AgentExecutor chain...
Thought: Do I need to use a tool? Yes
Action: VMware Index
Action Input: How to add users to VMware org
Observation: To add users to VMware org, follow these steps:
1. Open the Cloud Services Console and select Identity & Access Management > Active Users.
2. Click Add Users.
3. On the Add New Users page, enter the email address of the user you want to add to your Organization in the Users text box.
4. Assign the role the user will have in the Organization in the Assign Organization Roles section.
5. To assign the user service roles in the Organization, click Add service access and use the drop-down menus to make a selection.
6. Click Add to send an invitation to the user.
> Finished chain.Now onto creating some applications useful for business!
If these topics interest you, then reach out to me and I will appreciate any feedback. If you would like to work on such problems, you will generally find open roles as well! Please refer to LinkedIn.
