Open AI external access via endpoint

1

I need to establish an Azure OpenAI endpoint that will accept communication from a particular range of IP addresses (outside of the Azure cloud). This will allow me to investigate inferrence behavior through an API.

2

Microsoft support response

Thanks to Henry Williams for this help! He writes:

If you want to allow or limit access to your Azure Open AI resource [using
the Azure portal]: Expand the Resource Management section on the left
and click Networking. [This appears to be in the AI Studio context but
this may not matter.]

From there, you will have the option of selecting All networks, which allows anyone
with the endpoint and key to gain access. Alternatively, if you would like to limit
access by IP address or subnet, you can restrict to a virtual network or to a
specific IP address by entering that in the Firewall section.

[There is a screencap with emphasis on Resource Management > Networking
(left side option list)
and on the Firewalls and Virtual Networks tab and on Selected Networks
and Private Endpoints.]

LLM Response

Below I have transcribed instructions provided by an LLM deployment (GPT-4o). These instructions have not been tested and verified.

In particular the initial instructions do not cover limiting access to a range of IP addresses. I requested more on this; which is appended below as a second section.

We will test these instructions and
expand on access limits at our earliest opportunity, making corresponding revisions to this reply.

Step 1: Set Up Your Azure OpenAI Service

  1. Create an Azure Account: If you don’t have an Azure account, you’ll need to create one.
  2. Create an OpenAI Resource:
  • Go to the Azure Portal.
  • Click on “Create a resource” and search for “OpenAI”.
  • Follow the prompts to create your OpenAI resource.

Step 2: Obtain API Keys and Endpoint Information

  1. Navigate to Your OpenAI Resource:
  • Go to the resource you just created in the Azure Portal.
  1. Get API Keys:
  • Under the resource settings, find the “Keys and Endpoint” section.
  • Note down the API key and endpoint URL provided.

Step 3: Create an API Endpoint

You will need to create an endpoint that interacts with the Azure OpenAI service. This can be done using any web framework. Below is an example using Flask (a Python web framework):

  • Install flask:
pip install flask
  • Create a flask application
python  
from flask import Flask, request, jsonify  
import requests  

app = Flask(__name__)  

AZURE_OPENAI_ENDPOINT = 'https://<your-endpoint>.openai.azure.com'  
AZURE_OPENAI_API_KEY = '<your-api-key>'  
DEPLOYMENT_NAME = '<your-deployment-name>'  

headers = {  
    'Content-Type': 'application/json',  
    'api-key': AZURE_OPENAI_API_KEY  
}  

@app.route('/chat', methods=['POST'])  
def chat():  
    user_input = request.json.get('input')  
    if not user_input:  
        return jsonify({'error': 'No input provided'}), 400  

    payload = {  
        "prompt": user_input,  
        "max_tokens": 100  
    }  

    response = requests.post(f"{AZURE_OPENAI_ENDPOINT}/openai/deployments/{DEPLOYMENT_NAME}/completions?api-version=2022-12-01", headers=headers, json=payload)  
      
    if response.status_code != 200:  
        return jsonify({'error': 'Failed to get response from OpenAI'}), response.status_code  

    return jsonify(response.json())  

if __name__ == '__main__':  
    app.run(debug=True)

Step 4: Run Your Flask Application

Save the above code in a file, for example, app.py, and run it:

python app.py

Your API endpoint will be available at http://127.0.0.1:5000/chat.

Step 5: Test Your API Endpoint

You can test your API endpoint using tools like curl or Postman:

curl -X POST http://127.0.0.1:5000/chat -H "Content-Type: application/json" -d '{"input": "Hello, how are you?"}'

This should return a JSON response from the Azure OpenAI service.

Important Notes

  • Security: Make sure to secure your API keys and endpoint. Do not expose them in client-side code.
  • Error Handling: Implement robust error handling in your production code.
  • Rate Limiting: Be aware of rate limits and quotas for the Azure OpenAI service.

This guide should help you get started with setting up an API endpoint for conversing with an Azure OpenAI deployment.

Addendum: Constraining access to a range of IP addresses

This has (likewise) not been tested yet. See note at top.

To configure your API endpoint to accept only HTTP traffic from a constrained range of IP addresses, you can use several approaches depending on your deployment environment. Below are some common methods to achieve this:

Method 1: Using Azure API Management

Azure API Management (APIM) is a powerful tool that can help you secure your API endpoints. You can set up IP filtering in APIM to restrict access to your API.

  1. Create an API Management Service Instance:

    • Go to the Azure Portal and create a new API Management service instance if you don’t already have one.

  2. Add Your API to APIM:

    • Import your API into the API Management service.

  3. Configure IP Restrictions:

    • In the Azure Portal, navigate to your API Management instance.
    • Go to the “APIs” section and select your API.
    • Under the “Settings” tab, look for the “IP Restrictions” section.
    • Add the IP ranges you want to allow.

Method 2: Using Network Security Groups (NSGs)

If you are hosting your API on an Azure Virtual Machine (VM) or App Service, you can use Network Security Groups to restrict access by IP address.

  1. Create or Use an Existing NSG:

    • Navigate to the Azure Portal and create a new NSG or use an existing one.

  2. Add Inbound Security Rules:

    • In the NSG settings, add inbound security rules to allow traffic only from the specified IP ranges.
    • Set the priority and specify the IP ranges in the source field.

Method 3: Using Application Gateway with Web Application Firewall (WAF)

Azure Application Gateway with WAF can also help you secure your API endpoint by restricting IP addresses.

  1. Create an Application Gateway:

    • Go to the Azure Portal and create a new Application Gateway.
    • Enable WAF during the setup.

  2. Configure IP Restrictions:

    • After the Application Gateway is set up, navigate to the “HTTP settings”.
    • Add custom rules to allow traffic only from the specified IP ranges.

Method 4: Implementing IP Filtering in Your Application

If you prefer to handle IP filtering directly within your Flask application, you can implement middleware to restrict access based on IP address.

Here’s an example of how you can do this in Flask:

from flask import Flask, request, jsonify, abort  
   
app = Flask(__name__)  
   
ALLOWED_IPS = ['192.168.1.1', '192.168.1.2']  # Replace with your allowed IP addresses  
   
@app.before_request  
def limit_remote_addr():  
    if request.remote_addr not in ALLOWED_IPS:  
        abort(403)  # Forbidden  
   
@app.route('/chat', methods=['POST'])  
def chat():  
    user_input = request.json.get('input')  
    if not user_input:  
        return jsonify({'error': 'No input provided'}), 400  
  
    # Your existing code to handle the chat request  
    return jsonify({'response': 'This is a placeholder response'})  # Replace with actual response logic  
   
if __name__ == '__main__':  
    app.run(debug=True)

Important Notes

  • Security: Ensure that the IP filtering mechanism you choose is robust and meets your security requirements.
  • Maintenance: Keep in mind that you will need to maintain the list of allowed IP addresses, especially if they change frequently.
  • Scalability: Consider the scalability of your solution, especially if you expect high traffic.

Choose the method that best fits your deployment environment and security requirements.