Architecture for the Open Agentic Web: Universal URL extension¶

Introduction: A Universal Discovery Mechanism for AI Agents¶

Just as robots.txt revolutionized how search engine crawlers discover and interact with websites, the Open Agentic Web requires a similar standardized entry point for AI agents.

This simple yet powerful solution allows websites to either: - Expose structured metadata via OpenAPI specifications for static content discovery - Redirect to dynamic endpoints using protocols like SLOP (Simple Language Open Protocol) for interactive services

We're going to design a simple architecture for a static website, enabling Open Agentic Web AI agents to interact with a company's content or services (such as a web agency or consulting firm) via a universal URL extension, similar to robots.txt.

The aim is to create a standardized, machine-readable entry point that is either an OpenAPI-style file to provide structured content, or a set of endpoints based on a fictitious protocol called SLOP (Simple Language Open Protocol).

We will use a first-principles approach to decompose the problem, propose a clear and practical solution, and meet the needs of universality, simplicity and compatibility with AI agents.

1. First principles analysis¶

a. Fundamental objectives¶

• Universality : The architecture must be easily adopted by any website, like robots.txt, which is a standard recognized by crawlers.
• Simplicity: The solution must be lightweight, easy to implement for static sites without a complex backend, and accessible to AI agents without requiring proprietary technologies.
• Compatibility with AI agents: Agents must be able to discover, understand and interact with site content or services (e.g., ask questions, schedule appointments, or retrieve information).
• Standardization: The solution must use open formats or protocols (such as OpenAPI or a dummy protocol like SLOP) to guarantee interoperability.

b. Constraints¶

• Static site: No dynamic backend, so solutions must rely on static files (such as JSON) or external services accessible via an API.
• Automatic discovery: AI agents must find the entry point without manual configuration, like robots.txt is discovered at the root of a domain.
• Security: Interactions must be secure to prevent abuse (for example, limiting access to certain sensitive data).
• Scalability: The solution must be suitable for companies of all sizes, from SMEs to large corporations.

c. Assumptions¶

• Modern AI agents can interpret standardized formats such as JSON, YAML (OpenAPI), or REST endpoints.
• A URL extension (e.g. /.well-known/agent) can become a universal standard if it is simple and well documented.
• A sub-domain or external service (such as an agent hosting service) can handle dynamic interactions for static sites.

2. Proposal: A universal URL extension for AI agents¶

We propose a universal URL extension, located at /.well-known/agents.txt (inspired by conventions like /.well-known/security.txt), or simply /agents.txt which can :

• Provide a static OpenAPI file describing the site's content and services.
• redirect to an external sub-domain or service exposing SLOP endpoints for dynamic interactions.

This approach guarantees automatic discovery (agents systematically check /.well-known/agent) and flexibility (it adapts to both static and dynamic sites).

Option 1: Static OpenAPI file at /.well-known/agents.txt¶

Concept¶

An OpenAPI file (in JSON or YAML format) is placed at the address https://example.com/.well-known/agents.txt. This file acts as a structured description of the site's content and services, enabling AI agents to understand :

• Services offered (e.g. web design, consulting).
• Contact points (e-mail, telephone, form).
• Accessible external tools (calendar, CRM).
• Interaction rules (e.g. request limits, authentication).

OpenAPI file structure¶

A minimal OpenAPI file might look like this:

openapi: 3.0.3
info:
  title: Agent Interface for Example Agency
  description: Interface for AI agents to interact with Example Agency's services
  version: 1.0.0
servers:
  - url: https://example. com
paths:
  /services:
    get:
      summary: Retrieve list of services
      responses:
        '200':
          description: List of services offered
          content:
            application/json:
              schema:
                type: array
                items:
                  type: object
                  properties:
                    name:
                      type: string
                    description:
                      type: string
                    pricing:
                      type: string
  /contact:
    get:
      summary: Retrieve contact information
      responses:
        '200':
          description: Contact details
          content:
            application/json:
              schema:
                type: object
                properties:
                  email:
                    type: string
                  phone:
                    type: string
  /appointment:
    post:
      summary: Book an appointment
      requestBody:
        content:
          application/json:
            schema:
              type: object
              properties:
                date:
                  type: string
                  format: date-time
                purpose:
                  type: string
      responses:
        '200':
          description: Appointment booked
components:
  securitySchemes:
    apiKey:
      type: apiKey
      name: X-Agent-Api-Key
      in: header
security:
  - apiKey: []

Benefits¶

• Simplicity: A static file (JSON/YAML) can be hosted on any static site without a backend.
• Automatic discovery: AI agents check /.well-known/agents by convention, just as crawlers check robots.txt.
• Standardization: OpenAPI is a widely adopted format, readable by both machines and humans.
• Flexibility: The file can describe static services (site content) or redirect to external APIs for dynamic interactions (for example, a Calendly API).

Implementation¶

1. Create the file: Generate an OpenAPI file (JSON or YAML) describing services, contacts and external tools.
2. Host the file: Set the file to https://example.com/.well-known/agents.
3. Update content: Update the file when services or contact details are modified.
4. Secure access: Add an optional API key to restrict access to authorized agents.

Example of use¶

An AI agent accesses https://example.com/.well-known/agents, reads the OpenAPI file, discovers the services (for example, "website creation"), and uses the /appointment endpoint to book an appointment via an external API (such as Calendly).

Option 2: SLOP endpoints to an external sub-domain or service¶

Concept¶

Instead of a static file, the URL /.well-known/agents redirects to a sub-domain (for example, agents.example.com) or an external service (for example, an agent hosting service such as "iz"). This sub-domain exposes five endpoints based on a fictitious protocol called SLOP (Simple Language Open Protocol), designed to be lightweight and universal.

What is SLOP?¶

SLOP is a hypothetical protocol, inspired by the principles of REST, but simplified for interactions with AI agents. It uses standard HTTP requests (GET, POST) and JSON responses, with a minimal structure to reduce complexity.

Five proposed SLOP endpoints¶

The endpoints are designed to cover the typical interactions of a service company (web agency, consultancy):

1. GET /info: Provides general information about the company¶

Description: Returns a structured description of the company (name, mission, services).

Example response: ``json { "name": "Example Agency", "description": "Web agency specializing in website creation and SEO.", "services": [ { "name": "Website creation", "description": "Showcase and e-commerce websites."}, { "name": "SEO", "description": "Search engine optimization."} ] }

#### 2. GET /contact: Provides contact details

**Description**: Returns contact information (e-mail, telephone, social networks).

**Example response**:
json
{
  "email": "contact@example.com",
  "phone": "+33 1 23 45 67 89",
  "social": {"twitter":"@exampleagency"}
}

3. POST /query : Allows agents to ask questions¶

Description: Accepts a natural language query (e.g. "What are your prices for a showcase site?") and returns an answer based on the site's content or an AI.

Example query: ``json { "query": "What are your rates for a showcase site?"}

**Example response** :
```json
{ "answer": "Our rates for a showcase site start at 2000€, contact us for a quote."}

4. POST /appointment : Allows you to schedule an appointment¶

Description: Integrates an external calendar (e.g. Calendly) to reserve a slot.

Example request: json { "date": "2025-06-10T10:00:00Z", "purpose": "SEO Consultation" }

**Example response** :
```json
{ "status": "success", "confirmation": "Appointment confirmed for June 10, 2025 at 10:00 a.m."}

5. POST /lead : Registers a lead in a CRM¶

Description: Allows agents to submit contact information for sales follow-up.

Example request: json { "name": "Jean Dupont", "email": "jean.dupont@example.com", "message": "Interested in your SEO services." }

**Example response** :
```json
{ "status": "success", "message": "Lead registered, we'll contact you soon."}

Architecture¶

Redirection from /.well-known/agents¶

A text or JSON file at https://example.com/.well-known/agents redirects to the subdomain or external service:

``json { "agent_endpoint": "https://agents.example.com" } ```

Alternatively, an HTTP 301 redirect can point directly to agent.example.com.

Hosting endpoints¶

• Subdomain: Set up a subdomain (agent.example.com) on a lightweight server (e.g. Netlify Functions, AWS Lambda) to expose SLOP endpoints.
• External service: Use an agent hosting service (e.g., a hypothetical "iz") that hosts the endpoints and connects to the site content via a JSON-LD file or API.

Endpoint implementation¶

Endpoints are serverless functions that : - Retrieve static data (e.g., JSON-LD from the site). - Connect to external tools (Calendly, HubSpot) via APIs. - Use an LLM (such as xAI's API, see https://x.ai/api) to respond to natural language requests.

Security¶

• Authentication via API keys (X-Agent-Api-Key header).
• HTTPS encryption for all requests.
• RGPD compliance for personal data (leads, appointments).

Benefits¶

• Dynamic: Endpoints enable two-way interactions (questions, reservations, leads).
• Scalability: Endpoints can be extended to add new functionalities.
• Independence: An external service (such as "iz") can manage endpoints, eliminating the need for static sites to set up a backend.
• Universality: SLOP endpoints are simple and can become a standard if widely adopted.

Example of use¶

An AI agent accesses https://example.com/.well-known/agents, discovers the URL https://agents.example.com, and uses the /query endpoint to ask a question ("What services do you offer?"). The agent receives an answer based on the site's JSON-LD, then uses /appointment to book an appointment via Calendly.

3. Comparison of the two options¶

Recommendation¶

• For a basic static site: Use the OpenAPI option at /.well-known/agents. This is the simplest solution, quickest to set up, and universal thanks to the OpenAPI format.
• For dynamic interactions: Use SLOP endpoints on a sub-domain or external service, especially if the company wishes to offer functionalities such as appointment scheduling or lead management.

4. practical implementation: key steps¶

Choose the approach¶

• Option 1: Create an OpenAPI file (JSON/YAML) and place it at /.well-known/agents.
• Option 2: Set up a sub-domain (agents.example.com) or use an external service like "iz" to host SLOP endpoints.

Structuring content¶

• For OpenAPI: Compile services, contacts and external tools in a JSON-LD or OpenAPI file.
• For SLOP: Create a JSON-LD file for static data and connect endpoints to external tools (Calendly, HubSpot).

Configure hosting¶

• For OpenAPI: Host the file on the static site (for example, via Netlify or GitHub Pages).
• For SLOP: Use a serverless platform (AWS Lambda, Netlify Functions) or an agent hosting service.

Add automatic discovery¶

Place a file at /. well-known/agents (JSON or HTTP redirection) to indicate the location of the OpenAPI file or SLOP endpoints.

Secure and test¶

• Implement API keys to restrict access.
• Test with an AI agent (for example, via the xAI API) to verify that data and interactions are accessible.

5. Example: Web agency¶

Scenario: A web agency with a static site (https://example.com) wants to enable AI agents to access its services (site creation, SEO) and schedule appointments.

Option 1: OpenAPI file¶

• File at https://example.com/.well-known/agents (see YAML example above).
• Content: List of services, contact details, and link to a Calendly API for appointments.
• Result : An AI agent reads the file, discovers the services, and uses the Calendly API to book an appointment.

Option 2: SLOP endpoints¶

• File at /.well-known/agents redirecting to https://agents.example.com.
• Endpoints implemented : - /info: Returns the list of services. - /contact: Provides contact details. - /query : Responds to "What are your rates?" via an LLM. - /appointment : Connects to Calendly to reserve a slot. - /lead : Registers leads in HubSpot.
• Result: An AI agent asks a question via /query, books an appointment via /appointment, and submits a lead via /lead.

6. Conclusion¶

The URL extension /.well-known/agents is a simple, universal solution, inspired by robots.txt, which can become a standard for the Open Agentic Web. The OpenAPI option is ideal for static sites on a limited budget, offering a structured description of content. The SLOP option with five endpoints is more powerful for dynamic interactions, but requires a lightweight backend or external service. For a web agency or consultancy, starting with an OpenAPI file is the quickest solution, with the option of upgrading to SLOP to add functionality such as lead management or appointments.

If you'd like a code example to implement SLOP endpoints (for example, in Python with AWS Lambda) or a detailed specification of the SLOP protocol, let me know!