---
title: Build Your First Tabstack App | Tabstack
description: In this tutorial, you'll build a command-line tool that converts any documentation website into an AI-friendly AGENTS.md file. This file helps AI assistants understand and work with the documented API or tool.
---

## What You’ll Build

A Node.js application that:

1. Takes a documentation URL as input
2. Extracts all documentation pages from the site
3. Reads and processes the content
4. Generates a comprehensive AGENTS.md file summarizing the documentation

## What You’ll Learn

- How to use the Tabstack API to convert web pages to markdown
- How to extract and process multiple documentation pages
- How to use AI-powered transformations to generate structured documentation

## Prerequisites

- Node.js 20+ installed
- A Tabstack API key ([get one here](https://console.tabstack.ai/))
- Basic command-line knowledge
- 20 minutes of your time

## Setup Your Project

### 1. Create Your Project Directory

Terminal window

```
mkdir doc-parser
cd doc-parser
npm init -y
```

### 2. Install Dependencies

Terminal window

```
npm install axios dotenv
```

- `axios`: For making HTTP requests to the Tabstack API
- `dotenv`: For managing your API key securely

### 3. Set Up Your API Key

Create a `.env` file in your project root:

Terminal window

```
TABSTACK_API_KEY=your_api_key_here
```

Replace `your_api_key_here` with your actual API key from the [Tabstack Console](https://console.tabstack.ai/).

Never commit your `.env` file to version control! Add it to `.gitignore`:

Terminal window

```
echo ".env" >> .gitignore
```

## Understanding the Application Flow

Here’s how our application will work:

```
┌─────────────────┐
│  Documentation  │
│   Website URL   │
└────────┬────────┘
         │
         ▼
┌─────────────────────────┐
│  1. Fetch Main Page     │  ← /v1/extract/markdown endpoint
│     (Convert to MD)     │
└────────┬────────────────┘
         │
         ▼
┌─────────────────────────┐
│  2. Extract Doc Links   │  ← Parse markdown
│     from Content        │
└────────┬────────────────┘
         │
         ▼
┌─────────────────────────┐
│  3. Fetch All Doc       │  ← /v1/extract/markdown endpoint
│     Pages (Convert)     │     (multiple calls)
└────────┬────────────────┘
         │
         ▼
┌─────────────────────────┐
│  4. Generate AGENTS.md  │  ← /v1/generate/json endpoint
│     with AI             │
└────────┬────────────────┘
         │
         ▼
┌─────────────────────────┐
│   AGENTS.md File        │
│   Ready for AI Use!     │
└─────────────────────────┘
```

## Building the Application

### Step 1: Set Up the Basic Structure

Create `index.js`:

```
// Import required libraries
require("dotenv").config();
const axios = require("axios");
const fs = require("fs").promises;


// Configuration
const TABSTACK_API_BASE = "https://api.tabstack.ai";
const API_KEY = process.env.TABSTACK_API_KEY;


// Check if API key is set
if (!API_KEY) {
  console.error(
    "❌ Error: TABSTACK_API_KEY not found in environment variables",
  );
  console.error("Please create a .env file with your API key");
  process.exit(1);
}


// Main function
async function main() {
  // Get URL from command line arguments
  const url = process.argv[2];


  if (!url) {
    console.error("❌ Error: Please provide a documentation URL");
    console.error("Usage: node index.js <documentation-url>");
    process.exit(1);
  }


  console.log("🚀 Starting documentation parser...");
  console.log(`📄 Target URL: ${url}\n`);


  try {
    // We'll add our implementation here
    console.log("✅ Done!");
  } catch (error) {
    console.error("❌ Error:", error.message);
    process.exit(1);
  }
}


// Run the application
main();
```

### Step 2: Fetch the Main Documentation Page

Add this helper function before the `main()` function:

```
/**
 * Fetch a URL and convert it to markdown using Tabstack API
 * @param {string} url - The URL to fetch
 * @returns {Promise<{url: string, content: string}>} The markdown content
 */
async function fetchAsMarkdown(url) {
  console.log(`📥 Fetching: ${url}`);


  try {
    const response = await axios.post(
      `${TABSTACK_API_BASE}/v1/extract/markdown`,
      { url },
      {
        headers: {
          Authorization: `Bearer ${API_KEY}`,
          "Content-Type": "application/json",
        },
      },
    );


    return response.data;
  } catch (error) {
    if (error.response) {
      throw new Error(
        `API Error: ${error.response.status} - ${error.response.data.error || "Unknown error"}`,
      );
    }
    throw error;
  }
}
```

Now update the `main()` function to fetch the main page:

```
async function main() {
  const url = process.argv[2];


  if (!url) {
    console.error("❌ Error: Please provide a documentation URL");
    console.error("Usage: node index.js <documentation-url>");
    process.exit(1);
  }


  console.log("🚀 Starting documentation parser...");
  console.log(`📄 Target URL: ${url}\n`);


  try {
    // Step 1: Fetch the main documentation page
    console.log("Step 1: Fetching main documentation page...");
    const mainPage = await fetchAsMarkdown(url);
    console.log(`✅ Fetched ${mainPage.content.length} characters\n`);


    console.log("✅ Done!");
  } catch (error) {
    console.error("❌ Error:", error.message);
    process.exit(1);
  }
}
```

### Step 3: Extract Documentation Links

Add this function to extract links from markdown content:

```
/**
 * Extract documentation links from markdown content
 * @param {string} markdown - The markdown content
 * @param {string} baseUrl - The base URL for resolving relative links
 * @returns {string[]} Array of unique documentation URLs
 */
function extractDocLinks(markdown, baseUrl) {
  console.log("🔍 Extracting documentation links...");


  // Regular expression to match markdown links: [text](url)
  const linkRegex = /\[([^\]]+)\]\(([^)]+)\)/g;
  const links = new Set();


  let match;
  while ((match = linkRegex.exec(markdown)) !== null) {
    let link = match[2];


    // Skip anchors, mailto, and external protocol links
    if (
      link.startsWith("#") ||
      link.startsWith("mailto:") ||
      link.startsWith("tel:")
    ) {
      continue;
    }


    // Convert relative URLs to absolute
    try {
      const absoluteUrl = new URL(link, baseUrl).href;


      // Only include links from the same domain
      const baseDomain = new URL(baseUrl).hostname;
      const linkDomain = new URL(absoluteUrl).hostname;


      if (linkDomain === baseDomain) {
        links.add(absoluteUrl);
      }
    } catch (e) {
      // Skip invalid URLs
      continue;
    }
  }


  const uniqueLinks = Array.from(links);
  console.log(`✅ Found ${uniqueLinks.length} documentation links\n`);


  return uniqueLinks;
}
```

Update `main()` to extract links:

```
async function main() {
  const url = process.argv[2];


  if (!url) {
    console.error("❌ Error: Please provide a documentation URL");
    console.error("Usage: node index.js <documentation-url>");
    process.exit(1);
  }


  console.log("🚀 Starting documentation parser...");
  console.log(`📄 Target URL: ${url}\n`);


  try {
    // Step 1: Fetch the main documentation page
    console.log("Step 1: Fetching main documentation page...");
    const mainPage = await fetchAsMarkdown(url);
    console.log(`✅ Fetched ${mainPage.content.length} characters\n`);


    // Step 2: Extract documentation links
    console.log("Step 2: Extracting documentation links...");
    const docLinks = extractDocLinks(mainPage.content, url);


    // Limit to first 10 pages to avoid rate limits (remove this for production)
    const linksToFetch = docLinks.slice(0, 10);
    console.log(`📚 Will fetch ${linksToFetch.length} documentation pages\n`);


    console.log("✅ Done!");
  } catch (error) {
    console.error("❌ Error:", error.message);
    process.exit(1);
  }
}
```

### Step 4: Fetch All Documentation Pages

Add a function to fetch multiple pages with a delay to respect rate limits:

```
/**
 * Fetch multiple URLs as markdown with rate limiting
 * @param {string[]} urls - Array of URLs to fetch
 * @returns {Promise<Array<{url: string, content: string}>>} Array of markdown content
 */
async function fetchMultipleAsMarkdown(urls) {
  console.log("📥 Fetching documentation pages...");


  const results = [];


  for (let i = 0; i < urls.length; i++) {
    try {
      const result = await fetchAsMarkdown(urls[i]);
      results.push(result);
      console.log(`   ✅ ${i + 1}/${urls.length} complete`);


      // Add a small delay between requests to be respectful
      if (i < urls.length - 1) {
        await new Promise((resolve) => setTimeout(resolve, 500));
      }
    } catch (error) {
      console.log(`   ⚠️  Failed to fetch ${urls[i]}: ${error.message}`);
      // Continue with other URLs
    }
  }


  console.log(
    `✅ Successfully fetched ${results.length}/${urls.length} pages\n`,
  );
  return results;
}
```

Update `main()` to fetch all documentation pages:

```
async function main() {
  const url = process.argv[2];


  if (!url) {
    console.error("❌ Error: Please provide a documentation URL");
    console.error("Usage: node index.js <documentation-url>");
    process.exit(1);
  }


  console.log("🚀 Starting documentation parser...");
  console.log(`📄 Target URL: ${url}\n`);


  try {
    // Step 1: Fetch the main documentation page
    console.log("Step 1: Fetching main documentation page...");
    const mainPage = await fetchAsMarkdown(url);
    console.log(`✅ Fetched ${mainPage.content.length} characters\n`);


    // Step 2: Extract documentation links
    console.log("Step 2: Extracting documentation links...");
    const docLinks = extractDocLinks(mainPage.content, url);


    // Limit to first 10 pages to avoid rate limits
    const linksToFetch = docLinks.slice(0, 10);
    console.log(`📚 Will fetch ${linksToFetch.length} documentation pages\n`);


    // Step 3: Fetch all documentation pages
    console.log("Step 3: Fetching all documentation pages...");
    const allPages = await fetchMultipleAsMarkdown([url, ...linksToFetch]);


    console.log("✅ Done!");
  } catch (error) {
    console.error("❌ Error:", error.message);
    process.exit(1);
  }
}
```

### Step 5: Generate AGENTS.md with AI

Now for the magic! We’ll use the `/transform` endpoint to analyze all the documentation and generate a structured AGENTS.md file:

```
/**
 * Generate AGENTS.md content using AI transformation
 * @param {string} url - The base documentation URL
 * @param {Array<{url: string, content: string}>} pages - All fetched pages
 * @returns {Promise<string>} The generated AGENTS.md content
 */
async function generateAgentsMd(url, pages) {
  console.log("🤖 Generating AGENTS.md with AI...");


  // Combine all markdown content
  const combinedContent = pages
    .map((page) => {
      return `# Source: ${page.url}\n\n${page.content}\n\n---\n\n`;
    })
    .join("");


  // Create a temporary combined markdown file URL
  // For the transform endpoint, we'll use the original URL
  // and pass instructions for comprehensive analysis


  const instructions = `
Analyze this documentation comprehensively and create an AGENTS.md file that helps AI assistants understand and work with this API/tool.


The AGENTS.md should include:


1. **Project Overview**: Brief description of what this project/API does
2. **Key Concepts**: Important concepts, terminology, and architecture
3. **API Endpoints**: Summary of available endpoints (if applicable) with methods and purposes
4. **Authentication**: How to authenticate (if applicable)
5. **Common Use Cases**: Typical scenarios and workflows
6. **Code Examples**: Key usage patterns with code snippets from the docs
7. **Important Notes**: Rate limits, best practices, gotchas, limitations


Format the output as clear, well-structured markdown that an AI agent can easily parse and understand.
Focus on actionable information that helps an AI assistant answer questions and help users work with this tool.
  `.trim();


  // Define the schema for the output
  const schema = {
    type: "object",
    properties: {
      content: {
        type: "string",
        description: "The complete AGENTS.md file content in markdown format",
      },
    },
    required: ["content"],
    additionalProperties: false,
  };


  try {
    const response = await axios.post(
      `${TABSTACK_API_BASE}/v1/generate/json`,
      {
        url: url,
        instructions: instructions,
        json_schema: schema,
      },
      {
        headers: {
          Authorization: `Bearer ${API_KEY}`,
          "Content-Type": "application/json",
        },
      },
    );


    console.log("✅ AGENTS.md generated successfully\n");
    return response.data.content;
  } catch (error) {
    if (error.response) {
      throw new Error(
        `API Error: ${error.response.status} - ${error.response.data.error || "Unknown error"}`,
      );
    }
    throw error;
  }
}
```

### Step 6: Save the Generated File

Add a function to save the AGENTS.md file:

```
/**
 * Save content to a file
 * @param {string} filename - The filename to save to
 * @param {string} content - The content to save
 */
async function saveToFile(filename, content) {
  console.log(`💾 Saving to ${filename}...`);
  await fs.writeFile(filename, content, "utf8");
  console.log(`✅ Saved successfully\n`);
}
```

### Step 7: Complete the Main Function

Now let’s tie it all together in the `main()` function:

```
async function main() {
  const url = process.argv[2];


  if (!url) {
    console.error("❌ Error: Please provide a documentation URL");
    console.error("Usage: node index.js <documentation-url>");
    process.exit(1);
  }


  console.log("🚀 Starting documentation parser...");
  console.log(`📄 Target URL: ${url}\n`);


  try {
    // Step 1: Fetch the main documentation page
    console.log("Step 1: Fetching main documentation page...");
    const mainPage = await fetchAsMarkdown(url);
    console.log(`✅ Fetched ${mainPage.content.length} characters\n`);


    // Step 2: Extract documentation links
    console.log("Step 2: Extracting documentation links...");
    const docLinks = extractDocLinks(mainPage.content, url);


    // Limit to first 10 pages to avoid rate limits
    const linksToFetch = docLinks.slice(0, 10);
    console.log(`📚 Will fetch ${linksToFetch.length} documentation pages\n`);


    // Step 3: Fetch all documentation pages
    console.log("Step 3: Fetching all documentation pages...");
    const allPages = await fetchMultipleAsMarkdown([url, ...linksToFetch]);


    // Step 4: Generate AGENTS.md
    console.log("Step 4: Generating AGENTS.md...");
    const agentsMd = await generateAgentsMd(url, allPages);


    // Step 5: Save to file
    await saveToFile("AGENTS.md", agentsMd);


    console.log("✅ Done! Your AGENTS.md file is ready.");
    console.log(`📝 Generated from ${allPages.length} documentation pages`);
  } catch (error) {
    console.error("❌ Error:", error.message);
    process.exit(1);
  }
}
```

## Running Your Application

Now you can run your documentation parser!

Terminal window

```
node index.js https://docs.example.com
```

### Example Output

```
🚀 Starting documentation parser...
📄 Target URL: https://docs.stripe.com/api


Step 1: Fetching main documentation page...
📥 Fetching: https://docs.stripe.com/api
✅ Fetched 45231 characters


Step 2: Extracting documentation links...
🔍 Extracting documentation links...
✅ Found 87 documentation links


📚 Will fetch 10 documentation pages


Step 3: Fetching all documentation pages...
📥 Fetching documentation pages...
   ✅ 1/10 complete
   ✅ 2/10 complete
   ...
✅ Successfully fetched 10/10 pages


Step 4: Generating AGENTS.md...
🤖 Generating AGENTS.md with AI...
✅ AGENTS.md generated successfully


💾 Saving to AGENTS.md...
✅ Saved successfully


✅ Done! Your AGENTS.md file is ready.
📝 Generated from 10 documentation pages
```

### Example Generated AGENTS.md

Your generated file might look like this:

```
# Stripe API - AI Agent Documentation


## Project Overview


Stripe is a payment processing platform that provides APIs for accepting payments,
managing subscriptions, and handling financial transactions...


## Key Concepts


- **Payment Intents**: Represents the lifecycle of a customer payment
- **Customers**: Represents your customers in Stripe
- **Subscriptions**: Recurring payment schedules...


## API Endpoints


### Payments


- `POST /v1/payment_intents` - Create a payment intent
- `GET /v1/payment_intents/:id` - Retrieve payment details
- `POST /v1/payment_intents/:id/confirm` - Confirm payment


### Customers


- `POST /v1/customers` - Create a new customer
- `GET /v1/customers/:id` - Retrieve customer details...


[... more content ...]
```

## Testing Your Application

Try it with different documentation sites:

Terminal window

```
# Stripe API docs
node index.js https://docs.stripe.com/api


# Tabstack API docs (meta!)
node index.js https://docs.tabstack.ai


# GitHub REST API docs
node index.js https://docs.github.com/en/rest
```

## Complete Code

Here’s the complete `index.js` file for reference:

```
// Import required libraries
require("dotenv").config();
const axios = require("axios");
const fs = require("fs").promises;


// Configuration
const TABSTACK_API_BASE = "https://api.tabstack.ai";
const API_KEY = process.env.TABSTACK_API_KEY;


// Check if API key is set
if (!API_KEY) {
  console.error(
    "❌ Error: TABSTACK_API_KEY not found in environment variables",
  );
  console.error("Please create a .env file with your API key");
  process.exit(1);
}


/**
 * Fetch a URL and convert it to markdown using Tabstack API
 */
async function fetchAsMarkdown(url) {
  console.log(`📥 Fetching: ${url}`);


  try {
    const response = await axios.post(
      `${TABSTACK_API_BASE}/v1/extract/markdown`,
      { url },
      {
        headers: {
          Authorization: `Bearer ${API_KEY}`,
          "Content-Type": "application/json",
        },
      },
    );


    return response.data;
  } catch (error) {
    if (error.response) {
      throw new Error(
        `API Error: ${error.response.status} - ${error.response.data.error || "Unknown error"}`,
      );
    }
    throw error;
  }
}


/**
 * Extract documentation links from markdown content
 */
function extractDocLinks(markdown, baseUrl) {
  console.log("🔍 Extracting documentation links...");


  const linkRegex = /\[([^\]]+)\]\(([^)]+)\)/g;
  const links = new Set();


  let match;
  while ((match = linkRegex.exec(markdown)) !== null) {
    let link = match[2];


    if (
      link.startsWith("#") ||
      link.startsWith("mailto:") ||
      link.startsWith("tel:")
    ) {
      continue;
    }


    try {
      const absoluteUrl = new URL(link, baseUrl).href;
      const baseDomain = new URL(baseUrl).hostname;
      const linkDomain = new URL(absoluteUrl).hostname;


      if (linkDomain === baseDomain) {
        links.add(absoluteUrl);
      }
    } catch (e) {
      continue;
    }
  }


  const uniqueLinks = Array.from(links);
  console.log(`✅ Found ${uniqueLinks.length} documentation links\n`);


  return uniqueLinks;
}


/**
 * Fetch multiple URLs as markdown with rate limiting
 */
async function fetchMultipleAsMarkdown(urls) {
  console.log("📥 Fetching documentation pages...");


  const results = [];


  for (let i = 0; i < urls.length; i++) {
    try {
      const result = await fetchAsMarkdown(urls[i]);
      results.push(result);
      console.log(`   ✅ ${i + 1}/${urls.length} complete`);


      if (i < urls.length - 1) {
        await new Promise((resolve) => setTimeout(resolve, 500));
      }
    } catch (error) {
      console.log(`   ⚠️  Failed to fetch ${urls[i]}: ${error.message}`);
    }
  }


  console.log(
    `✅ Successfully fetched ${results.length}/${urls.length} pages\n`,
  );
  return results;
}


/**
 * Generate AGENTS.md content using AI transformation
 */
async function generateAgentsMd(url, pages) {
  console.log("🤖 Generating AGENTS.md with AI...");


  const instructions = `
Analyze this documentation comprehensively and create an AGENTS.md file that helps AI assistants understand and work with this API/tool.


The AGENTS.md should include:


1. **Project Overview**: Brief description of what this project/API does
2. **Key Concepts**: Important concepts, terminology, and architecture
3. **API Endpoints**: Summary of available endpoints (if applicable) with methods and purposes
4. **Authentication**: How to authenticate (if applicable)
5. **Common Use Cases**: Typical scenarios and workflows
6. **Code Examples**: Key usage patterns with code snippets from the docs
7. **Important Notes**: Rate limits, best practices, gotchas, limitations


Format the output as clear, well-structured markdown that an AI agent can easily parse and understand.
Focus on actionable information that helps an AI assistant answer questions and help users work with this tool.
  `.trim();


  const schema = {
    type: "object",
    properties: {
      content: {
        type: "string",
        description: "The complete AGENTS.md file content in markdown format",
      },
    },
    required: ["content"],
    additionalProperties: false,
  };


  try {
    const response = await axios.post(
      `${TABSTACK_API_BASE}/v1/generate/json`,
      {
        url: url,
        instructions: instructions,
        json_schema: schema,
      },
      {
        headers: {
          Authorization: `Bearer ${API_KEY}`,
          "Content-Type": "application/json",
        },
      },
    );


    console.log("✅ AGENTS.md generated successfully\n");
    return response.data.content;
  } catch (error) {
    if (error.response) {
      throw new Error(
        `API Error: ${error.response.status} - ${error.response.data.error || "Unknown error"}`,
      );
    }
    throw error;
  }
}


/**
 * Save content to a file
 */
async function saveToFile(filename, content) {
  console.log(`💾 Saving to ${filename}...`);
  await fs.writeFile(filename, content, "utf8");
  console.log(`✅ Saved successfully\n`);
}


/**
 * Main function
 */
async function main() {
  const url = process.argv[2];


  if (!url) {
    console.error("❌ Error: Please provide a documentation URL");
    console.error("Usage: node index.js <documentation-url>");
    process.exit(1);
  }


  console.log("🚀 Starting documentation parser...");
  console.log(`📄 Target URL: ${url}\n`);


  try {
    // Step 1: Fetch the main documentation page
    console.log("Step 1: Fetching main documentation page...");
    const mainPage = await fetchAsMarkdown(url);
    console.log(`✅ Fetched ${mainPage.content.length} characters\n`);


    // Step 2: Extract documentation links
    console.log("Step 2: Extracting documentation links...");
    const docLinks = extractDocLinks(mainPage.content, url);


    // Limit to first 10 pages to avoid rate limits
    const linksToFetch = docLinks.slice(0, 10);
    console.log(`📚 Will fetch ${linksToFetch.length} documentation pages\n`);


    // Step 3: Fetch all documentation pages
    console.log("Step 3: Fetching all documentation pages...");
    const allPages = await fetchMultipleAsMarkdown([url, ...linksToFetch]);


    // Step 4: Generate AGENTS.md
    console.log("Step 4: Generating AGENTS.md...");
    const agentsMd = await generateAgentsMd(url, allPages);


    // Step 5: Save to file
    await saveToFile("AGENTS.md", agentsMd);


    console.log("✅ Done! Your AGENTS.md file is ready.");
    console.log(`📝 Generated from ${allPages.length} documentation pages`);
  } catch (error) {
    console.error("❌ Error:", error.message);
    process.exit(1);
  }
}


// Run the application
main();
```

## Learn More

Ready to explore more?

- **[API Reference](/api/index.md)** - Detailed documentation of all endpoints
- **[Quickstart Guide](/getting-started/quick-start/index.md)** - Get your API key and make your first call
- **[Examples](/examples/price-monitor/index.md)** - More real-world examples

## Need Help?

- Check out the [full API documentation](/index.md)
- Support: <support@tabstack.ai>