← Back to Login

Frequently Asked Questions

Everything you need to know about using the Schema Crawler API

How do I get an API key?

Getting an API key is completely self-service and takes less than a minute:

  1. Visit the login page: https://testing.nlweb.ai/
  2. Click "Sign in with GitHub"
  3. Authorize the application when GitHub prompts you
  4. View your API key: After login, visit https://testing.nlweb.ai/api/me

The system automatically creates your account and generates a unique API key. You can then use this key for all API requests:

curl -H "X-API-Key: your_api_key" https://testing.nlweb.ai/api/sites
How do I use the API key?

Include your API key in the X-API-Key header with every API request:

curl -H "X-API-Key: your_api_key_here" https://testing.nlweb.ai/api/status

All API endpoints require authentication using this header.

What can I do with the API?

The Schema Crawler API allows you to:

  • Add and manage websites for schema.org data extraction
  • Submit schema map URLs (sitemaps with schema.org markup)
  • Monitor crawling status and progress
  • Access extracted schema.org data from your sites
  • Remove sites and their associated data

All data is isolated to your account - you only see and manage your own sites.

Is my data secure?

Yes! The system implements multi-tenant data isolation:

  • Each user's data is completely isolated in the database
  • You can only access sites and data associated with your API key
  • API keys are securely stored and never exposed except to you
  • All operations are scoped to your user account
What endpoints are available?

Key API endpoints include:

  • GET /api/me - View your user info and API key
  • GET /api/status - Get status of all your sites
  • GET /api/sites - List your sites
  • POST /api/sites - Add a new site
  • POST /api/sites/{site_url}/schema-files - Add schema map to a site
  • DELETE /api/sites/{site_url} - Remove a site and its data
Do I need to involve an administrator?

No! The entire process is self-service. You can:

  • Create your account and get an API key instantly
  • Add and manage sites independently
  • Access all features without approval

Just sign in with GitHub and you're ready to go!