KDL (Kaman Data Lake) API
The KDL API allows you to query data in your Kaman Data Lake. KDL provides versioned, queryable data storage with full SQL support.
Base URL
All KDL API requests go through the marketplace proxy:
/api/kdl
For self-hosted installations, prepend your instance URL: http://kaman.ai/api/kdl
List Data Lakes
Returns a list of data lakes accessible to the authenticated user.
Endpoint
GET /api/kdl/datalakes
Example
bash
curl -X GET "http://kaman.ai/api/kdl/datalakes" \
-H "Authorization: Bearer kam_your_api_key"
Response
json
[
{
"name": "analytics",
"description": "Analytics data lake",
"created_at": "2024-01-01T00:00:00Z"
},
{
"name": "sales",
"description": "Sales data lake",
"created_at": "2024-02-15T00:00:00Z"
}
]
List Schemas
Returns a list of schemas in a data lake.
Endpoint
GET /api/kdl/datalakes/{lake_name}/schemas
Example
bash
curl -X GET "http://kaman.ai/api/kdl/datalakes/analytics/schemas" \
-H "Authorization: Bearer kam_your_api_key"
Response
json
[
{
"schema_name": "main",
"table_count": 15
},
{
"schema_name": "staging",
"table_count": 5
}
]
List Tables
Returns a list of tables in a data lake.
Endpoint
GET /api/kdl/datalakes/{lake_name}/tables
Example
bash
curl -X GET "http://kaman.ai/api/kdl/datalakes/analytics/tables" \
-H "Authorization: Bearer kam_your_api_key"
Response
json
[
{
"schema_name": "main",
"table_name": "sales",
"row_count": 150000,
"columns": [
{"name": "id", "type": "INTEGER", "nullable": false},
{"name": "date", "type": "DATE", "nullable": false},
{"name": "region", "type": "VARCHAR", "nullable": true},
{"name": "amount", "type": "DECIMAL", "nullable": false}
]
}
]
Query Data
Execute a SQL query against your data lake.
Endpoint
POST /api/kdl/datalakes/{lake_name}/data/query
Request Body
| Field | Type | Description |
|---|---|---|
| query | string | SQL query to execute |
Example
bash
curl -X POST "http://kaman.ai/api/kdl/datalakes/analytics/data/query" \
-H "Authorization: Bearer kam_your_api_key" \
-H "Content-Type: application/json" \
-d '{
"query": "SELECT * FROM main.sales WHERE region = '\''APAC'\'' LIMIT 10"
}'
Response
json
{
"columns": ["id", "date", "region", "amount"],
"data": [
[1, "2024-01-15", "APAC", 15000],
[2, "2024-01-16", "APAC", 22000]
],
"rowCount": 2,
"executionTime": 45
}
Time Travel (Versioning)
KDL supports querying historical versions of your data using the AT VERSION syntax.
Query at Specific Version
bash
curl -X POST "http://kaman.ai/api/kdl/datalakes/analytics/data/query" \
-H "Authorization: Bearer kam_your_api_key" \
-H "Content-Type: application/json" \
-d '{
"query": "SELECT * FROM main.sales AT VERSION 150 LIMIT 10"
}'
Code Examples
TypeScript
typescript
const API_KEY = process.env.KAMAN_API_KEY;
const BASE_URL = 'http://kaman.ai';
interface QueryResult<T> {
columns: string[];
data: any[][];
rowCount: number;
executionTime: number;
}
// List data lakes
async function listDataLakes() {
const response = await fetch(`${BASE_URL}/api/kdl/datalakes`, {
headers: {
'Authorization': `Bearer ${API_KEY}`
}
});
return response.json();
}
// Query data
async function queryData(lakeName: string, sql: string) {
const response = await fetch(
`${BASE_URL}/api/kdl/datalakes/${lakeName}/data/query`,
{
method: 'POST',
headers: {
'Authorization': `Bearer ${API_KEY}`,
'Content-Type': 'application/json'
},
body: JSON.stringify({ query: sql })
}
);
const result: QueryResult<any> = await response.json();
// Convert array data to objects
return result.data.map(row => {
const obj: Record<string, any> = {};
result.columns.forEach((col, i) => {
obj[col] = row[i];
});
return obj;
});
}
// Example usage
const lakes = await listDataLakes();
const sales = await queryData('analytics',
'SELECT * FROM main.sales WHERE amount > 10000 LIMIT 100'
);
Python
python
import requests
import os
from typing import List, Dict, Any
class KamanKDL:
def __init__(self, base_url: str):
self.base_url = f'{base_url}/api/kdl'
self.headers = {
'Authorization': f'Bearer {os.getenv("KAMAN_API_KEY")}',
'Content-Type': 'application/json'
}
def list_datalakes(self) -> List[Dict]:
"""List all accessible data lakes."""
response = requests.get(
f'{self.base_url}/datalakes',
headers=self.headers
)
return response.json()
def list_tables(self, lake_name: str) -> List[Dict]:
"""List tables in a data lake."""
response = requests.get(
f'{self.base_url}/datalakes/{lake_name}/tables',
headers=self.headers
)
return response.json()
def query(self, lake_name: str, sql: str) -> List[Dict[str, Any]]:
"""Execute a SQL query and return results as list of dicts."""
response = requests.post(
f'{self.base_url}/datalakes/{lake_name}/data/query',
json={'query': sql},
headers=self.headers
)
result = response.json()
# Convert to list of dicts
return [
dict(zip(result['columns'], row))
for row in result['data']
]
# Example usage
kdl = KamanKDL('http://kaman.ai')
# List data lakes
lakes = kdl.list_datalakes()
# Query data
sales = kdl.query('analytics',
'SELECT * FROM main.sales WHERE amount > 10000 LIMIT 100'
)
for row in sales:
print(f"Sale: {row['id']} - ${row['amount']}")
cURL
bash
#!/bin/bash
API_KEY="${KAMAN_API_KEY}"
BASE_URL="http://kaman.ai/api/kdl"
# List data lakes
curl -s "$BASE_URL/datalakes" \
-H "Authorization: Bearer $API_KEY"
# List tables
curl -s "$BASE_URL/datalakes/analytics/tables" \
-H "Authorization: Bearer $API_KEY"
# Query data
curl -s -X POST "$BASE_URL/datalakes/analytics/data/query" \
-H "Authorization: Bearer $API_KEY" \
-H "Content-Type: application/json" \
-d '{"query": "SELECT * FROM main.sales LIMIT 10"}'
Best Practices
- Use Pagination: Always use
LIMITfor large tables to avoid timeouts - Parameterize Queries: Avoid SQL injection by properly escaping user inputs
- Use Time Travel Wisely: Query historical versions for auditing, not real-time operations
- Monitor Query Performance: Use
executionTimein responses to identify slow queries
Error Handling
| Status | Error | Description |
|---|---|---|
| 400 | Invalid query | SQL syntax error |
| 401 | Unauthorized | Invalid or missing authentication |
| 404 | Lake/table not found | Resource doesn't exist |
| 403 | Access denied | No permission for this data |
Error Response
json
{
"error": "Invalid query",
"details": "Syntax error at position 45: unexpected token 'FORM'"
}
Next Steps
- Workflows API - Use KDL data in workflows
- Authentication - Learn about API authentication
- Getting Started - Quick start guide