Connectors
Connectors are configured data sources that feed data into the DataPool. Each connector defines where to get data, how often to fetch it, and how to map it into the DataPool’s stream/item/field hierarchy.
Connector Types
| Type | Data Source | Fetch Mode |
|---|---|---|
| SQL | SQL Server, MySQL, PostgreSQL, SQLite | Polling (configurable interval) |
| File | CSV, Excel, JSON, XML | Upload, URL polling, or watch folder |
| Webhook | HTTP POST | Push (real-time) |
| Premium | Custom API integration | Push via ingest API |
Creating a Connector
Navigate to Data Sources
From the sidebar, click Data Sources.
Create new connector
Click New Connector and select the connector type.
Configure the connection
Each connector type has its own configuration. See the individual connector pages for details.
Map the data
Define how source data maps to DataPool streams, items, and fields:
- Stream name — The top-level collection name
- Item key — Which source field uniquely identifies each item
- Field mapping — Which source columns/properties become DataPool fields
Test the connection
Click Test to verify the connection and see sample data.
Save and activate
Save the connector. Data begins flowing immediately.
Ingest Keys
Each connector is assigned an ingest key (prefixed with ik_). This key:
- Identifies the connector in API calls
- Authenticates data submissions for webhook and premium connectors
- Can be rotated without reconfiguring the connector
Treat ingest keys like passwords. Do not embed them in client-side code or commit them to version control.
Connector State
Each connector displays its current state:
| State | Description |
|---|---|
| Active | Running normally, data flowing |
| Idle | Connected but no recent data |
| Error | Connection or data error — check logs |
| Disabled | Manually paused by a user |
Connector Logs
Each connector maintains a log of recent activity:
- Successful data fetches with item count
- Errors with details and timestamps
- Connection attempts and results
View logs from the connector detail page. Logs are retained for 30 days.
Monitoring
The Data Sources page shows a summary of all connectors with:
- Status indicators (green/yellow/red)
- Last successful fetch time
- Item count
- Error count in the last 24 hours
Best Practices
| Practice | Why |
|---|---|
| Set appropriate polling intervals | Too frequent wastes resources; too slow means stale data |
| Use unique item keys | Prevents duplicate items in the DataPool |
| Monitor error counts | Catch data source issues before they affect displays |
| Name streams descriptively | sales-by-region is better than data1 |
| Test before deploying | Verify data mapping before going live |