> ## Documentation Index
> Fetch the complete documentation index at: https://documentation.kodelabs.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Data collection

> How KODE OS collects sensor and non-sensor data from connected systems

> How KODE OS collects sensor and non-sensor data from your connected systems

Data collection is the continuous process of gathering data from connected external systems. Once devices and points are discovered and registered, KODE OS begins collecting their data according to configured schedules and protocols.

## Collection methods

KODE OS supports two primary collection methods. The method depends on the integration's communication protocol and the external system's capabilities.

### Scheduled pull (polling)

KODE OS periodically connects to the external system to request the latest data. The polling interval is configurable per connector and per entity type.

```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
sequenceDiagram
    participant KODE as KODE OS
    participant Vendor as External System
    loop Every polling interval
        KODE->>Vendor: Request latest data
        Vendor-->>KODE: Return data payload
        KODE->>KODE: Process and store
    end
```

**Protocols used**: REST, SOAP, SDK, OpenSSL/TCP, SFTP/FTP, Modbus/BACnet

Each integration defines default, minimum, and maximum polling intervals for its entity types. You can adjust the interval within the allowed range from the connector's entity configuration.

### Real-time push

The external system sends data to KODE OS when events occur, without KODE OS asking for it.

```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
sequenceDiagram
    participant Vendor as External System
    participant KODE as KODE OS
    Vendor->>KODE: Event occurs, push data
    KODE->>KODE: Process and store
    Vendor->>KODE: Another event, push data
    KODE->>KODE: Process and store
```

**Protocols used**: Webhooks, MQTT, WebSocket

Push-based integrations receive data as soon as it is available, which lowers latency compared to polling. The external system must send data to the correct KODE OS endpoint.

### Hybrid

Some integrations support both pull and push. For example, an integration might use webhooks for real-time point history and polling for periodic alarm collection.

## Collection modes

The collection mode determines how KODE OS interprets the data it receives. Each integration entity uses one of three modes:

### Snapshot

A single view of the current state at a specific moment. Each collection cycle captures the full current value of all points.

**Use case**: Occupancy counts, current temperature readings, door status

### Delta

Only changes since the last collection are reported. KODE OS stores the differences between polls.

**Use case**: Controller data from BMS systems such as Niagara, where only changed values need to transfer

### Historical

A complete log of past values over a time range. The external system provides timestamped historical data, which KODE OS ingests in bulk.

**Use case**: Historical backfill, energy meter readings, audit logs

## Data types

### Sensor data

Time-series data from physical sensors on devices. These are measurable, often timestamped values from device points.

| Entity            | Description                 | Example                      |
| ----------------- | --------------------------- | ---------------------------- |
| **Point History** | Time-series sensor readings | Temperature: 22.5°C at 14:30 |

### Non-sensor data

Structured data from external systems that is not generated by a physical sensor but matters for building operations.

| Entity            | Description                       | Example                                         |
| ----------------- | --------------------------------- | ----------------------------------------------- |
| **Alarm**         | Alert or fault notifications      | "High temperature alarm on AHU-1"               |
| **Audit Log**     | Access and security event records | "Badge swipe at Main Entrance at 08:15"         |
| **Work Order**    | Maintenance task records          | "Replace filter on RTU-3, due 2026-04-01"       |
| **Booking**       | Space reservation records         | "Conference Room A booked 10:00-11:00"          |
| **EV Charging**   | Charging session records          | "Station 5: 45 kWh delivered, session complete" |
| **Schedule Sync** | Schedule state synchronization    | "Weekday schedule active on AHU-2"              |

## Polling intervals

Each integration defines its polling schedule with three parameters:

| Parameter            | Description                                                |
| -------------------- | ---------------------------------------------------------- |
| **Default interval** | Recommended polling frequency for the entity               |
| **Minimum interval** | Fastest allowed polling frequency (avoids API rate limits) |
| **Maximum interval** | Slowest allowed polling frequency                          |

Polling intervals are configured per entity type within a connector. For example, a Niagara connector might poll point history every 30 seconds but alarms every five minutes.

<Warning>
  Setting polling intervals below the minimum can cause rate limiting or authentication failures with the external system. Always stay within the allowed range.
</Warning>

## Collection status

You can monitor collection status for each connector from the `Data Sources` page. Status indicators show:

* **Up / Active**: Data is collecting successfully on schedule
* **Down / Error**: The last collection attempt failed — check the connector logs
* **Paused**: Collection has been manually paused
* **Disconnected**: The connector cannot reach the external system

Connector detail pages often show Up or Down for overall health. Entity and collection views may also use Active, Error, Paused, or Disconnected. Treat Up with Active, and Down with Error or Disconnected, as the same health signal in different parts of the UI.

## Troubleshooting

<AccordionGroup>
  <Accordion title="Data is not being collected">
    First, verify the connector shows a successful test connection. Then check that the entities are enabled and the polling interval is configured. If the external system requires re-authentication (for example, expired OAuth tokens), update the connector credentials.
  </Accordion>

  <Accordion title="Data appears delayed">
    For pull-based integrations, data freshness depends on the polling interval. If you need more frequent updates, reduce the polling interval within the allowed minimum. For push-based integrations, verify that the external system's webhook or MQTT configuration points to the correct KODE OS endpoint.
  </Accordion>

  <Accordion title="Duplicate data points">
    Some integrations may deliver the same data across overlapping collection cycles. KODE OS deduplicates data based on the point identifier and timestamp. If you see duplicates, check whether the connector's collection mode matches the external system's behavior (snapshot vs. delta).
  </Accordion>
</AccordionGroup>