Assets & Templates

MapexOS uses Assets and Asset Templates to connect heterogeneous data sources (IoT and non‑IoT) and convert raw payloads into standardized, searchable, and automatable events.

Assets provide the operational identity (ownership, location, metadata). Templates provide the integration contract (decode → validate → transform) and the field model (EVA) that makes data queryable at scale.

Applies to v1.0.0 — HTTP and MQTT ingestion with ES6+ template scripts.
Additional protocols (CoAP, LoRaWAN) will be introduced in future versions—see the Roadmap.

Data sources

A Data Source is the entry point into MapexOS. It defines how external systems connect, authenticate, and deliver payloads.

Data sources can represent:

device telemetry ingestion
gateway fan‑in (multiple devices behind a single endpoint)
third‑party platforms (webhooks, SaaS integrations, enterprise systems)

Supported protocols (v1.0.0)

Protocol	Description
HTTP	REST endpoints for webhooks, integrations, and third‑party platforms
MQTT	Pub/sub messaging for IoT devices and gateways

Authentication methods

Method	Description
API Key	Simple key-based authentication
JWT	Token-based authentication with signature validation
OAuth2	OAuth 2.0 flows for third‑party integrations
IP Allowlist	Restrict access by source IP address
None	No authentication (development/internal use)

Enterprise note
Prefer JWT or OAuth2 for tenant-grade integrations and IP allowlists for restricted networks. Use None only for controlled environments.

Assets

Debug logs are opt-in (recommended for troubleshooting only)

By default, MapexOS persists only error logs for an asset to minimize storage and maximize throughput.

debugEnabled = true → persists debug + history logs (step-by-step processing traces)
debugEnabled = false → persists error-only logs

If you want to inspect debug traces, you must enable debugEnabled on the asset.

An Asset represents any data-producing entity connected to MapexOS:

IoT sensors and devices
Gateways and concentrators
Third-party platform integrations
Custom hardware
Webhook producers (business events)

Assets are always scoped to an organization and optionally mapped to the physical hierarchy (site/building/floor/zone) for contextual routing, search, and governance.

Asset properties

Property	Description
id	MongoDB ObjectId (internal identifier)
assetUUID	External device identifier (e.g., `devEUI`, `deviceId`) used for routing and lookup
orgId	Organization ID (tenant isolation boundary)
pathKey	Hierarchical organization path (optimized range queries)
enabled	Enables/disables processing for this asset
debugEnabled	Opt-in debug/history logging for troubleshooting
assetTemplateId	Template used to decode, validate, and transform payloads
assetTemplateOrgId	Template owner org (`mapexos_public` for system templates)
routeGroupIds	Route groups (fan-out destinations configured in Router)
protocol	Protocol configuration (`http` or `mqtt` in v1.0.0)
latitude / longitude	Optional geolocation for mapping and geo-based queries
created / updated	Audit timestamps

Protocol support
In v1.0.0, MapexOS supports http and mqtt. The schema already reserves lorawan for future versions—see the Roadmap.

Asset binding (mapping payloads to assets)

Asset binding defines how incoming payloads are associated with an asset. This is configured at the Data Source level and supports both single-asset and multi-asset ingestion patterns.

Binding strategies

Strategy	Description	Typical use case
fixedAssetId	Always maps to a specific asset	Single-device endpoints
uuidField	Extracts asset identity from payload fields/paths	Gateways and multi-device fan-in

Fixed asset binding

json

{
  "type": "fixedAssetId",
  "data": {
    "assetId": "asset-123"
  }
}

All payloads from this data source map to asset-123.

Dynamic binding (uuidField)

json

{
  "type": "uuidField",
  "data": {
    "uuidField": ["/device/id", "/devEui", "/meta/deviceId"]
  }
}

Multiple paths are supported. The first valid match is used.

Example payload:

json

{
  "device": {
    "id": "asset-456"
  },
  "data": {
    "temperature": 25.5
  }
}

MapexOS extracts asset-456 from /device/id and routes the payload to that asset.

Enterprise note
Dynamic binding is critical for gateway architectures: one endpoint can ingest data for thousands of downstream devices without per-device endpoints.

Asset Templates (integration contracts)

An Asset Template defines how raw payloads are converted into standardized events. Templates act as the integration catalog of MapexOS.

Instead of building point‑to‑point code per vendor/device, teams build reusable integration contracts that can be applied across organizations and fleets.

Template components

Component	Description
Scripts	Decode, Validate, Transform functions (ES6+ JavaScript)
Dynamic Fields	EVA field mappings for universal extraction and search

Scope	Description
System Template	Provided by MapexOS, available to all organizations
Organization Template	Created by an organization, private by default
Shared Template	Organization template shared with child organizations

Enterprise note
System templates let you ship “golden integrations” at scale, while organization templates enable customer-specific customizations without breaking global baselines.

Template scripts (decode → validate → transform)

Every template includes three scripts executed in sequence:

txt

Raw Payload → Decode → Validate → Transform → Standard Event

Script responsibilities

Script	Purpose	Example use
Decode	Pre-process transport encoding	Base64 decode, binary parsing, decompression
Validate	Enforce integrity and correctness	Required fields, ranges, schema checks
Transform	Normalize into Standard Event	Map vendor fields to MapexOS schema

Decode script (example)

javascript

function decode(payload, metadata) {
  // Decode base64 payload if needed
  if (typeof payload === "string") {
    const decoded = atob(payload);
    return JSON.parse(decoded);
  }
  return payload;
}

Validate script (example)

javascript

function validate(payload, metadata) {
  // Required fields
  if (payload.temperature === undefined) {
    throw new Error("Missing required field: temperature");
  }

  // Range validation
  if (payload.temperature < -50 || payload.temperature > 100) {
    throw new Error("Temperature out of valid range");
  }

  return true;
}

Validation helpers (Mapex Validator)

MapexOS provides a lightweight built-in validation helper (Mapex Validator) that can be used in the Validate step to perform consistent, low-overhead payload validation—without external dependencies.

Why it exists

Fast and deterministic validation for high-frequency ingestion
Structured errors with field paths (useful for debugging and audit logs)
Schema-like DSL that is easy to maintain inside templates

What it supports (v1.0.0)

Type validators: string, number, boolean, date, array, object, any
Constraints: min, max, integer, pattern, email
Presence rules: required(), optional()
Nested objects and typed arrays via object({...}) and array().items(schema)

Example (Validate script)

javascript

// Available in the template runtime as: $mv
const schema = $mv.object({
  deviceId: $mv.string().min(1).required(),
  temperature: $mv.number().min(-50).max(100).required(),
  humidity: $mv.number().min(0).max(100).optional()
});

function validate(payload, metadata) {
  const { error } = $mv.validate(payload, schema);

  if (error) {
    // error.message includes a human-readable description
    // error.path includes the failing field path (e.g., "temperature")
    throw new Error(error.message);
  }

  return true;
}

Enterprise note
Use Mapex Validator for baseline integrity checks inside templates. For complex domain validation (e.g., cross-field constraints, conditional schemas), implement additional logic in the Validate script while keeping validation deterministic and low-cost per execution.

Transform script (example)

javascript

function transform(payload, metadata) {
  return {
    eventType: "telemetry.temperature",
    eventId: crypto.randomUUID(),
    data: {
      temperature: payload.temperature,
      humidity: payload.humidity,
      unit: "celsius"
    },
    metadata: {
      assetId: metadata.assetId,
      orgId: metadata.orgId,
      receivedAt: metadata.timestamp
    },
    created: new Date().toISOString()
  };
}

Enterprise note
Treat templates as versioned contracts. Introduce breaking changes using template versioning and staged rollout (e.g., new template version per device cohort).

Secure script execution (V8 isolates)

Templates execute securely using V8 isolates with strict limits.

Execution characteristics

Feature	Description
Sandboxed	Isolated runtime per execution
Memory limited	Configurable memory limits per execution
Time limited	Execution timeout to prevent runaway scripts
No network	Scripts cannot perform external calls
No filesystem	Scripts cannot access local files

This model enables flexible integration logic while protecting platform stability and tenant isolation.

Performance: multi-level caching

To support high-frequency ingestion, compiled scripts are cached.

Level	Storage	Purpose
L0	RAM	Hot script objects for fastest execution
L1	NVMe/SSD	Local bytecode cache for fast startup
L2	MinIO/S3	Shared bytecode for horizontal scaling
Fallback	On-demand	Compile from source on cache miss

Scripts are compiled once and reused across large execution volumes.

Dynamic Fields (EVA)

Templates define Dynamic Fields to extract structured values from heterogeneous payloads using the EVA pattern (Entity–Value–Attribute).

Field mapping

Component	Description
Field Name	Logical name (e.g., `temperature`)
Field Type	Data type (`number`, `string`, `boolean`, `date`)
Payload Path	JSONPath to the value

Example mapping

Raw payload:

json

{
  "sensor": {
    "readings": {
      "temp": 23.5,
      "humidity": 65
    }
  }
}

Field mappings:

Field Name	Type	Path
`temperature`	number	`$.sensor.readings.temp`
`humidity`	number	`$.sensor.readings.humidity`

Why EVA matters (enterprise)

Universal search — Query across event types regardless of payload structure
UI autocomplete — Field suggestions in dashboards and query builders
Flexible indexing — No schema migrations when payloads change
Cross-vendor analytics — Compare data from different device families

Processing pipeline (end-to-end)

txt

1. Data Source receives payload (HTTP/MQTT)
2. Authentication validated
3. Asset binding resolves target asset
4. Asset template loaded from cache
5. Scripts executed: Decode → Validate → Transform
6. Standard Event produced
7. Router dispatches to destinations
8. Events Service stores for query

Pipeline walkthrough (example)

txt

Raw Payload:
  { "temp": 25.5, "hum": 65, "devId": "sensor-001" }

Binding:
  uuidField → /devId → "sensor-001"

Template:
  Decode → Validate → Transform

Standard Event:
  eventType: "telemetry.temperature"
  data: { temperature: 25.5, humidity: 65, unit: "celsius" }
  metadata: { assetId: "sensor-001", orgId: "org-456" }

Best practices (enterprise)

Practice	Recommendation
Reuse templates	Create templates per device family, not per device
Validate early	Reject invalid payloads in the Validate script
Normalize units	Convert to standard units in Transform (e.g., Celsius, meters)
Use EVA fields	Extract critical fields for query, alerts, and dashboards
Version templates	Treat templates as contracts; version breaking changes
Test with fixtures	Validate scripts with representative payload samples
Prefer dynamic binding	Use `uuidField` for gateways and large fleets
Design for scale	Keep scripts deterministic and low-cost per execution

Next steps

Events & Pipeline — Storage, retention, and query patterns
Rules & Business Rules — Automation, state, and triggers
Core Concepts — Platform model and Standard Event contract

Assets & Templates ​

Data sources ​

Supported protocols (v1.0.0) ​

Authentication methods ​

Assets ​

Asset properties ​

Asset binding (mapping payloads to assets) ​

Binding strategies ​

Fixed asset binding ​

Dynamic binding (uuidField) ​

Asset Templates (integration contracts) ​

Template components ​

Template sharing and governance ​

Template scripts (decode → validate → transform) ​

Script responsibilities ​

Decode script (example) ​

Validate script (example) ​

Validation helpers (Mapex Validator) ​

Example (Validate script) ​

Transform script (example) ​

Secure script execution (V8 isolates) ​

Execution characteristics ​

Performance: multi-level caching ​

Dynamic Fields (EVA) ​

Field mapping ​

Example mapping ​

Why EVA matters (enterprise) ​

Processing pipeline (end-to-end) ​

Pipeline walkthrough (example) ​

Best practices (enterprise) ​

Next steps ​

Assets & Templates

Data sources

Supported protocols (v1.0.0)

Authentication methods

Assets

Asset properties

Asset binding (mapping payloads to assets)

Binding strategies

Fixed asset binding

Dynamic binding (uuidField)

Asset Templates (integration contracts)

Template components

Template sharing and governance

Template scripts (decode → validate → transform)

Script responsibilities

Decode script (example)

Validate script (example)

Validation helpers (Mapex Validator)

Example (Validate script)

Transform script (example)

Secure script execution (V8 isolates)

Execution characteristics

Performance: multi-level caching

Dynamic Fields (EVA)

Field mapping

Example mapping

Why EVA matters (enterprise)

Processing pipeline (end-to-end)

Pipeline walkthrough (example)

Best practices (enterprise)

Next steps