Architecture Overview ​
MapexOS is an enterprise-grade event platform, designed for high-scale event ingestion and processing, with multi-tenancy, stateful automation, and end-to-end observability.
Applies to v1.0.0 — Microservices, Event-Driven (NATS JetStream), script execution (V8), automation (Rule Engine), analytical persistence (ClickHouse), and operational data in MongoDB.
Interactive Architecture Flow ​
Event Processing Architecture
Click to view interactive animation
Goals of this architecture ​
The MapexOS architecture was built to deliver:
- Real horizontal scale per component (ingestion, script execution, routing, rules, storage)
- Extensibility (new protocols, templates, operators, triggers, and domains)
- Multi-tenant isolation with governance and RBAC
- Predictable performance with multi-level caching and isolated execution
- Reliability with retry, replay, and reconciliation patterns
- Audit and compliance through persistent logs and traceability by
eventId
Design philosophy (Enterprise) ​
MapexOS is not just "a set of services". It is a platform engineered with clear design principles to sustain growth and continuous operation in critical environments.
Principles and benefits ​
| Principle | What it means in practice | Benefit to the user |
|---|---|---|
| DDD (Domain-Driven Design) | Services organized by domains: Assets, Router, Rules, Events | Predictable evolution, extensibility, and maintenance |
| Hexagonal (Ports & Adapters) | Ports isolate business rules from infrastructure and integrations | Swap dependencies without rewriting the core |
| Microservices | Independent services with single responsibility | Horizontal scale and modular evolution |
| Event-Driven (NATS JetStream) | Communication via streams and fan-out | Decoupling, replay, operational robustness |
| Explicit contracts | Normalized payloads and schemas per domain | Stable and predictable integrations |
Platform macro view ​
The platform is organized by functional layers that compose the event and automation pipeline.
Ingestion → Template Processing → Routing → Storage → Automation/ActionsMain components (v1.0.0) ​
flowchart LR
A[HTTP Gateway] --> NATS[(NATS JetStream)]
B[MQTT Ingestion] --> NATS
NATS --> JSE[JS Execution Service]
JSE --> R[Router Service]
R --> EV[Events Service]
R --> RE[Rule Engine]
RE --> TR[Triggers Service]
EV --> CH[(ClickHouse)]
CORE[Core Platform] --> MDB[(MongoDB)]
AS[Asset Service] --> MDB
AS --> JSE
JSE --> CACHE[Cache L0/L1/L2]
CACHE --> MINIO[(MinIO/S3)]
RE --> REDIS[(Redis)]Service map ​
Each service has a well-defined role and can scale independently.
| Service | Responsibility | Scales by |
|---|---|---|
| HTTP Gateway | HTTP ingestion + authentication (API Key, JWT, IP whitelist, OAuth2) | Requests/second |
| MQTT Ingestion | MQTT connections + validation + ingestion via NATS | Concurrent connections |
| Asset Service | Assets, Asset Templates, EVA fields, classification | Active assets / queries |
| JS Execution Service | Decode → Validate → Transform (V8 isolates) | Scripts/second |
| Router Service | Fan-out, validation, and routing by Route Groups | Events/second |
| Rule Engine | Stateful automation (Redis), Business Rules execution | Rules Ă— events |
| Triggers Service | Action execution (HTTP, Slack, Teams, Email, NATS, etc.) | Actions/second |
| Events Service | Event and log persistence in ClickHouse (TTL) | Storage throughput + QPS |
| Core Platform | Orgs, RBAC, governance, and multi-tenant context | Governance |
Horizontal scale (how MapexOS grows) ​
MapexOS was designed to answer the most common enterprise question:
"Can I process millions of events?"
The answer comes from the fact that each pipeline stage scales independently:
- HTTP Gateway Ă— N (behind load balancer)
- MQTT Ingestion Ă— N (cluster + leaf nodes)
- JS Execution Ă— N (pools of V8 isolates and worker threads)
- Router Ă— N (parallel fan-out)
- Rule Engine Ă— N (partitioned by org / workload)
- Events Service Ă— N (parallel consumption + optimized tables)
- ClickHouse (replication/sharding as needed)
Data architecture and storage ​
MapexOS separates operational data from analytical data, adopting a clear architecture of responsibility per component.
Responsibilities by technology ​
| Component | Role | Typical workload |
|---|---|---|
| MongoDB | Operational source of truth (configurations and governance) | Frequent reads + moderate writes |
| ClickHouse | Persistent events and logs (OLAP, EVA queries, TTL) | Very high ingestion throughput + analytical queries |
| Redis | Distributed state for automation (counters, cooldowns, flags) | Ultra-low latency |
| MinIO/S3 | Artifacts and distributed cache (L2) | Shared storage / cold storage |
| NATS JetStream | Event backbone (streams + fan-out + replay) | High throughput + durability |
What lives in MongoDB ​
MongoDB acts as the Operational Store, including:
- Organization structure (tenant hierarchy) and context (
orgId,pathKey) - Assets and Asset Templates (including scripts and EVAs)
- Rules and Business Rules definitions
- Triggers, configurations, and governance metadata
- RBAC and access controls associated with tenant context
What lives in ClickHouse ​
ClickHouse is reserved for high-volume data:
events.rawandevents.processed- Persistent logs (router, JS executor, business rule logs, triggers)
- EVA queries (typed dynamic fields)
- TTL per organization/stream
Caching strategy (Performance) ​
For efficient template and script execution at scale, JS Execution uses multi-level caching:
- L0 (RAM): "hot" scripts and bytecode
- L1 (NVMe/SSD): "warm" scripts
- L2 (MinIO/S3): shared storage ("cold cache")
- Fallback: on-demand compilation
L0 RAM → L1 SSD/NVMe → L2 MinIO/S3 → Compile fallbackMulti-tenancy and governance ​
MapexOS implements multi-tenancy by design.
Tenant isolation ​
- Isolation by
orgId+pathKey - Hierarchical filtering via organization context
- Permissions applied on each API via RBAC
State keys (Rule Engine) ​
State is always scoped to avoid collisions:
state:{orgId}:{businessRuleId}:{var}Examples:
state:org-123:br-456:counter
state:org-123:br-456:incidentOpenRetention per tenant (TTL) ​
Retention is configurable by:
- Organization
- Stream (raw, processed, logs)
This enables compliance and controlled cost.
Reliability and fault tolerance ​
MapexOS adopts enterprise patterns for operational robustness:
- Retry with backoff (for triggers and external executions)
- Reconciliation jobs (recovery of lost jobs/executions)
- Event replay via NATS JetStream
- Separation of concerns to contain blast radius
- Circuit breaker (recommended pattern for external executions)
Observability (Production operation) ​
The platform offers observability per pipeline stage:
- Structured logs (JSON)
- Metrics (Prometheus-ready)
- Audit trail and persistent logs for investigations
Debug mode per asset ​
MapexOS allows granular debug per asset for controlled troubleshooting.
debugEnabled = true → Persists detailed logs (history/debug)
debugEnabled = false → Persists only error logs (default)Enterprise note To avoid cost and noise in production environments, detailed log history is opt-in per asset. It is recommended to enable debug only during investigation.
Security (Enterprise baseline) ​
MapexOS was designed to operate securely in corporate environments:
- Authentication via API Key, JWT, IP allowlist, and OAuth2 (HTTP Gateway)
- RBAC enforcement on endpoints and streams
- Secrets in secure storage (referenced via
secrets.*in templates) - Multi-tenant isolation applied at the pipeline core
What users should understand about MapexOS ​
For enterprise documentation, the points that best "educate" the user are:
- MapexOS is an event platform (not just "an IoT system")
- Templates normalize data (Standard Event + EVA fields)
- Decoupled routing (Route Groups provide fan-out with governance)
- Rules are stateful (Redis for counters, incident lifecycle, and cooldowns)
- Logs are persistent and queryable (ClickHouse + EVA queries)
- Scale is per component (you scale ingestion, JS, router, rules separately)
- Multi-tenancy is native (orgId, pathKey, TTL per tenant, RBAC)