ADR: Replacing Evidence Dashboards with In-Platform Web Component Analytics
Status: Accepted Date: 2026-02-23 Owner: Engineering
Context
Skyledge currently delivers analytics dashboards using Evidence, an open-source BI framework. The current architecture works as follows:
- Dagster produces a per-organization DuckDB file (
organization_duck_dbasset), materialized from daily device telemetry data and uploaded to S3. - A separate Dagster asset (
control_room_dashboard) clones the Evidence project, copies the DuckDB file, runsnpm run sources && npm run build, and uploads the resulting static HTML/JS/CSS to S3. - The static sites are served via CloudFront at
reports.skyledge.com/{controlRoomId}/. - The Angular app embeds the Evidence site in an iframe, using a signed CloudFront URL obtained from
GET /controlrooms/{controlRoomId}/report-url. This endpoint generates short-lived CloudFront signed cookies scoped to/{controlRoomId}/*. - If the Evidence site fails to load (detected via
postMessagefrom the S3 error page), the app falls back to a legacy Angular report view (sl-asset-report-view). - The feature is gated behind the
ff-evidence-dashboardfeature flag.
graph LR
subgraph "Current Architecture"
A[Angular App] -->|GET /controlrooms/:id/report-url| B[Java Backend]
B -->|Signed CloudFront cookies + URL| A
A -->|iframe src| C[CloudFront / S3]
C --> D[Evidence Static Site]
E[Dagster] -->|build per control room| F[clone Evidence repo]
F -->|npm run build| G[Static HTML/JS/CSS]
G -->|upload| C
E -->|organization_duck_db| H[DuckDB on S3]
H -->|copied into build| F
end
This approach has served us well for initial delivery but has reached its limits:
- Not integrated into the platform. Dashboards live in an iframe on a separate domain with no shared navigation or interaction with platform features.
- Build-time scaling. Each control room requires a full Evidence build (clone repo, install deps, build, upload). Build failures require Slack notification and manual intervention.
- No reuse of platform components. Evidence uses Svelte. We cannot use any of our existing Stencil or Angular components, charting libraries, or design system.
- Limited interactivity. Evidence produces static pages. There is no ability to deep-link into platform entities, trigger actions, or respond to platform events.
- Maintenance overhead. The
control_room_dashboardDagster asset manages git clones, shell scripts (evidence.sh), S3 sync, CloudFront invalidation, and Slack error reporting — significant operational complexity for what is ultimately a reporting view. - Iframe boundary. Auth requires a separate signed cookie flow. Styling cannot be shared. Cross-frame communication is limited to
postMessage.
Decision
We will replace the Evidence static-site dashboards with in-platform dashboard web components backed by a new lightweight query service, using the existing widget plugin infrastructure.
graph LR
subgraph "New Architecture"
A[Angular App] -->|WidgetLibraryLoaderService| B["CDN: analytics.esm.js"]
B -->|registers custom elements| A
A -->|"document.createElement('slc-fleet-overview')"| C[Dashboard Component]
C -->|"HTTP (JWT or scoped token)"| D[Dashboard Query Service]
D -->|"SQL query"| E["DuckDB (per-org file from S3)"]
F[Dagster] -->|"organization_duck_db (unchanged)"| E
end
Specifically:
-
Dashboard components will be built as Stencil web components in a new library (e.g.,
analytics) withinskyledge-official-web-components, following the same patterns as the existingdefaultandfleetslibraries. -
Runtime loading will use the existing
WidgetLibraryLoaderService, which dynamically injects<script type="module">tags pointing to ESM bundles hosted onwebcomponents.prod.skyledge.com. No changes to the host app loading mechanism are required. -
Host integration will use the existing
WidgetComponent, which creates custom elements viadocument.createElement(tagName)and passes theWidgetSkyledgeInterface(API access, tenant context, entity data). Dashboard components will receive tenant and control room context through this existing contract. -
A new dashboard query service (Python or Node, not the Java backend) will serve dashboard data by querying the existing per-organization DuckDB files. The SQL currently embedded in Evidence markdown files will move to this service.
-
The Evidence project,
control_room_dashboardDagster asset, CloudFront distribution, andreports.skyledge.comS3 bucket will be retired once all control rooms are migrated.
Existing Infrastructure (Reused, Not New)
The following infrastructure is already in production and will be reused without modification:
| Component | Location | What It Does |
|---|---|---|
WidgetLibraryLoaderService |
control-rooms/webapp/web |
Loads remote ESM bundles via dynamic <script type="module">, deduplicates, tracks load state |
WidgetLibraryEndpointRetrieverService |
control-rooms/webapp/web |
Constructs CDN URLs: {baseUrl}/{library}/{library}.esm.js |
WidgetComponent |
control-rooms/webapp/web |
Creates custom elements by tag name, passes skyledge interface, handles lifecycle |
WidgetSkyledgeInterface |
@skyledge/components-dev |
Contract between host and widgets: API access, auth, entity context |
webcomponents.prod.skyledge.com |
CDN | Hosts compiled Stencil ESM bundles |
| Stencil component patterns | skyledge-official-web-components |
@Prop, @Event, @Method patterns, slc-* tag naming, scoped CSS |
| Per-org DuckDB files | Dagster organization_duck_db asset |
DuckDB files with asset_data, assets, control_room_asset_map tables |
| Existing chart components | skyledge-official-web-components/default |
slc-bar-chart, slc-line-chart, slc-gauge-chart, etc. |
How the Plugin Loader Works
The WidgetLibraryLoaderService is a simple runtime script loader:
- Constructs the URL:
https://webcomponents.prod.skyledge.com/{library}/{library}.esm.js - Creates a
<script type="module" src="...">element and appends it todocument.body - Tracks loaded/loading state to prevent duplicate loads
- The
defaultlibrary is loaded eagerly on app startup; others load on demand
The WidgetComponent then creates elements by tag name and passes context:
const el = document.createElement(widget.tagName); // e.g., 'slc-fleet-overview'el.skyledge = widgetSkyledgeApiService.skyledgeInstance(controlRoomId);el.context = { ... };container.appendChild(el);New dashboard components simply need to be deployed to the CDN as a new library. No host app changes required.
What’s New
| Change | Where | Detail |
|---|---|---|
New analytics Stencil library |
skyledge-official-web-components/analytics/ |
Dashboard components: slc-fleet-overview, slc-fuel-consumption, etc. |
| Dashboard query service | New repo / service | Python or Node service querying DuckDB, serving JSON |
| Dashboard SQL queries | Dashboard query service | Migrated from Evidence markdown inline SQL |
| Response caching | Dashboard query service | Keyed on (tenantId, controlRoomId, params, duckDbLastModified) |
Extend WidgetSkyledgeInterface |
@skyledge/components-dev |
Add dashboard data fetching methods if needed |
| Retire Evidence pipeline | skyledge_dagster, evidence |
Remove control_room_dashboard asset, Evidence repo, CloudFront distro |
Backend: Dashboard Query Service
The control-rooms Java backend will not serve dashboard queries. A new lightweight service (Python or Node) will be responsible for querying DuckDB and returning data to dashboard components.
Current Flow (Evidence via Iframe)
sequenceDiagram
participant App as Angular App
participant API as Java Backend
participant CF as CloudFront
participant S3 as S3 (reports bucket)
App->>API: GET /controlrooms/{crId}/report-url (JWT)
API->>API: Check reportPermission
API->>API: Generate CloudFront signed cookies
API-->>App: { url, Set-Cookie headers }
App->>CF: iframe src (with signed cookies)
CF->>S3: Fetch static Evidence site
S3-->>CF: HTML/JS/CSS
CF-->>App: Rendered in iframe
Note over App: If load fails, postMessage → fallback to legacy view
New Flow (In-Platform Web Components)
sequenceDiagram
participant App as Angular App
participant CDN as CDN (webcomponents.prod)
participant WC as slc-fleet-overview
participant QS as Dashboard Query Service
participant S3 as S3 (DuckDB files)
App->>CDN: Load analytics/analytics.esm.js
CDN-->>App: Registers custom elements
App->>WC: createElement + set skyledge props
WC->>QS: GET /dashboards/fleet-overview (JWT or scoped token)
QS->>QS: Validate tenant + control room access
QS->>S3: Read per-org DuckDB file
QS->>QS: Execute SQL query
QS-->>WC: JSON response
WC->>WC: Render using slc-line-chart, slc-bar-chart, etc.
Query Service Responsibilities
- Accept the user’s JWT directly or a short-lived scoped token obtained from the Java backend
- Validate tenant and control room access
- Execute SQL queries against per-org DuckDB files (downloaded from S3 or queried via
httpfs) - Cache responses keyed on
(tenantId, controlRoomId, params, duckDbLastModified) - Enforce query timeouts
- Serve one endpoint per dashboard view
Auth Options
| Option | How It Works | Trade-off |
|---|---|---|
| JWT passthrough | Dashboard components pass the user’s existing JWT to the query service, which validates it directly (e.g., via Auth0 JWKS) | Simpler; query service needs Auth0 integration |
| Short-lived token | Java backend issues a scoped token (similar to current report-url endpoint); query service validates the token |
More isolation; adds a round-trip |
Architecture Overview
graph TB
subgraph "Angular Host App (platform.skyledge.com)"
WLS[WidgetLibraryLoaderService]
WC_HOST[WidgetComponent]
WLS -->|"load analytics.esm.js"| WC_HOST
end
subgraph "CDN (webcomponents.prod.skyledge.com)"
ESM["analytics/analytics.esm.js"]
end
subgraph "Dashboard Component (Stencil)"
DASH["slc-fleet-overview"]
DASH_CHARTS["slc-line-chart, slc-bar-chart, etc."]
DASH --> DASH_CHARTS
end
subgraph "Dashboard Query Service (Python/Node)"
QS_AUTH[Auth Validation]
QS_CACHE[Response Cache]
QS_DUCK[DuckDB Query Engine]
QS_AUTH --> QS_CACHE
QS_CACHE --> QS_DUCK
end
subgraph "Data Pipeline (Dagster)"
DAG[organization_duck_db asset]
end
subgraph "AWS S3"
S3_DUCK["per-org .duckdb files"]
end
WLS -->|"fetch"| ESM
WC_HOST -->|"createElement + props"| DASH
DASH -->|"HTTP request"| QS_AUTH
QS_DUCK -->|"read"| S3_DUCK
DAG -->|"upload daily"| S3_DUCK
Migration Strategy
Phase 1: Proof of Concept
- Build
slc-fleet-overviewreplicating the Evidence “Overview” tab - Stand up the dashboard query service with the overview SQL queries
- Deploy as
analyticslibrary to CDN - Wire into the platform behind a new feature flag
Phase 2: Parallel Running
- Both Evidence (iframe) and new dashboard available
- Validate data parity between Evidence SQL and query service SQL
- Gather feedback from users
Phase 3: Full Migration
- Migrate remaining Evidence views (Asset Utilisation, Fuel Consumption) as additional components
- Route all control rooms to the new in-platform dashboards
Phase 4: Retirement
- Remove
ff-evidence-dashboardfeature flag and iframe code path - Retire
control_room_dashboardDagster asset - Retire Evidence project repo
- Retire
reports.skyledge.comCloudFront distribution and S3 bucket - Remove
report-service-signer-cloudfrontmodule from Java backend - Remove
GetReportUrlForControlRoomControllerendpoint
Trade-offs
Pros
- Dashboards become native platform features with shared navigation, auth, and entity linking
- Eliminates iframe boundary — no more signed cookie flow,
postMessageerror detection, or cross-origin styling issues - Reuses all existing plugin infrastructure — no new loading mechanism needed
- Eliminates per-control-room build pipeline (Dagster asset, shell scripts, S3 sync, CloudFront invalidation, Slack error reporting)
- Dashboard components can reuse existing Stencil chart components (
slc-bar-chart,slc-line-chart, etc.) - Independent deployment via existing CDN pipeline
- Dashboard components can emit events (
sky:select-asset,sky:navigate) for deep integration with platform features
Cons
- SQL queries move from declarative Evidence markdown to a backend service — less visible to analysts
- Introduces a new service to deploy and operate (dashboard query service)
- Multiple Stencil libraries to manage (now
default,fleets, andanalytics) - Evidence provided free interactive filtering, pagination, and chart rendering; these must be reimplemented in Stencil components
Operational Considerations
- DuckDB in the query service: Python has excellent DuckDB support (
duckdbpackage). Node hasduckdb-nodeorduckdb-async. Consider whether to download.duckdbfiles from S3 and cache locally, or use DuckDB’shttpfsextension to query S3 directly. - Shared chart components: The
analyticsStencil library should reuse existing chart components from thedefaultlibrary.WidgetLibraryLoaderServicealready loadsdefaulteagerly, so these are available whenanalyticscomponents render. - Query visibility: Consider organising dashboard SQL into named query files within the query service (mirroring Evidence’s approach), rather than embedding SQL in application code.
- Monitoring: The current Dagster pipeline sends Slack notifications on build failure. The query service should have equivalent health monitoring and error alerting.
Future Revisit Triggers
Reconsider this architecture if:
- Dashboard count exceeds ~30-50 (may need a composition framework or config-driven system)
- Non-engineers require dashboard authoring (may need a low-code layer or Evidence-like tool for a different audience)
- Query complexity outgrows single DuckDB files (may need a dedicated analytics database)
- Cross-dashboard metric governance becomes important (may need a semantic layer)
Summary
We are replacing Evidence static-site dashboards (currently served via CloudFront and embedded in an iframe) with in-platform Stencil web components, loaded at runtime through the existing WidgetLibraryLoaderService plugin infrastructure, backed by a new lightweight dashboard query service (Python or Node) that queries the existing per-org DuckDB files on S3. This eliminates the per-control-room build pipeline, removes the iframe boundary, brings analytics into the platform UX, and reuses the entire existing widget ecosystem.