InDI: Industrial Digital Twin Platform

Transforming Industrial Operations Through Immersive 3D Digital Twins and Real-Time Data Integration

Executive Summary

InDI (Industrial Digital Integration) is an enterprise-grade digital twin platform that revolutionizes how industrial facilities are visualized, monitored, and managed. By combining high-fidelity 3D visualization with real-time operational data, InDI enables organizations to make faster, more informed decisions about their critical infrastructure.

Built on a modern .NET 8/9 architecture with Unity 3D integration, InDI bridges the gap between static CAD/design models and live operational intelligence. The platform serves mining, oil & gas, manufacturing, and utilities sectors with a comprehensive solution that spans from CAD processing pipelines through to immersive VR experiences.

Key Value Propositions

For Business Stakeholders:

• Operational Clarity: Transform complex CAD models into interactive 3D environments with live data overlays
• Rapid Insights: Navigate massive industrial facilities with instant access to equipment status and metadata
• Risk Mitigation: Train personnel in virtual environments before engaging with real equipment
• Cost Reduction: Identify issues and plan maintenance without physical site visits
• Multi-Tenant Architecture: Support multiple industrial sites from a single deployment with flexible storage backends
• Vendor Independence: Deploy on Azure, AWS, on-premises, or hybrid cloud with zero code changes

For Software Developers:

• Clean Architecture: Domain-driven design with clear separation of concerns across 36+ reusable libraries
• Scalable Processing: THOR pipeline transforms millions of CAD assets into optimized 3D tilesets
• High-Performance Graphics: Unity-based rendering with DOTS/ECS for millions of entities at 60+ FPS
• gRPC Performance: 4x throughput improvement over REST with bidirectional streaming
• Storage Abstraction: Unified API supporting FileSystem, Azure Blob, AWS S3, and MinIO
• Lucene Search: Sub-second full-text search across millions of assets with faceted filtering
• Comprehensive Testing: 15 test projects with 700+ tests covering unit, integration, and E2E scenarios
• Modern Build Pipeline: Type-safe C# build automation with Nuke and Azure Pipelines

Target Users & Deployments

• Mining Operations: BHP, Rio Tinto, Fortescue Metals Group (FMG), Roy Hill
• Oil & Gas: Shell, Chevron, Woodside
• Manufacturing: Processing plants, refineries
• Utilities: Power generation, water treatment

Repository Scope

This monorepo contains the complete InDI platform:

• 36+ Libraries: Modular components for storage, search, data integration, and visualization
• 5 Applications: Web apps, Unity viewers, console tools, and background services
• 1,560+ C# Files: Professional codebase with extensive documentation
• 15 Test Projects: NUnit, BUnit, and Selenium tests ensuring quality
• .NET 8/9: Modern framework with cross-platform support

Project Overview

What is InDI?

InDI is a comprehensive digital twin platform designed for large-scale industrial operations. The platform encompasses:

Complete Data Pipeline:

1. THOR Pipeline: CAD processing engine that transforms raw Navisworks exports into optimized 3D assets
2. Storage Layer: Multi-backend abstraction supporting Azure, AWS S3, MinIO, and local filesystems
3. Search Engine: Lucene.NET-powered indexing for millions of assets with hierarchical queries
4. Integration Layer: Real-time data streaming from SCADA, IoT, PIMS, and health management systems

Visualization & Interaction:

1. Unity Viewer: High-performance 3D rendering with VR support and DOTS/ECS for millions of entities
2. Blazor Web App: Server-side UI with real-time SignalR updates and embedded Unity integration
3. Multi-Layer Interop: Seamless communication between Blazor ↔ JavaScript ↔ Unity ↔ Embedded Browser
4. Asset Status System: Real-time equipment health visualization with colored highlights and contextual cards

Key Capabilities:

• High-Performance 3D Visualization: Render millions of CAD assets in real-time with smooth navigation
• Real-Time Data Integration: Overlay live equipment status, sensor readings, and operational data onto 3D models
• Immersive Experiences: Support for VR headsets (Oculus, HTC Vive) and 360° immersive environments (Igloo systems)
• Intelligent Search: Full-text search across complex CAD hierarchies with sub-second response times
• Multi-Platform Delivery: Desktop viewers, web applications, and VR experiences from a single codebase
• Configuration-Driven: Zero code changes required between dev, staging, and production environments

Problems We Solve

For Operations Teams:

• Navigating complex facilities with thousands of equipment items is overwhelming
• Understanding equipment health requires context switching between multiple systems
• Training on complex operations is expensive and risky
• Planning maintenance activities without complete facility understanding leads to inefficiencies

For Engineering Teams:

• CAD models are static and don’t reflect current operational state
• Integrating design documentation with as-built reality is manual and error-prone
• Identifying spatial relationships in 2D drawings is difficult and time-consuming

For Management:

• Lack of visibility into facility-wide equipment health
• Difficulty communicating complex technical issues to stakeholders
• High cost of physical site visits for remote facilities
• Slow response times when incidents occur

Technical Architecture

System Architecture Overview

InDI employs a multi-tier architecture that separates concerns between CAD processing, 3D rendering, web UI, data integration, and storage. The platform follows layered architecture principles with clear separation between presentation, business logic, data access, and infrastructure concerns.

For Business Stakeholders: Architecture Benefits

Scalability: The tileset-based architecture allows InDI to handle facilities with millions of individual components without performance degradation. Whether you’re visualizing a small processing unit or an entire mine site, the system maintains smooth, responsive performance.

Maintainability: Clear separation of concerns means updates to the UI don’t affect 3D rendering, and changes to data sources don’t require reprocessing CAD models. This reduces maintenance costs and allows rapid deployment of new features.

Reliability: Transactional indexing and immutable data artifacts ensure data integrity. If a processing step fails, the system can retry without corrupting existing data.

Cost Efficiency: The processing pipeline runs once during CAD model ingestion, producing optimized assets that require minimal cloud compute resources during normal operation. This keeps operational costs predictable and low.

Flexibility: The platform integrates with any data source accessible via REST, SOAP, gRPC, or MQTT protocols. This means InDI can connect to legacy systems, modern cloud APIs, and everything in between.

For Software Developers: Technical Deep Dive

Processing Pipeline (THOR)

The THOR pipeline transforms raw CAD exports into optimized, spatially-organized 3D assets:

1. Navisworks Export: CAD models are exported in batches of ~500,000 triangles using the Navisworks API
2. Metadata Extraction: Equipment properties, hierarchies, and relationships are stored in SQLite
3. Node Graph Construction: A complete graph of CAD nodes is built, tracking original Navisworks relationships and handling virtual duplication
4. Tileset Demarcation: The site is subdivided spatially using polygon count and bounding box heuristics
5. Geometry Optimization: Meshes are optimized (vertex welding, normal recalculation, triangle reduction)
6. CAD Index Assignment: Each node receives a unique integer ID, assigned spatially for cache coherence
7. Format Export: Assets are exported to glTF, 3D Tiles (b3dm), or Unity AssetBundles

Key Innovation: Tileset Port Architecture

Traditional scene-based pipelines process geometry linearly, blocking metadata processing until geometry is ready. The Tileset Port architecture decouples these concerns:

• Three Data Channels: SceneHandles (geometry), TilesetData (spatial organization), NodeGraph (metadata)
• Parallel Processing: Tileset demarcation begins as soon as node bounds are available, even while geometry is still being optimized
• Type-Safe Message Passing: DataFlow blocks ensure correct sequencing without manual synchronization
• Flexible Step Configuration: Each processing step declares which port types it requires and produces

This architecture reduced pipeline processing time by 40% for typical facility models.

Rendering Architecture (Unity Viewer)

The Unity-based viewer leverages:

• Universal Render Pipeline (URP): Modern render pipeline with mobile and desktop optimization
• Data-Oriented Technology Stack (DOTS): High-performance systems for visibility culling and screen analysis
• Entity Component System (ECS): Efficient memory layout for millions of entities
• Custom Shaders: Status-based emission for equipment health visualization
• LOD System: Automatic level-of-detail switching based on distance and importance
• Occlusion Culling: GPU-based occlusion queries to skip rendering hidden geometry

Search System (Lucene.NET)

Full-text search across CAD hierarchies using Lucene indexes:

• Field Types: Tokenized text fields for names/descriptions, non-tokenized string fields for IDs/GUIDs
• Hierarchical Queries: Navigate parent-child relationships in O(1) time using stored path fields
• Incremental Updates: Add/remove/update documents without full reindexing
• Memory-Mapped Indexes: Direct file access for minimal memory footprint
• Custom Analyzers: Domain-specific tokenization for equipment naming conventions

// Example: Searching for pumps in a specific area
var query = new BooleanQuery
{
    { new TermQuery(new Term("Type", "Pump")), Occur.MUST },
    { new WildcardQuery(new Term("Path", "*/Area51/*")), Occur.MUST }
};
var results = indexSearcher.Search(query, 100);

Web Application (Blazor Server)

Blazor provides:

• Component-Based Architecture: Reusable UI components with C# code-behind
• SignalR Communication: WebSocket-based real-time updates from server to client
• JS Interop: Seamless integration with JavaScript libraries for specialized functionality
• Dependency Injection: Scoped services for user-specific data and state management

Multi-Layer JavaScript Interoperability

One of the most complex aspects of InDI is the three-layer JavaScript interop:

1. Blazor ↔ JavaScript: Standard Blazor JS interop for web components
2. JavaScript ↔ Unity: Custom web component system with mutation observers
3. Unity ↔ Embedded Browser: ZFBrowser API for direct DOM manipulation

This architecture enables:

• Real-time synchronization between 3D scene and web UI
• Performant updates using web workers for heavy calculations
• Bidirectional communication without polling

Technology Stack

Backend Technologies

Frontend Technologies

Data Layer

Integration Layer

Development & DevOps

Core Subsystems and Components

1. THOR Pipeline (CAD Processing)

Purpose: Transform raw CAD models into optimized, spatially-organized 3D assets suitable for real-time rendering.

Key Technologies: .NET 8, Aspose.3D, TPL Dataflow, System.Threading.Channels

Design Patterns:

• Pipeline Pattern: Processing steps connected via typed channels
• Actor Model: Each step processes messages independently and asynchronously
• Immutable Data Structures: Scene handles represent immutable geometry, new versions are created for modifications

Integration Points:

• Input: Navisworks CAD exports (FBX, OBJ), metadata SQLite database
• Output: Unity AssetBundles, glTF files, 3D Tiles (b3dm), Lucene indexes, CAD index binary files

Notable Implementation Details:

Tileset Port Base Class: All processing steps inherit from TilesetPortStepBase, which provides:

• Automatic port management (SceneHandle, TilesetData, NodeGraph)
• Dependency tracking (RequireNodeGraphForSceneProcessing, RequireTilesetForSceneProcessing)
• Auto-propagation of completion signals
• Fault handling and cancellation support

Worker Pool Pattern: The Optimize Tiles step spawns ~64 worker tasks that continuously pull tiles from a queue and optimize them in parallel. This maximizes CPU utilization on multi-core servers.

Unmerge Geometry: Identifies “islands” of connected vertices within a single mesh and splits them into separate meshes. This is critical for creating sensible tile boundaries. Uses a union-find algorithm for connected component detection.

2. Unity Viewer (3D Rendering)

Purpose: Render large-scale 3D environments with real-time data overlays, support VR interaction, and communicate with web UI layer.

Key Technologies: Unity 2022.3 LTS, Universal Render Pipeline, ECS/DOTS, Zenject

Design Patterns:

• Entity Component System: Data-oriented design for performance-critical systems
• Dependency Injection: Zenject container for service location and lifecycle management
• Observer Pattern: Reactive Extensions (Rx) for event streams
• Command Pattern: VR controller actions as commands
• Singleton Pattern: Managers for global services (PlayerManager, CadNodeManager, EmissionManager)

Integration Points:

• Input: Unity AssetBundles from THOR, glTF/b3dm files, CAD index binary files
• Output: Rendered frames, VR tracking data, screen analysis results
• Communication: Blazor Web App (via custom web components), Embedded Browser (via ZFBrowser)

Screen Analysis System (DOTS)

The screen analysis system identifies all CAD assets visible on screen in a given frame. This drives the asset status highlighting feature.

How it works:

1. Frustum Culling: Filter entities to those within camera frustum (DOTS job)
2. Occlusion Culling: Query GPU for visibility (async)
3. Entity → CAD Index Mapping: Convert visible entities to CAD indexes (DOTS job)
4. Aggregation: Collect results into a NativeList<OnScreenAsset>

Performance: Processes ~100,000 entities in <2ms on modern hardware

// Simplified screen analysis job
[BurstCompile]
struct ScreenAnalysisJob : IJobChunk
{
    [ReadOnly] public ComponentTypeHandle<LocalToWorld> TransformType;
    [ReadOnly] public ComponentTypeHandle<RenderBounds> BoundsType;
    [ReadOnly] public ComponentTypeHandle<CadIndexComponent> CadIndexType;
    [ReadOnly] public FrustumPlanes Frustum;

    public NativeList<OnScreenAsset>.ParallelWriter Results;

    public void Execute(ArchetypeChunk chunk, int chunkIndex, int firstEntityIndex)
    {
        var transforms = chunk.GetNativeArray(TransformType);
        var bounds = chunk.GetNativeArray(BoundsType);
        var cadIndices = chunk.GetNativeArray(CadIndexType);

        for (int i = 0; i < chunk.Count; i++)
        {
            var worldBounds = AABB.Transform(transforms[i].Value, bounds[i].Value);
            if (Frustum.Intersect(worldBounds))
            {
                Results.AddNoResize(new OnScreenAsset
                {
                    CadIndex = cadIndices[i].Value,
                    ScreenPosition = WorldToScreen(worldBounds.Center)
                });
            }
        }
    }
}

3. Blazor Web Application

Purpose: Provide interactive web UI, integrate external data sources, display asset status information, and communicate with Unity Viewer.

Key Technologies: Blazor Server, SignalR, JavaScript Interop, Reactive Extensions

Design Patterns:

• MVVM Pattern: Blazor components as views, C# classes as view models
• Repository Pattern: Service classes encapsulate data access
• Factory Pattern: Service providers create scoped instances
• Observer Pattern: Rx observables for real-time data streams
• Circuit Breaker Pattern: Resilient HTTP clients with Polly

Integration Points:

• Input: REST APIs (AHM, Equipment, SCADA), MQTT brokers, PowerBI, Lucene search
• Output: HTML/CSS/JavaScript to browser, SignalR messages to Unity
• Communication: Unity Viewer (via JavaScript web components)

Asset Status Highlighting Implementation

This feature demonstrates the complexity and elegance of the InDI architecture:

1. User navigates to Asset Status page in Blazor
2. Blazor appends <screen-analysis> custom element to DOM
3. Unity detects element via mutation observer, creates Indi3DScreenAnalysis proxy
4. User moves camera in Unity
5. Unity runs DOTS screen analysis job, collects visible CAD indexes
6. Unity executes JavaScript via ZFBrowser: screenAnalysis.onScreenAnalysisResults(json)
7. JavaScript forwards data to SharedWorker for position calculations
8. JavaScript invokes Blazor: blazorRef.invokeMethodAsync('onObjectsOnScreenChanged', json)
9. Blazor queries Equipment API to map asset IDs → functional locations
10. Blazor queries AHM API (via cache) to get health status for each functional location
11. Blazor generates IndiCardData and serializes to JSON
12. Blazor sends JSON to JavaScript: jsRuntime.InvokeVoidAsync('indicards.onCardData', json)
13. JavaScript updates card DOM elements with new data and positions
14. JavaScript updates <indi-highlighting> elements with colors
15. Unity applies emission colors via DOTS batch update

4. Storage Abstraction Layer

Purpose: Provide a unified interface for interacting with diverse storage backends (local file system, Azure, AWS S3, MinIO) without vendor lock-in.

Key Technologies: .NET 8, Azure.Storage.Blobs, AWSSDK.S3

Design Patterns:

• Strategy Pattern: Swap storage implementations at runtime via configuration
• Factory Pattern: IndiStorageProvider constructs appropriate storage instances
• Repository Pattern: A unified IStorage API hides underlying storage details

Integration Points:

• Input: Configuration from appsettings.json defining storage keys and endpoints
• Output: IStorage instances for blob manipulation (list, get, upload, delete)

Configuration Example:

{
  "Endpoints": {
    "Tileset": { "StorageKey": "Azure.Production", "RelativePath": "Tilesets" },
    "AssetData": { "StorageKey": "MinIO.Dev", "RelativePath": "AssetData" },
    "Model3d": { "StorageKey": "Local.Content", "RelativePath": "Indexes/Model3d" }
  }
}

5. Search System (Lucene)

Purpose: Provide fast full-text search across millions of CAD assets with hierarchical queries.

Key Technologies: Lucene.NET, Memory-Mapped Files

Design Patterns:

• Index Pattern: Pre-computed data structures for O(log n) lookups
• Document Store Pattern: Denormalized data for read optimization
• Repository Pattern: IAssetObjectQuery abstracts Lucene details

Index Structure: Each CAD asset is a Lucene document with fields for Name, Description, Category, Type, and Path. The Path field enables efficient hierarchical queries (e.g., find all pumps under Area 51).

Performance Optimizations:

• Memory-Mapped Indexes: OS-managed caching for frequently accessed index pages
• Term Vectors: Store term positions for fast highlighting
• Compound Files: Bundle index files to reduce file handles
• Warm-Up Queries: Execute common queries on startup to populate OS page cache

6. MQTT Real-Time Integration

Purpose: Ingest raw telemetry from industrial IoT sensors and PLCs, transform it, and route it to visualization clients.

Key Technologies: MQTTnet, .NET Worker Service, gRPC

Design Patterns:

• Message Broker Pattern: Decouple data producers (sensors) from consumers (clients)
• ETL (Extract, Transform, Load): A message transformation pipeline subscribes to MQTT topics, applies business logic, and publishes enriched messages

Design Patterns and Best Practices

Architectural Patterns

• Pipeline Pattern (THOR): Processing steps connected via typed channels for clear data flow and parallelism.
• Entity Component System (Unity Viewer): Data-oriented design with DOTS/ECS for cache-friendly memory layout and scalability to millions of entities.
• Repository Pattern (Blazor Web App): Service classes encapsulate data access for testability and flexibility.
• Observer Pattern (Reactive Extensions): Composable event streams for declarative event handling and subscription management.
• Dependency Injection (Zenject & ASP.NET Core): Inversion of Control for loose coupling, testability, and lifecycle management.

Code Organization

• Vertical Slice Architecture: Feature folders group related concerns (UI, code-behind, services, models).
• Namespace Alignment: Namespaces match folder structure for intuitive navigation.
• Separation of Concerns: Domain models have no dependencies; implementations depend on abstractions.

Testing Strategies

A multi-layered testing strategy with 15 test projects:

• Unit Tests (NUnit): >80% code coverage for business logic with NSubstitute for mocking.
• Integration Tests (WebApplicationFactory): Multi-component testing with TestContainers for database dependencies.
• Component Tests (BUnit): Blazor UI component testing with rendered output verification.
• End-to-End Tests (Playwright/Selenium): Critical user workflow testing with browser automation.

Error Handling

• Global Exception Middleware: Catches unhandled exceptions and logs to Application Insights.
• Circuit Breaker Pattern (Polly): Resilient HTTP clients for external APIs with automatic retry and exponential backoff.
• Result Pattern: For critical operations, returning explicit success/failure states instead of throwing exceptions.

Notable Technical Achievements

1. Scalability to Millions of Assets

• Challenge: Oil & gas facilities can have 5-10 million individual CAD assets.
• Solution: A multi-pronged approach combining spatial subdivision (tilesets), LOD generation, occlusion culling, instancing, and DOTS/ECS processing.
• Result: 60 FPS with 10 million assets on mid-range hardware (RTX 3060).

2. Sub-Second Search Across Massive Hierarchies

• Challenge: Users expect Google-like search speed on CAD hierarchies with millions of nodes.
• Solution: Lucene-based indexing with hierarchical path fields, prefix trees, and memory-mapped files.
• Result: <100ms search time for queries across 5 million assets.

3. Real-Time Asset Status with Low API Overhead

• Challenge: External APIs are slow, lack bulk support, and have rate limits. Querying 100 assets could take 50+ seconds.
• Solution: Multi-layer caching, background data synchronization, Reactive Extensions for batching, and speculative prefetching.
• Result: <2 second latency from camera movement to status visualization (95th percentile).

4. Seamless Multi-Layer JavaScript Interop

• Challenge: Three separate systems (Blazor Server, JavaScript, Unity Viewer) need to communicate in real-time.
• Solution: A custom web component system using mutation observers and direct DOM manipulation via an embedded browser.
• Result: <16ms end-to-end latency for most interactions.

5. Network Graph Alignment for 3D Piping

• Challenge: P&ID diagrams and 3D scans describe the same piping network in different formats and coordinate systems.
• Solution: Dual graph construction (one from P&ID, one from 3D point cloud) and automatic alignment using graph isomorphism algorithms.
• Result: 90%+ automatic tagging accuracy with human-in-the-loop validation.

6. Memory-Efficient CAD Index Lookups

• Challenge: Mapping CAD indexes to metadata requires random access to millions of records, which is impractical to load into RAM.
• Solution: Memory-mapped binary files with fixed-width records.
• Result: O(1) lookup time (~100 nanoseconds) with minimal RAM footprint.

Deployment and Operations

Deployment Architecture

The platform supports both cloud-native (Azure, AWS) and on-premises deployments. A hybrid cloud model is common, leveraging different providers for different storage types (e.g., Azure for indexes, S3 for tilesets, MinIO for models).

CI/CD Pipeline

• Source Control: Git with trunk-based development (dev, test, prod branches).
• Build Automation: Nuke.Build for type-safe build scripts in C#.
• Pipelines: Azure DevOps for CI/CD orchestration.
• Release Strategy: Staging slots for zero-downtime deployments with smoke tests and automatic rollbacks.

Monitoring and Observability

• Application Insights: Request telemetry, exception tracking, custom metrics, and user flows.
• Unity Telemetry: Custom events and performance counters sent to Application Insights.
• Alerts: Configured for high error rates, slow response times, API failures, and resource exhaustion.

Conclusion

InDI represents a sophisticated integration of 3D graphics, web technologies, real-time data, and enterprise systems. It demonstrates mastery of performance-critical systems, distributed architecture, modern web development, 3D graphics, and data engineering.

The project showcases the ability to:

• Design scalable architectures that handle millions of entities
• Integrate disparate technologies (Unity, Blazor, external APIs)
• Solve complex technical challenges (graph alignment, real-time highlighting)
• Deliver business value (reduced training costs, faster incident response)
• Maintain code quality (testing, CI/CD, monitoring)

InDI is not just a technical achievement—it’s a practical solution to real industrial problems, deployed in production at some of the world’s largest mining and energy companies.

Appendix: C4 Diagrams

A. System Context Diagram

B. Container Diagram