A New Standard for Building Semantic Layers for DWH

The AI agent for KPI & Dimension Registry (KDR) generation is a specialized tool that automates one of the most time-consuming and critical stages of building an analytical system: creating the business semantic layer of data. The product analyzes the structure of a data warehouse, extracts meaning from tables and fields, and transforms a technical model into a complete business-oriented registry ready for import into DataForge.

In traditional DWH projects, creating a KPI & Dimension Registry may take weeks or even months: analysts manually study table structures, align metric definitions, eliminate duplicates, and establish a unified data language. The AI agent removes this bottleneck by automatically generating a structured and validated registry that can immediately be used in analytics.

The product not only accelerates the process — it fundamentally changes the approach to data governance by making the semantic layer reproducible, scalable, and independent of specific individuals.

What the AI Agent Does

The AI agent performs the full cycle of transforming a technical DWH structure into business semantics, including classification, normalization, and KPI generation.

The input can be either DDL scripts (database structure) or a direct PostgreSQL connection. The system automatically analyzes tables, identifies their roles (facts, dimensions, reference entities), extracts key attributes, and builds a logical data model.

Unlike traditional tools, the solution uses a hybrid approach: deterministic algorithms (rules and heuristics) are applied first, followed by AI-powered semantic enrichment and interpretation. This ensures not only stable results but also a level of understanding comparable to that of an experienced analyst.

As a result, the system generates a complete KPI & Dimension Registry including:

• reference entities (logical groups of business objects),
• dimensions (analytical attributes),
• facts (numerical data),
• KPIs and metrics (business calculations and formulas).

How It Works (Process Architecture)

The process is designed as a managed data processing pipeline consisting of several logical stages.

First, the system loads and analyzes the source structure: tables, columns, data types, keys, and comments are extracted. Then the heuristic classification stage begins, where object roles are identified based on naming conventions, structure, and data types — for example, fact tables or dimensions.

Next, the AI model refines the classification, resolves ambiguities, and interprets the meaning of the data. This is especially important when table and field names are unclear or not standardized.

After that, the agent performs data curation: it выделяет dimensions, removes technical fields from facts, groups entities into reference structures, and builds the logical model. The system then generates business-friendly names and descriptions by automatically translating technical identifiers into business terminology.

The key stage is KPI design. The AI agent creates business metrics based on facts and dimensions, including formulas in DataForge syntax while ensuring consistency and correctness.

The final stages include normalization and validation: duplicate removal, formula verification, integrity checks, and relationship validation. Even if the AI cannot fully resolve a specific case, the system applies fallback algorithms to guarantee completion of the process.

Integration with DataForge

The AI agent was originally designed as part of the DataForge ecosystem and is fully compatible with its semantic layer format.

The generated result is exported into a standardized Excel file that can be directly imported into DataForge without additional processing. The file structure fully complies with platform requirements and contains all required entities and attributes.

This allows the tool to be seamlessly integrated into existing DWH and analytics workflows:

• accelerate deployment of new data marts,
• ensure KPI consistency,
• minimize manual analyst work,
• reduce time-to-insight.

In practice, the product becomes a bridge between the physical data model and business semantics.

Who This Product Is For

The AI agent is designed for teams working with analytical data and data warehouses:

• For DWH developers, it eliminates the need to manually document data structures and business semantics.
• For BI developers, it provides a ready-to-use and consistent KPI & dimension model for reporting.
• For business analysts, it creates a transparent and understandable metric structure, reducing discrepancies in calculations.
• For data architects, it serves as a tool for semantic layer standardization and scalability.

Key Value

The primary value of the product lies not only in automation, but also in standardizing the approach to data.

The AI agent enables organizations to:

• reduce KDR creation time from weeks to hours,
• eliminate metric inconsistencies between teams,
• formalize business logic explicitly,
• improve trust in data,
• ensure scalability of analytics.

It transforms a process traditionally dependent on individual expertise into a reproducible technology.

Why This Matters for Modern DWH Architectures

In modern data architectures (DWH, Lakehouse, Data Platforms), the semantic layer becomes a critical component.

Without it, organizations cannot:

• maintain consistent metrics,
• effectively use AI and NLQ,
• scale self-service analytics,
• avoid “report wars”.

The AI agent solves this problem at the source level by automatically creating the foundation for the semantic layer, which can then be used in DataForge and any BI system.