TELDAR AI
LAUSANNE · CH
TELDAR AI combines approximately 30 billion parameters of specialised model capacity through a sparse expert architecture. This technical report details the model foundation, shared attention topology, parameter-efficient fine-tuning protocols, and internal domain validation.
TELDAR AI builds directly upon open foundation research, integrating domain-specific architectural innovations rather than relying on commercial third-party APIs.
TELDAR AI is based on the Apertus 8B family, with current engineering work utilizing Apertus v1.5 8B as the primary base. Prior to training specialized domain branches, a reasoning and logical adaptation stage was integrated into the foundational base checkpoint.
The engineering work utilizes a merged base checkpoint running directly on dedicated infrastructure. TELDAR develops its own specialized sovereign architecture on top of and around the Apertus foundation model—including domain dataset synthesis, parameter-efficient fine-tuning, feed-forward expert isolation, selective routing layers, and sovereign deployment runtimes.
Scientific Attribution & Scope: Apertus is an independent open foundation model family. TELDAR does not claim to have created Apertus itself; rather, TELDAR engineers the sovereign sparse expert architecture, training pipelines, legal and financial corpora, and on-premise execution layers built around it.
A shared attention backbone maintains coherent general language and reasoning behaviour, while domain-specific feed-forward capacity provides specialised expertise.
An important architectural principle in TELDAR AI is the avoidance of unnecessary parameter duplication. If an organization maintains four completely separate 8B models, it quadruples infrastructure overhead while needlessly repeating identical syntactic and general reasoning weights across all models.
Instead, TELDAR AI structures the transformer's 32 layers into two distinct functional regimes:
Multi-head self-attention mechanisms remain shared across all inference operations. This guarantees consistent context parsing, linguistic fluency, and cross-domain reasoning.
Domain-specific fine-tuning is isolated within the feed-forward network layers. Expert delta detection isolates specialist weight shifts to FFN projections while preserving the shared attention backbone.
Total expert capacity scales to ~30B parameters, yet runtime resource demands correspond only to the actively routed expert path.
Each branch addresses high-value, highly regulated enterprise domains that require precise terminology, deep structural understanding, and absolute data confidentiality.
Engineered for legal document analysis, contractual interpretation, compliance verification, and statutory comprehension. Grounded in Swiss case law, civil legislation, and cantonal precedent.
Trained on corporate balance sheets, audit guidelines, market filings, and specialized financial language. Capable of analyzing complex institutional statements without public disclosure risks.
Commercial strategy, client communication, multilingual brand narratives, and enterprise proposition synthesis across European commercial markets.
Syntax comprehension, enterprise code refactoring, infrastructure-as-code scripting, and technical debugging across modern backend languages and systems.
Expert development employs parameter-efficient fine-tuning (PEFT) targeting transformer feed-forward projection matrices on modern NVIDIA accelerator clusters.
To maintain absolute stability in the base model's generalized reasoning capabilities while imparting deep sector-specific competence, the adaptation protocol isolates gradient updates strictly to designated Feed-Forward Network projection layers.
The expert training runs use a low-rank adaptation (LoRA) configuration with rank 32 applied across 64 FFN targets. This isolates approximately 52.4 million trainable parameters per specialized expert run, preserving model convergence without catastrophic forgetting of general language features.
Training Context vs Inference Window: The current expert-specialisation runs utilize an effective sequence length of 1024 tokens with an effective batch size of 32. This training sequence length is an engineering hyperparameter optimized for loss convergence during domain adaptation and does not constrain the maximum inference context of the deployed runtime.
| Adaptation Technique | LoRA / PEFT |
|---|---|
| Target Modules | FFN Projections (64 targets) |
| LoRA Rank (r) | 32 |
| Trainable Params | ~52.4 Million / expert run |
| Training Context Length | 1024 tokens |
| Effective Batch Size | 32 |
| Compute Infrastructure | NVIDIA GB10 / DGX Spark-class |
The legal expert represents our most thoroughly documented domain specialisation. Below are the verified metrics and methodology from our internal domain validation runs.
The legal data pipeline incorporated authoritative Swiss legal material including Swiss case law, federal and cantonal legislation, parliamentary debate records, and extensive structured legal commentary.
A domain-specific question-and-answer generation pipeline produced 197,574 strictly unique validated legal Q&A examples after multi-pass deduplication and semantic hygiene filtering.
57,000 training examples and 3,000 validation examples (5% split) across 43,395 source documents. Strict document-level leakage controls limited selection to a maximum of 4 examples per source document.
137,574 training rows and 3,000 validation rows. Validated on NVIDIA GPU infrastructure with effective batch size 32 and sequence length 1024.
* Measured internal convergence and continuation metrics. Not third-party synthetic benchmarks.
A routing layer determines which specialised capacity should participate in inference, allowing TELDAR AI to combine general intelligence with domain-specific expertise without activating every expert for every task.
In dense LLMs, all parameters must compute intermediate vectors for every generated token. TELDAR AI's sparse approach selectively pathways queries to the relevant domain FFN expert, optimizing memory bandwidth and enabling low-latency execution on enterprise GPU hardware.
Deployable directly on enterprise-controlled hardware (such as NVIDIA GB10 or DGX Spark-class systems) or private Kubernetes clusters. Inference runs with zero external internet dependencies or telemetry callbacks.
A comprehensive overview of foundational parameters, transformer dimensions, adaptation rank, and deployment requirements.
| Foundation Model Family | Apertus 8B family |
|---|---|
| Current Engineering Base | Apertus v1.5 8B (integrated reasoning adaptation) |
| Architectural Design | Sparse specialised expert model |
| Expert Domains | LAW / FINANCE / MARKETING / CODE |
| Combined Specialised Capacity | ~30 billion parameters |
| Transformer Layers | 32 layers |
| Attention Mechanism | Shared foundation across all domain branches |
| Domain Specialisation Target | Feed-Forward Network (FFN) projections (64 targets) |
| Expert Adaptation Protocol | LoRA / Parameter-Efficient Fine-Tuning (PEFT) |
| LoRA Rank (r) | 32 for FFN expert-specialisation runs |
| Trainable Expert Parameters | ~52.4 million parameters per specialised run |
| Training Sequence Context | 1024 tokens (domain training run length) |
| Corpus Engineering Scale | 215M+ records processed, deduplicated and filtered |
| GPU Platform | NVIDIA CUDA / GB10 / DGX Spark-class infrastructure |
| Deployment Capability | Customer-controlled / on-premise capable / private enclave |
| External Dependencies | None; zero mandatory commercial LLM API egress |