Fabric 1.5

Fabric 1.5 is an open-source small language model built by Fabric AI: a 0.7B-parameter, decoder-only causal transformer released under Apache 2.0 and hosted on Hugging Face. Its defining feature is Chunked Fabric Memory, a mechanism that compresses past context into learned chunk summaries and blends them with local attention through a learned gate, which lets the model reach 32,768-token contexts at sub-quadratic memory cost. Architecturally, Fabric 1.5 has 24 layers (16 LocalBlock plus 8 FabricMemoryBlock, with every third layer a memory block), a hidden dimension of 1,536, and SwiGLU feed-forward networks with intermediate size 4,096. LocalBlocks use Grouped Query Attention with a 2,048-token causal sliding window and Pre-RMSNorm residual connections. The model was trained on 12 billion tokens using 8 NVIDIA H100 GPUs on a single DGX node. Because the weights, system card, and license are all public, developers can download the model directly, quantize it, and self-host it rather than call a hosted API. It targets researchers and engineers who want a compact long-context model they can inspect, fine-tune, and run on modest hardware.

Best for ML engineers and researchers who need a compact, inspectable long-context model they can self-host and fine-tune.