What is Fabric 1.5?
Fabric 1.5 is an open-source small language model built by Fabric AI: a 0.7B-parameter, decoder-only causal transformer released under Apache 2.0 and hosted on Hugging Face. Its defining feature is Chunked Fabric Memory, a mechanism that compresses past context into learned chunk summaries and blends them with local attention through a learned gate, which lets the model reach 32,768-token contexts at sub-quadratic memory cost. Architecturally, Fabric 1.5 has 24 layers (16 LocalBlock plus 8 FabricMemoryBlock, with every third layer a memory block), a hidden dimension of 1,536, and SwiGLU feed-forward networks with intermediate size 4,096. LocalBlocks use Grouped Query Attention with a 2,048-token causal sliding window and Pre-RMSNorm residual connections. The model was trained on 12 billion tokens using 8 NVIDIA H100 GPUs on a single DGX node. Because the weights, system card, and license are all public, developers can download the model directly, quantize it, and self-host it rather than call a hosted API. It targets researchers and engineers who want a compact long-context model they can inspect, fine-tune, and run on modest hardware.
What is Fabric 1.5 used for?
Best for ML engineers and researchers who need a compact, inspectable long-context model they can self-host and fine-tune.
How much does Fabric 1.5 cost?
Open source (Apache 2.0)
Free to download, self-host, and fine-tune from Hugging Face under the Apache 2.0 license
Pros and cons
Pros
- Chunked Fabric Memory compresses past context into learned summaries, reaching 32,768-token contexts at sub-quadratic memory cost
- Small 0.7B-parameter footprint (24 layers, 1,536 hidden dim) that runs on modest hardware and supports quantization
- Apache 2.0 license with weights and a published system card on Hugging Face, so it can be self-hosted and fine-tuned freely
- Documented architecture: Grouped Query Attention with a 2,048-token sliding window and SwiGLU FFNs, trained on 12B tokens
Cons
- At 0.7B parameters, reasoning and knowledge depth are far below frontier-scale models
- Trained on only 12 billion tokens, which limits factual breadth compared with large-corpus models
- Chunked memory is a custom mechanism, so it needs the FabricMemoryBlock code path rather than plain transformer tooling
- No hosted API or managed endpoint published — you must run inference yourself
Questions about Fabric 1.5
What is Fabric 1.5?
Fabric 1.5 is a 0.7B-parameter open-source causal language model from Fabric AI, released under Apache 2.0 on Hugging Face, that uses Chunked Fabric Memory to handle 32,768-token contexts at sub-quadratic memory cost.
Is Fabric 1.5 free to use?
Yes. Fabric 1.5 is released under the Apache 2.0 license and its weights are available on Hugging Face, so you can download, self-host, quantize, and fine-tune it at no cost.
What makes Chunked Fabric Memory different?
Chunked Fabric Memory compresses past context into learned chunk summaries and blends them with local attention through a learned gate, enabling 32K-token contexts without the quadratic memory growth of standard full attention.
How large is Fabric 1.5 and how was it trained?
It is a 0.7B-parameter, 24-layer decoder-only transformer trained on 12 billion tokens using 8 NVIDIA H100 GPUs on a single DGX node.
What are the best alternatives to Fabric 1.5?
Reviews
Sign in to rate and review Fabric 1.5.
Sign in to reviewNo reviews yet. Be the first to review Fabric 1.5!