Filter by type:
Filter by tags:
qwen3.6-35b-a3b-ud-q4_k_xl-strata
Qwen3.6-35B-A3B, a 35B-parameter mixture-of-experts model with 3B active
parameters, in Unsloth's 4-bit (Unsloth Dynamic), the smallest GGUF with the
MTP layer (about 24 GB to download). It reads images through Unsloth's vision
projector, and the Strata backend parses its reasoning and tool calls.
Needs an NVIDIA or AMD GPU (the Intel Arc and Apple builds of Strata do not
run this model) and about 32 GB of RAM: the engine keeps every expert it can
in VRAM and computes the rest on the CPU. Runs only on the experimental strata
backend.
On the first load the strata backend exports the tokenizer and chat template
from the GGUF file (seconds); the engine reads the GGUF as it is. The MTP
draft layer inside the file is used for speculative decoding.
Licensed under the Apache License 2.0 (Qwen/Qwen3.6-35B-A3B). The GGUF files
are the Unsloth quantization; Strata is by Niko1221.
Repository: localaiLicense: apache-2.0
qwen3.6-35b-a3b-ud-q6_k_xl-strata
Qwen3.6-35B-A3B, a 35B-parameter mixture-of-experts model with 3B active
parameters, in Unsloth's 6-bit (Unsloth Dynamic), closer to the full model
GGUF with the MTP layer (about 34 GB to download). It reads images through
Unsloth's vision projector, and the Strata backend parses its reasoning and
tool calls.
Needs an NVIDIA or AMD GPU (the Intel Arc and Apple builds of Strata do not
run this model) and about 44 GB of RAM: the engine keeps every expert it can
in VRAM and computes the rest on the CPU. Runs only on the experimental strata
backend.
On the first load the strata backend exports the tokenizer and chat template
from the GGUF file (seconds); the engine reads the GGUF as it is. The MTP
draft layer inside the file is used for speculative decoding.
Licensed under the Apache License 2.0 (Qwen/Qwen3.6-35B-A3B). The GGUF files
are the Unsloth quantization; Strata is by Niko1221.
Repository: localaiLicense: apache-2.0
qwen3.6-35b-a3b-q8_0-strata
Qwen3.6-35B-A3B, a 35B-parameter mixture-of-experts model with 3B active
parameters, in Unsloth's 8-bit, the closest to the full model GGUF with the
MTP layer (about 39 GB to download). It reads images through Unsloth's vision
projector, and the Strata backend parses its reasoning and tool calls.
Needs an NVIDIA or AMD GPU (the Intel Arc and Apple builds of Strata do not
run this model) and about 52 GB of RAM: the engine keeps every expert it can
in VRAM and computes the rest on the CPU. Runs only on the experimental strata
backend.
On the first load the strata backend exports the tokenizer and chat template
from the GGUF file (seconds); the engine reads the GGUF as it is. The MTP
draft layer inside the file is used for speculative decoding.
Licensed under the Apache License 2.0 (Qwen/Qwen3.6-35B-A3B). The GGUF files
are the Unsloth quantization; Strata is by Niko1221.
Repository: localaiLicense: apache-2.0
deepseek-v4.1-flash-ista-iq1_m
DeepSeek-V4.1-Flash-1017, a 763B-parameter mixture-of-experts model (384 experts
per layer, 6 used per token), in ISTA-DAS's 1.75 bits per weight GSQ-RCO
quantization (IQ1_M), the smallest size of the three (about 239 GB to download).
It writes a reasoning trace before the answer, and the Strata backend splits it
from the reply and parses the model's DSML tool calls. Text only: images are not
supported by this backend.
Needs a GPU with 12 GB of VRAM or more and about 34 GB of RAM at the least
(Strata's figure; its measurements use 46 to 64 GB: an RTX 5070 with 64 GB
answered at about 5 tokens per second, an RTX 3060 with 46 GB at 8 to 10). Less
RAM still runs, slowly. Slightly lower quality than 2.00 bpw, and noticeably
weaker on plain English prose; ISTA-DASLab measured 98.7% of the base model on
its task average (AIME25, GPQA-Diamond, LiveCodeBench v6).
Needs about 240 GB of free disk, on a fast NVMe drive: the 111 GB n-gram table
(Engram) stays on disk and is read for every token. The Engram table and the 8
GB DSpark draft model are shared by all DeepSeek-V4.1 entries and downloaded
once into strata/models/deepseek-v4.1-flash-shared.
Runs only on the experimental strata backend, on NVIDIA CUDA 13 (x86_64 and
arm64), AMD Radeon RX 9070 and R9700 (ROCm) and Apple Silicon (Metal). Strata
ships no DeepSeek engine for CUDA 12, Intel or other AMD cards.
On the first load the strata backend builds the Strata pack from the GGUF files
(under a minute, a few MB of extra disk); loading the model afterwards takes a
few minutes. The default context is 8192 tokens, the size Strata checked; raise
context_size if the GPU has room.
Licensed under the MIT License (DeepSeek-V4.1-Flash-1017 and the quantized
files). The GGUF files are the GSQ-RCO quantization by ISTA-DAS (ISTA-DASLab);
the DSpark draft model is Unsloth's; Strata is by Niko1221.
Repository: localaiLicense: mit
deepseek-v4.1-flash-ista-iq2_xxs
DeepSeek-V4.1-Flash-1017, a 763B-parameter mixture-of-experts model (384 experts
per layer, 6 used per token), in ISTA-DAS's 2.00 bits per weight GSQ-RCO
quantization (IQ2_XXS), the middle size of the three (about 257 GB to download).
It writes a reasoning trace before the answer, and the Strata backend splits it
from the reply and parses the model's DSML tool calls. Text only: images are not
supported by this backend.
Needs a GPU with 12 GB of VRAM or more and about 40 GB of RAM at the least
(Strata's figure; its measurements use 46 to 64 GB: an RTX 5070 with 64 GB
answered at about 5 tokens per second, an RTX 3060 with 46 GB at 8 to 10). Less
RAM still runs, slowly. Strata measured a perplexity 15.5% above the original
model on 4,995 positions (code 8.6%, agent work 5.3%).
Needs about 258 GB of free disk, on a fast NVMe drive: the 111 GB n-gram table
(Engram) stays on disk and is read for every token. The Engram table and the 8
GB DSpark draft model are shared by all DeepSeek-V4.1 entries and downloaded
once into strata/models/deepseek-v4.1-flash-shared.
Runs only on the experimental strata backend, on NVIDIA CUDA 13 (x86_64 and
arm64), AMD Radeon RX 9070 and R9700 (ROCm) and Apple Silicon (Metal). Strata
ships no DeepSeek engine for CUDA 12, Intel or other AMD cards.
On the first load the strata backend builds the Strata pack from the GGUF files
(under a minute, a few MB of extra disk); loading the model afterwards takes a
few minutes. The default context is 8192 tokens, the size Strata checked; raise
context_size if the GPU has room.
Licensed under the MIT License (DeepSeek-V4.1-Flash-1017 and the quantized
files). The GGUF files are the GSQ-RCO quantization by ISTA-DAS (ISTA-DASLab);
the DSpark draft model is Unsloth's; Strata is by Niko1221.
Repository: localaiLicense: mit
deepseek-v4.1-flash-ista-iq2_xs
DeepSeek-V4.1-Flash-1017, a 763B-parameter mixture-of-experts model (384 experts
per layer, 6 used per token), in ISTA-DAS's 2.25 bits per weight GSQ-RCO
quantization (IQ2_XS), the largest size of the three (about 274 GB to download).
It writes a reasoning trace before the answer, and the Strata backend splits it
from the reply and parses the model's DSML tool calls. Text only: images are not
supported by this backend.
Needs a GPU with 12 GB of VRAM or more and about 44 GB of RAM at the least
(Strata's figure; its measurements use 46 to 64 GB: an RTX 5070 with 64 GB
answered at about 5 tokens per second, an RTX 3060 with 46 GB at 8 to 10). Less
RAM still runs, slowly. Strata measured a perplexity 7.3% above the original
model on 4,995 positions (code 3.2%, agent work 2.2%).
Needs about 275 GB of free disk, on a fast NVMe drive: the 111 GB n-gram table
(Engram) stays on disk and is read for every token. The Engram table and the 8
GB DSpark draft model are shared by all DeepSeek-V4.1 entries and downloaded
once into strata/models/deepseek-v4.1-flash-shared.
Runs only on the experimental strata backend, on NVIDIA CUDA 13 (x86_64 and
arm64), AMD Radeon RX 9070 and R9700 (ROCm) and Apple Silicon (Metal). Strata
ships no DeepSeek engine for CUDA 12, Intel or other AMD cards.
On the first load the strata backend builds the Strata pack from the GGUF files
(under a minute, a few MB of extra disk); loading the model afterwards takes a
few minutes. The default context is 8192 tokens, the size Strata checked; raise
context_size if the GPU has room.
Licensed under the MIT License (DeepSeek-V4.1-Flash-1017 and the quantized
files). The GGUF files are the GSQ-RCO quantization by ISTA-DAS (ISTA-DASLab);
the DSpark draft model is Unsloth's; Strata is by Niko1221.
Repository: localaiLicense: mit
deepseek-v4.1-flash-ista-coder-iq2_xxs
ISTA-DAS's coding version of DeepSeek-V4.1-Flash-1017: in every layer 192 of the
384 routed experts are kept and the other 192 removed, stored in the 2.00 bpw
(IQ2_XXS) layout. It is the smallest DeepSeek-V4.1 set here (about 191 GB to
download).
It writes a reasoning trace before the answer, and the Strata backend splits it
from the reply and parses the model's DSML tool calls. Text only: images are not
supported by this backend.
Needs a GPU with 12 GB of VRAM or more and about 24 GB of RAM at the least
(Strata's figure; its measurements use 46 to 64 GB: an RTX 5070 with 64 GB
answered at about 5 tokens per second, an RTX 3060 with 46 GB at 8 to 10). Less
RAM still runs, slowly. Strata measured a perplexity 48% above the original
model on its test set (KL 0.40), so it is weaker than the full sets on prose and
general knowledge; it is meant for code and agent work.
Needs about 192 GB of free disk, on a fast NVMe drive: the 111 GB n-gram table
(Engram) stays on disk and is read for every token. The Engram table and the 8
GB DSpark draft model are shared by all DeepSeek-V4.1 entries and downloaded
once into strata/models/deepseek-v4.1-flash-shared.
Runs only on the experimental strata backend, on NVIDIA CUDA 13 (x86_64 and
arm64), AMD Radeon RX 9070 and R9700 (ROCm) and Apple Silicon (Metal). Strata
ships no DeepSeek engine for CUDA 12, Intel or other AMD cards.
On the first load the strata backend builds the Strata pack from the GGUF files
(under a minute, a few MB of extra disk); loading the model afterwards takes a
few minutes. The default context is 8192 tokens, the size Strata checked; raise
context_size if the GPU has room.
Licensed under the MIT License (DeepSeek-V4.1-Flash-1017 and the quantized
files). The GGUF files are the GSQ-RCO quantization by ISTA-DAS (ISTA-DASLab);
the DSpark draft model is Unsloth's; Strata is by Niko1221.
Repository: localaiLicense: mit