Filter by type:
Filter by tags:
qwen3.8-flash-next-coder-iq1_m
Qwen3.8-Flash-Next, a 125B-parameter hybrid mixture-of-experts model, in
ISTA-DASLab's Coder release (IQ1_M): half of the experts (256 of 512 per
layer) are kept, chosen on code, agent and image data, so it is smaller and
faster but weaker outside code and in languages other than English (Chinese
and other CJK text included) (about 59 GB to download).
It reads images through the vision projector published for the original model,
and the Strata backend parses its reasoning and tool calls.
Needs about 32 GB of RAM (23 GB of experts in RAM) with a suitable GPU, and
fast SSD storage: the 29 GB n-gram table stays on disk. Needs a GPU the strata
backend supports; NVIDIA with 12 GB of VRAM or more is the class Strata tests.
Runs only on the experimental strata backend.
On the first load the strata backend builds the Strata pack from the GGUF
files (a few minutes, a few GB of extra disk); a finished pack is never
rebuilt. The experts are read from the GGUF files in place (mapped); add
--resident-experts to engine_args to copy them into RAM when there is room.
Licensed under the Qwen Community License 1.0 (inherited from
Qwen/Qwen3.8-Flash-Next): free use, but a Model as a Service or AI Work
Assistant business needs a licence from Qwen. Read the LICENSE file in the
repository first.
The GGUF files are the Coder GSQ-RCO quantization by ISTA-DASLab; Strata is by
Niko1221. The MTP draft layer is not configured here.
Repository: localaiLicense: qwen-community-1.0
qwen3.8-flash-next-gyro-s
Qwen3.8-Flash-Next, a 125B-parameter hybrid mixture-of-experts model, in
agentionai's Gyro-S quantization (trellis-coded experts), the smallest option,
for machines with little memory and lower in quality (about 59 GB to
download).
It reads images through the vision projector published for the original model,
and the Strata backend parses its reasoning and tool calls.
Needs about 28 GB of VRAM and RAM together for the weights. Half of the file
is the n-gram table, which stays on disk and is read for every token, so keep
it on a fast local SSD. Needs a GPU the strata backend supports; NVIDIA with
12 GB of VRAM or more is the class Strata tests. Runs only on the experimental
strata backend.
On the first load the strata backend builds the Strata pack from the GGUF
files (a few minutes, a few GB of extra disk); a finished pack is never
rebuilt. The experts are read from the GGUF files in place (mapped); add
--resident-experts to engine_args to copy them into RAM when there is room.
Licensed under the Qwen Community License 1.0 (inherited from
Qwen/Qwen3.8-Flash-Next): free use, but a Model as a Service or AI Work
Assistant business needs a licence from Qwen. Read the LICENSE file in the
repository first.
The GGUF file is the Gyro quantization by agentionai; the vision projector is
ISTA-DASLab's file for the original model; Strata is by Niko1221. The MTP
draft layer is not configured here.
Repository: localaiLicense: qwen-community-1.0
qwen3.8-flash-next-gyro-m
Qwen3.8-Flash-Next, a 125B-parameter hybrid mixture-of-experts model, in
agentionai's Gyro-M quantization (trellis-coded experts), about IQ3_XXS
quality with about 20% less GPU memory, 16 to 21% slower than Q2_0 (about 93
GB to download).
It reads images through the vision projector published for the original model,
and the Strata backend parses its reasoning and tool calls.
Needs about 35 GB of VRAM and RAM together for the weights. Half of the file
is the n-gram table, which stays on disk and is read for every token, so keep
it on a fast local SSD. Needs a GPU the strata backend supports; NVIDIA with
12 GB of VRAM or more is the class Strata tests. Runs only on the experimental
strata backend.
On the first load the strata backend builds the Strata pack from the GGUF
files (a few minutes, a few GB of extra disk); a finished pack is never
rebuilt. The experts are read from the GGUF files in place (mapped); add
--resident-experts to engine_args to copy them into RAM when there is room.
Licensed under the Qwen Community License 1.0 (inherited from
Qwen/Qwen3.8-Flash-Next): free use, but a Model as a Service or AI Work
Assistant business needs a licence from Qwen. Read the LICENSE file in the
repository first.
The GGUF file is the Gyro quantization by agentionai; the vision projector is
ISTA-DASLab's file for the original model; Strata is by Niko1221. The MTP
draft layer is not configured here.
Repository: localaiLicense: qwen-community-1.0
deepseek-v4.1-flash-ista-iq2_xxs
DeepSeek-V4.1-Flash-1017, a 763B-parameter mixture-of-experts model (384 experts
per layer, 6 used per token), in ISTA-DAS's 2.00 bits per weight GSQ-RCO
quantization (IQ2_XXS), the middle size of the three (about 257 GB to download).
It writes a reasoning trace before the answer, and the Strata backend splits it
from the reply and parses the model's DSML tool calls. Text only: images are not
supported by this backend.
Needs a GPU with 12 GB of VRAM or more and about 40 GB of RAM at the least
(Strata's figure; its measurements use 46 to 64 GB: an RTX 5070 with 64 GB
answered at about 5 tokens per second, an RTX 3060 with 46 GB at 8 to 10). Less
RAM still runs, slowly. Strata measured a perplexity 15.5% above the original
model on 4,995 positions (code 8.6%, agent work 5.3%).
Needs about 258 GB of free disk, on a fast NVMe drive: the 111 GB n-gram table
(Engram) stays on disk and is read for every token. The Engram table and the 8
GB DSpark draft model are shared by all DeepSeek-V4.1 entries and downloaded
once into strata/models/deepseek-v4.1-flash-shared.
Runs only on the experimental strata backend, on NVIDIA CUDA 13 (x86_64 and
arm64), AMD Radeon RX 9070 and R9700 (ROCm) and Apple Silicon (Metal). Strata
ships no DeepSeek engine for CUDA 12, Intel or other AMD cards.
On the first load the strata backend builds the Strata pack from the GGUF files
(under a minute, a few MB of extra disk); loading the model afterwards takes a
few minutes. The default context is 8192 tokens, the size Strata checked; raise
context_size if the GPU has room.
Licensed under the MIT License (DeepSeek-V4.1-Flash-1017 and the quantized
files). The GGUF files are the GSQ-RCO quantization by ISTA-DAS (ISTA-DASLab);
the DSpark draft model is Unsloth's; Strata is by Niko1221.
Repository: localaiLicense: mit
deepseek-v4.1-flash-ista-iq2_xs
DeepSeek-V4.1-Flash-1017, a 763B-parameter mixture-of-experts model (384 experts
per layer, 6 used per token), in ISTA-DAS's 2.25 bits per weight GSQ-RCO
quantization (IQ2_XS), the largest size of the three (about 274 GB to download).
It writes a reasoning trace before the answer, and the Strata backend splits it
from the reply and parses the model's DSML tool calls. Text only: images are not
supported by this backend.
Needs a GPU with 12 GB of VRAM or more and about 44 GB of RAM at the least
(Strata's figure; its measurements use 46 to 64 GB: an RTX 5070 with 64 GB
answered at about 5 tokens per second, an RTX 3060 with 46 GB at 8 to 10). Less
RAM still runs, slowly. Strata measured a perplexity 7.3% above the original
model on 4,995 positions (code 3.2%, agent work 2.2%).
Needs about 275 GB of free disk, on a fast NVMe drive: the 111 GB n-gram table
(Engram) stays on disk and is read for every token. The Engram table and the 8
GB DSpark draft model are shared by all DeepSeek-V4.1 entries and downloaded
once into strata/models/deepseek-v4.1-flash-shared.
Runs only on the experimental strata backend, on NVIDIA CUDA 13 (x86_64 and
arm64), AMD Radeon RX 9070 and R9700 (ROCm) and Apple Silicon (Metal). Strata
ships no DeepSeek engine for CUDA 12, Intel or other AMD cards.
On the first load the strata backend builds the Strata pack from the GGUF files
(under a minute, a few MB of extra disk); loading the model afterwards takes a
few minutes. The default context is 8192 tokens, the size Strata checked; raise
context_size if the GPU has room.
Licensed under the MIT License (DeepSeek-V4.1-Flash-1017 and the quantized
files). The GGUF files are the GSQ-RCO quantization by ISTA-DAS (ISTA-DASLab);
the DSpark draft model is Unsloth's; Strata is by Niko1221.
Repository: localaiLicense: mit
deepseek-v4.1-flash-ista-coder-iq2_xxs
ISTA-DAS's coding version of DeepSeek-V4.1-Flash-1017: in every layer 192 of the
384 routed experts are kept and the other 192 removed, stored in the 2.00 bpw
(IQ2_XXS) layout. It is the smallest DeepSeek-V4.1 set here (about 191 GB to
download).
It writes a reasoning trace before the answer, and the Strata backend splits it
from the reply and parses the model's DSML tool calls. Text only: images are not
supported by this backend.
Needs a GPU with 12 GB of VRAM or more and about 24 GB of RAM at the least
(Strata's figure; its measurements use 46 to 64 GB: an RTX 5070 with 64 GB
answered at about 5 tokens per second, an RTX 3060 with 46 GB at 8 to 10). Less
RAM still runs, slowly. Strata measured a perplexity 48% above the original
model on its test set (KL 0.40), so it is weaker than the full sets on prose and
general knowledge; it is meant for code and agent work.
Needs about 192 GB of free disk, on a fast NVMe drive: the 111 GB n-gram table
(Engram) stays on disk and is read for every token. The Engram table and the 8
GB DSpark draft model are shared by all DeepSeek-V4.1 entries and downloaded
once into strata/models/deepseek-v4.1-flash-shared.
Runs only on the experimental strata backend, on NVIDIA CUDA 13 (x86_64 and
arm64), AMD Radeon RX 9070 and R9700 (ROCm) and Apple Silicon (Metal). Strata
ships no DeepSeek engine for CUDA 12, Intel or other AMD cards.
On the first load the strata backend builds the Strata pack from the GGUF files
(under a minute, a few MB of extra disk); loading the model afterwards takes a
few minutes. The default context is 8192 tokens, the size Strata checked; raise
context_size if the GPU has room.
Licensed under the MIT License (DeepSeek-V4.1-Flash-1017 and the quantized
files). The GGUF files are the GSQ-RCO quantization by ISTA-DAS (ISTA-DASLab);
the DSpark draft model is Unsloth's; Strata is by Niko1221.
Repository: localaiLicense: mit
qwen3.5-4b-3d-coder-q8
CCORE's early preview fine-tune of Qwen3.5-4B for coding and Three.js
scene generation. This text-only Q8_0 build pins a preview
checkpoint and uses the embedded chat template. MTP is not enabled.
Repository: localaiLicense: apache-2.0
qwen3.5-0.8b-artemis-q5
TurinTech Artemis Q5_K_M quantization of Qwen3.5 0.8B for text chat,
reasoning, and tool use. This text-only build omits the vision encoder
and MTP layer. Uses the embedded chat template and an 8192-token context.
Repository: localaiLicense: apache-2.0
qwen3.5-0.8b-artemis-q4
TurinTech Artemis Q4_K_M quantization of Qwen3.5 0.8B for text chat,
reasoning, and tool use. This text-only build omits the vision encoder
and MTP layer. Uses the embedded chat template and an 8192-token context.
Repository: localaiLicense: apache-2.0
simplicio-27b-q4
Simplicio-27B is a Qwen3.8-27B fine-tune for code editing and software engineering.
This Q4_K_M GGUF build uses llama.cpp and includes the F16 vision projector.
Repository: localaiLicense: apache-2.0
simplicio-27b-q8
Simplicio-27B is a Qwen3.8-27B fine-tune for code editing and software engineering.
This Q8_0 GGUF build uses llama.cpp and includes the F16 vision projector.
Repository: localaiLicense: apache-2.0
deepseek-v4-flash-vision-exp
# DeepSeek-V4-Flash-Vision-Exp
## Introduction
We are excited to introduce **DeepSeek-V4-Flash-Vision-Exp**, our first experimental multimodal model in the DeepSeek-V4 family. It builds on the DeepSeek-V4-Flash architecture by incorporating visual modules and undergoing continued training to unlock visual understanding capabilities.
Compared to DeepSeek-V4-Flash-0731, DeepSeek-V4-Flash-Vision-Exp achieves substantial improvements on its multimodal agent capabilities, while maintaining comparable performance on text-only agent tasks.
Notes:
1. For the text agent benchmarks above, DeepSeek models are evaluated with the minimal mode of DeepSeek Harness as the agent framework, using the `max` reasoning effort level with `temperature = 1.0, top_p = 0.95`.
2. † For ApexBench and Agents' Last Exam, DeepSeek-V4-Flash-0731 ignores the multimodal elements in the input.
## Repository layout
This repository contains the tokenizer, prompt encoding reference, and a
minimal PyTorch inference implementation for DeepSeek-V4 Flash Vision. The
reference inference covers the vision encoder and aligner, DFlash attention,
MoE, Hyper-Connections, and the DSpark forward path.
...
Repository: localaiLicense: mit
qwopus3.8-27b-flash
Qwopus3.8-27B-Flash is a Qwen3.8-27B fine-tune for reasoning and agent
workloads. This Q4_K_M GGUF includes the F32 vision projector and uses
llama.cpp's embedded chat template with MTP speculative decoding.
The publisher reports a known Python code indentation issue.
Repository: localaiLicense: apache-2.0
qwopus3.8-27b-flash-q8
Qwopus3.8-27B-Flash is a Qwen3.8-27B fine-tune for reasoning and agent
workloads. This Q8_0 GGUF includes the F32 vision projector and uses
llama.cpp's embedded chat template with MTP speculative decoding.
The publisher reports a known Python code indentation issue.
Repository: localaiLicense: apache-2.0
qwen3.8-27b-turbo-fable-cold-fusion-735-882-heretic-uncensored-neo-coder-max-mtp
RELEASE #1 GGUFS [including detailed notes, how to use, benches and much more]:
https://huggingface.co/DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NEO-CODER-MAX-MTP-GGUF
Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU (release #1, others pending...)
( repo has 10+ other versions (and 3 branches) noted below that EXCEED the performance of all QWEN 27B models, including fine tunes. )
First, special thanks to Nightmedia for working on the first three stages prior to heretic'ing/post staging and benching everything (3 sections below).
A number of my finetunes - both released and non-released - were used here as well as some third parties. Full details will be disclosed upon final release as the project shores up.
THREE example generations [snippets] from STAGE1-PART2, STAGE1b-PART2 and STAGE2-rplus2 at the bottom of the page.
Release(s) will be GGUFS first (linked here directly) then source code shortly thereafter [now released/open].
Some additional work and/ spawning of new branches from branch(es) below is still going on.
NEW: Branch 3 added, see below.
COMPLETED AND PENDING RELEASES:
...
Repository: localaiLicense: apache-2.0
dfm-mimir:vllm
DFM Mimir is an Apache-2.0, instruction-tuned HRM-Text model from Danish
Foundation Models. It has about 1 billion parameters and a 4,096-token
context window. The model focuses on Danish and English chat, reasoning,
mathematics, and code generation, and uses only permissible post-training
data. This entry serves the official BF16 safetensors checkpoint with vLLM.
Repository: localaiLicense: apache-2.0
cyber-tiel-coder-35b-a3b-q4-mtp
Cyber-Tiel-Coder is a 35B mixture-of-experts coding model with 3B active parameters,
based on Huihui's abliterated Ornith-1.5. This UD-Q4_K_XL build includes
MTP speculative decoding, the embedded Sharp chat template, and a BF16 vision projector.
Repository: localaiLicense: mit
cyber-tiel-coder-35b-a3b-q8-mtp
Cyber-Tiel-Coder is a 35B mixture-of-experts coding model with 3B active parameters,
based on Huihui's abliterated Ornith-1.5. This UD-Q8_K_XL build includes
MTP speculative decoding, the embedded Sharp chat template, and a BF16 vision projector.
Repository: localaiLicense: mit
tiel-coder-35b-a3b-q4
Tiel-Coder-35B-A3B is a 35B-parameter mixture-of-experts model for coding,
reasoning, tool use, and vision tasks. This default entry uses the
Q4_K_XL GGUF and BF16 vision projector.
Repository: localaiLicense: mit
tiel-coder-35b-a3b-q4-mtp
Tiel-Coder-35B-A3B in Q4_K_XL format with MTP speculative decoding and a
BF16 vision projector.
Repository: localaiLicense: mit
tiel-coder-35b-a3b-q5-mtp
Tiel-Coder-35B-A3B in Q5_K_XL format with MTP speculative decoding and a
BF16 vision projector.
Repository: localaiLicense: mit
Page 1