DAREdare_tiesmultilingualgeneralllama3mistralmulti-model

Solar DARE — Llama 3 × Mistral DARE-TIES

DARE-TIES merge using random delta dropping to regularize a three-model combination of Llama-3 and two Mistral fine-tunes. Reduces interference between specialist models, producing one of the cleanest multi-model blends for general instruction following.

Author

mlabonne

Published

October 15, 2025

Last updated

March 1, 2026

Versions

2

Best Score

↑ 73.8

Stars

156

Mistral-7B-v0.1 · 7BMeta-Llama-3-8B-Instruct · 8BMistral-7B-Instruct-v0.3 · 7BMistral-Coder-7B · 7B
Share

Merge Lineage

Merge Lineage

4 source models

Source Models

Mistral-Coder-7B

7B

Output

Click any model node to open its Hugging Face page

Config YAML

solar-dare-llama-mistral-v1.1.yaml
merge_method: dare_ties
base_model: mistralai/Mistral-7B-v0.1
models: - model: meta-llama/Meta-Llama-3-8B-Instruct
    parameters: weight: 0.4
      density: 0.6
  - model: mistralai/Mistral-7B-Instruct-v0.3
    parameters: weight: 0.3
      density: 0.6
  - model: codestral/Mistral-Coder-7B
    parameters: weight: 0.3
      density: 0.5
parameters: normalize: true
dtype: bfloat16

Benchmark Scores

BenchmarkMergedLlama-3-8BMistral-7BMistral-CoderΔ Best
MMLUtop
73.871.970.165.3+1.9
HumanEval
67.061.060.263.0+4.0
MT-Bench
8.18.07.77.4+0.1
ARC-C
68.465.263.161.5+3.2

Model Weights & Density — DARE

Meta-Llama-3-8B-Instruct8B
weight
0.40
density
0.60
Mistral-7B-Instruct-v0.37B
weight
0.30
density
0.60
Mistral-Coder-7B7B
weight
0.30
density
0.50

How I Built This

Hugging FacePaperVideo

Embed Badge

Add this to your Hugging Face model card to link back to this recipe.

MergeKitRecipe
markdown
[![MergeKit Recipe](https://img.shields.io/badge/MergeKit-Recipe-10b981?style=flat-square)](https://www.mergekit.com/recipes/solar-dare-llama-mistral)

Version History

  1. v1.1latest
    ↑ 73.8

    March 1, 2026

    Added third model (Mistral-coding fine-tune) at weight 0.3, improved HumanEval +4 pts

  2. v1.0
    ↑ 71.2

    October 15, 2025

    Initial two-model DARE-TIES release

Use this Model

Run, deploy, or interact with Solar DARE — Llama 3 × Mistral DARE-TIES directly.

MergeKit Cloud

Run this model on serverless GPU infrastructure — zero setup, pay-per-second.

RunPod Serverless

Select GPU

A10G · 24 GB~45s cold
A100 · 80 GB~30s cold
H100 · 80 GB~20s cold
Est. ~$0.001 / inference · billed per second
Coming Soon

Serverless GPU · Powered by RunPod

Reproduce Locally

Run this exact merge on your own machine in three steps:

bash
pip install mergekit
yaml
merge_method: dare_ties
base_model: mistralai/Mistral-7B-v0.1
models:
  - model: meta-llama/Meta-Llama-3-8B-Instruct
    parameters:
      weight: 0.4
      density: 0.6
  - model: mistralai/Mistral-7B-Instruct-v0.3
    parameters:
      weight: 0.3
      density: 0.6
  - model: codestral/Mistral-Coder-7B
    parameters:
      weight: 0.3
      density: 0.5
parameters:
  normalize: true
dtype: bfloat16
bash
mergekit-yaml solar-dare-llama-mistral.yaml ./output

Want to build your own merge?

Use the MergeKit config generator to build a YAML recipe visually — no code required.

Open Config GeneratorBrowse More Recipes