PassthroughFeaturedpassthroughfrankenmergelargellama3experimental

FrankenLlama 120B — Passthrough Layer Stack

A passthrough (frankenmerge) recipe stacking layers from Llama-3-70B and Llama-3-8B to produce a 120B architecture. Demonstrates the power of layer-level composition for creating models larger than any single input. Layers 0-39 from 70B, layers 40-79 spliced from 8B.

Author

chargoddard

Published

January 5, 2026

Last updated

January 5, 2026

Versions

1

Best Score

↑ 79.4

Stars

244

Meta-Llama-3-70B-Instruct · 70BMeta-Llama-3-8B-Instruct · 8B
Share

Merge Lineage

Merge Lineage

2 source models

Layer composition

Meta-Llama-3-70B-Instruct
Meta-Llama-3-8B-Instruct
Meta-Llama-3-70B-Instruct (70B) · L0–L39
Meta-Llama-3-8B-Instruct (8B) · L0–L39

Click any model node to open its Hugging Face page

Config YAML

frankenllama-120b-passthrough-v1.0.yaml
merge_method: passthrough
slices: - sources:
      - model: meta-llama/Meta-Llama-3-70B-Instruct
        layer_range: [0, 39]
  - sources: - model: meta-llama/Meta-Llama-3-8B-Instruct
        layer_range: [0, 39]
dtype: bfloat16

Benchmark Scores

BenchmarkMergedLlama-3-70BLlama-3-8BΔ Best
MMLUtop
79.478.966.9+0.5
MT-Bench
8.98.78.0+0.2
HumanEval
72.071.561.0+0.5

How I Built This

Hugging FaceGitHubDiscord

Embed Badge

Add this to your Hugging Face model card to link back to this recipe.

MergeKitRecipe
markdown
[![MergeKit Recipe](https://img.shields.io/badge/MergeKit-Recipe-10b981?style=flat-square)](https://www.mergekit.com/recipes/frankenllama-120b-passthrough)

Use this Model

Run, deploy, or interact with FrankenLlama 120B — Passthrough Layer Stack directly.

MergeKit Cloud

Run this model on serverless GPU infrastructure — zero setup, pay-per-second.

RunPod Serverless

Select GPU

A10G · 24 GB~45s cold
A100 · 80 GB~30s cold
H100 · 80 GB~20s cold
Est. ~$0.001 / inference · billed per second
Coming Soon

Serverless GPU · Powered by RunPod

Reproduce Locally

Run this exact merge on your own machine in three steps:

bash
pip install mergekit
yaml
merge_method: passthrough
slices:
  - sources:
      - model: meta-llama/Meta-Llama-3-70B-Instruct
        layer_range: [0, 39]
  - sources:
      - model: meta-llama/Meta-Llama-3-8B-Instruct
        layer_range: [0, 39]
dtype: bfloat16
bash
mergekit-yaml frankenllama-120b-passthrough.yaml ./output

Want to build your own merge?

Use the MergeKit config generator to build a YAML recipe visually — no code required.

Open Config GeneratorBrowse More Recipes