{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Miner Tips 00: Start Here\n",
        "\n",
        "This is the plain-English starting point for the BitSota Qwen **binary** and **ternary** compression competitions.\n",
        "\n",
        "The goal is simple: submit a compressed model artifact that keeps public/dev perplexity low while meeting the competition format. This notebook does **not** use private validator data, private endpoints, auth secrets, or local operator files.\n",
        "\n",
        "## Mental model\n",
        "\n",
        "```text\n",
        "public text examples\n",
        "        |\n",
        "        v\n",
        "reference Qwen tokenizer\n",
        "        |\n",
        "        v\n",
        "candidate compressed model artifact\n",
        "        |\n",
        "        v\n",
        "next-token cross entropy\n",
        "        |\n",
        "        v\n",
        "public/dev PPL estimate\n",
        "```\n",
        "\n",
        "Lower PPL is better. A good public/dev estimate does not guarantee leaderboard rank, but it is much better than optimizing only file size or reconstruction error."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## What binary and ternary mean here\n",
        "\n",
        "```text\n",
        "Dense weights:     many possible floating-point values\n",
        "Binary weights:    mostly two values, usually sign-like values\n",
        "Ternary weights:   mostly three values, usually negative / zero / positive\n",
        "Q4 rescue rows:    a small fraction of sensitive rows kept in 4-bit form\n",
        "```\n",
        "\n",
        "A practical recipe is usually mixed, not pure low-bit everywhere:\n",
        "\n",
        "```text\n",
        "measure sensitivity per row/module\n",
        "        |\n",
        "        +--> easy rows: binary or ternary\n",
        "        |\n",
        "        +--> fragile rows: q4 rescue or denser fallback\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Paste this brief into a coding agent if you want it to build a first miner recipe.\n",
        "AGENT_BRIEF = \"\"\"\n",
        "Build a public/dev-only Qwen compression experiment for the BitSota binary or ternary competition.\n",
        "\n",
        "Rules:\n",
        "1. Use the task repository and its public onboarding instructions.\n",
        "2. Use only public or self-created calibration/evaluation text.\n",
        "3. Score with the reference Qwen tokenizer and shifted next-token cross entropy.\n",
        "4. Try mixed compression: binary/ternary for easy rows, q4 rescue for fragile rows.\n",
        "5. Save an artifact with deterministic packaging, SHA-256, and byte size.\n",
        "6. Do not include private data, local operator paths, auth secrets, or generated cache files.\n",
        "\n",
        "Start with:\n",
        "- row sensitivity + q4 rescue baseline\n",
        "- layerwise distillation\n",
        "- Net2Net-style widening before quantization\n",
        "\"\"\"\n",
        "print(AGENT_BRIEF)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Suggested order\n",
        "\n",
        "1. Run `01_public_ppl_and_packaging.ipynb` so you know how to measure and package.\n",
        "2. Try `02_rowmix_q4_rescue_baseline.ipynb` for a fast low-risk baseline.\n",
        "3. Try `03_layerwise_distillation.ipynb` when simple rounding is not enough.\n",
        "4. Try `04_net2net_widen_then_quantize.ipynb` when you want a bigger student that starts from a sane function.\n",
        "5. Use `05_diagnostics_and_traps.ipynb` to avoid experiments that look good on proxy metrics but fail on text quality."
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "pygments_lexer": "ipython3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}