Authoring `do/train` Techniques¶

A do/train technique is a directory. Drop one into do/training/ and do/train discovers it — no change to the script is needed. Add an optional manifest.json and the technique also gets a typed, tiered, validated hyperparameter UI built on the same config-builder core that powers do/tune.

See Custom Training for the job-level workflow (datasets, instances, lifecycle). This page is the reference for the technique plugin surface. New to it? Start with the walkthrough, which adds the sft-neftune sample (examples/train-plugins/sft-neftune/) to a project step by step.

Layout¶

do/training/
├── config.yaml                 # the job (technique, image, instance, dataset, hyperparameters)
├── manifest.schema.json        # GENERATED — JSON schema for technique manifests (editor hint)
└── <technique>/
    ├── train.py                # REQUIRED — your training script
    ├── manifest.json           # optional — knob metadata (types, bounds, tiers, help text)
    ├── defaults.yaml           # optional — knob VALUES (the lowest-priority value layer)
    └── accelerate_config.yaml  # optional — distributed launcher config

train.py is the only required file. A directory without it is ignored.
manifest.json describes knobs; it does not carry values. Put values in defaults.yaml (one value source per technique).
prompts.json is the pre-Wave-7 way to add interactive questions. It still works for techniques without a manifest. When a manifest exists, prompts.json is ignored.

The shipped sft, dpo, and grpo techniques have manifests. custom intentionally has none: it is the reference for descriptor-less techniques.

The manifest¶

{
  "$schema": "../manifest.schema.json",
  "schema_version": 1,
  "technique": "mytech",
  "display_name": "My technique",
  "description": "What it trains and on what data.",
  "hyperparameters": {
    "learning_rate": {
      "type": "number",
      "required": false,
      "tier": "primary",
      "display_name": "Learning rate",
      "description": "optimizer learning rate"
    },
    "lora_dropout": {
      "type": "number",
      "required": false,
      "min": 0,
      "max": 1,
      "tier": "advanced",
      "display_name": "LoRA dropout"
    },
    "scheduler": {
      "type": "string",
      "required": false,
      "enum": ["linear", "cosine"],
      "tier": "advanced"
    }
  }
}

Top-level keys:

Key	Required	Meaning
`$schema`	no	Editor hint. `"../manifest.schema.json"` gives VS Code validation and autocomplete. Ignored by `do/train`.
`schema_version`	yes	Always `1`.
`technique`	yes	Must equal the directory name.
`display_name`, `description`	no	Human-readable text.
`hyperparameters`	yes	Knob name → knob entry. Knob names are the keys `train.py` reads.

Each knob entry uses the same contract as the do/tune descriptor (templates/code/tune.d/manifest.schema.json#/$defs/hyperparameter):

Key	Required	Meaning
`type`	yes	`string`, `integer`, `number`, or `boolean`. Values are coerced and checked against it.
`required`	yes	`true` makes a missing value an error.
`min`, `max`	no	Inclusive numeric bounds. Declare them only for a real invariant (a ratio in 0–1, a count ≥ 1).
`step`	no	Numeric step. Enforced for integer knobs; a UI hint for floats.
`enum`	no	Allowed values (non-empty list). Renders as a dropdown.
`default`	no	A fallback value. Prefer `defaults.yaml`; the shipped manifests declare none.
`tier`	no	`primary` (default), `advanced`, or `tertiary`. See below.
`control`	no	`number_input`, `dropdown`, or `text_input`. Derived when absent (`enum` → dropdown, numeric → number input, else text).
`display_name`, `description`, `hint`	no	Prompt label and help text.

do/training/manifest.schema.json is generated by npm run codegen from the repo's config/train-descriptor.schema.json. Don't edit the generated copy.

Tiers¶

Tier	Prompted with `-i`	Prompted with `-i --advanced`	Value when not prompted
`primary`	yes	yes	from the value layers
`advanced`	no	yes	from the value layers
`tertiary`	never	never	from the value layers

Every knob can always be set with --set, whatever its tier.

Setting values¶

./do/train --technique mytech --set learning_rate=1e-4 --set scheduler=cosine
./do/train --technique sft --learning-rate 1e-4      # legacy alias for --set learning_rate=1e-4
./do/train -i                                         # job questions, then primary knobs
./do/train -i --advanced                              # ... plus advanced knobs

do/train is non-interactive unless you pass -i/--interactive. --advanced without -i prints a notice and has no effect. The legacy flags (--learning-rate, --epochs, --batch-size, --lora-r, --beta) are aliases for --set learning_rate=…, epochs=…, batch_size=…, lora_r=…, beta=….

Precedence¶

Lowest to highest — the same order do/tune uses:

#	Layer	Source label in the summary
1	manifest `default`	`manifest`
2	`<technique>/defaults.yaml`	`defaults.yaml`
3	`config.yaml` `hyperparameters:`	`config.yaml`
4	`--set` / legacy flag, or an interactive answer	`flag` / `prompt`

With -i, every knob you did not set by flag is prompted (by tier), and the prompt is pre-filled with the value from layers 1–3. So your config.yaml values come back as defaults. Your answers are written back to the hyperparameters: block of config.yaml. Only that block is rewritten; every other setting stays as you had it. Comments inside the block are not kept.

Before submitting, do/train prints every hyperparameter the job will get and where each value came from:

⚙️  Hyperparameters this job will use (sft, manifest.json):
   batch_size      4      (defaults.yaml)
   epochs          5      (config.yaml)
   learning_rate   1e-4   (flag)

Values reach SageMaker as strings, so do/train passes on the string you wrote: 2e-4 stays 2e-4. It validates a typed copy of the value but never re-renders it.

Keys the manifest does not declare¶

From defaults.yaml or config.yaml: passed to train.py unchanged, with a one-line notice. These files are deliberate edits to a script you own.
From --set or a legacy flag: rejected before anything is submitted. This catches typos that train.py would otherwise ignore without a word:

❌ Hyperparameter validation failed for technique 'sft':
   • unknown knob 'learnign_rate' for technique 'sft' — did you mean 'learning_rate'? Declared: batch_size, ... (use --allow-unknown to pass it to train.py anyway)

A legacy flag for a knob the technique doesn't declare is rejected the same way (for example, --beta on sft).

--allow-unknown passes undeclared --set/flag keys through to train.py with the same notice. Use it when your train.py reads a key you haven't added to the manifest yet. The better long-term fix is to declare the knob.

Validation errors¶

When a technique has a manifest.json, do/train validates it before resolving any value or calling AWS. Every problem is reported in one pass, and the run exits with code 2:

❌ Invalid training manifest: do/training/mytech/manifest.json
   • hyperparameters.learning_rate: 'min' (1.0) is greater than 'max' (0.1)
   • hyperparameters.scheduler: unknown key 'choices' (did you mean 'enum'?)
   → Fix the manifest, or delete it to fall back to defaults.yaml + flags.

Each line names the knob (or top-level key) and the problem. Checks cover: valid JSON (with line/column), technique matching the directory, known keys only, type and required, a non-empty enum, numeric min/max/step with min ≤ max, tier and control values, and a default that fits its own type, bounds, and enum.

A value error (out of range, wrong type, not in enum, unknown --set key) exits with code 1, also before any AWS call. No extra Python dependency is needed: the validator is plain Python.

Descriptor-less techniques¶

A technique without manifest.json (such as custom, or any directory you add) works exactly as before:

values merge defaults.yaml < config.yaml < flags, as strings, with no validation;
--set KNOB=VALUE works too and passes any key through;
--advanced and --allow-unknown have no effect;
-i asks the generic epochs/learning-rate/LoRA-rank questions plus any prompts.json questions.

Add a manifest whenever you want typed prompts and fail-fast validation.

Upgrading an existing project¶

Projects generated before this change keep working with the new do/train:

What the project has	What happens
`<t>/prompts.json`, no `manifest.json`	Descriptor-less path, today's behavior (including the `prompts.json` questions).
`manifest.json` and a leftover `prompts.json`	The manifest wins; `prompts.json` is ignored. You can delete it.
`config.yaml` with `hyperparameters:`	Read as the `config.yaml` layer.

What mcc regenerate does to do/training/:

It re-renders every shipped template file and overwrites it: config.yaml, and sft/, dpo/, grpo/, custom/ files such as train.py and defaults.yaml. Back up config.yaml (and any edited shipped recipe) before regenerating, then restore your settings.
It adds the new files: manifest.schema.json and sft/, dpo/, grpo/ manifest.json.
It deletes nothing: a leftover sft/prompts.json or dpo/prompts.json stays, and is ignored because the manifest now exists.
It leaves technique directories you added yourself untouched.

If you don't regenerate, your project keeps its current files and stays on the descriptor-less path for every technique until you add manifests.

Authoring do/train Techniques¶