AttributeError: ‘TrainingArguments’ Object Has No Attribute ‘Model_Init_Kwargs’ | Fast Fix For TRL Config

Hitting a trace that ends with AttributeError: ‘TrainingArguments’ object has no attribute ‘model_init_kwargs’ can stop a training run right when you think the setup is ready. The stack trace often points at TRL trainers such as SFTTrainer or DPOTrainer and leaves many users wondering what changed in their stack.

This guide walks through what the error message actually means, how it relates to recent TRL releases, and how to switch your script to the config classes that the trainers now expect. By the end, you can get your fine tuning run working again without trial and error.

Why You See AttributeError: ‘TrainingArguments’ Object Has No Attribute ‘Model_Init_Kwargs’

The message AttributeError: ‘TrainingArguments’ object has no attribute ‘model_init_kwargs’ tells you that the trainer code is trying to read a field on the args object that is not present. In older TRL versions, the args parameter of SFTTrainer or DPOTrainer commonly held a transformers.TrainingArguments instance, so many public notebooks still build arguments this way.

Newer TRL releases added extra options on top of TrainingArguments, such as model_init_kwargs, packing, dataset_text_field and several dataset settings. These extra fields live on TRL specific config classes such as SFTConfig and DPOConfig, not on TrainingArguments. When you pass a plain TrainingArguments object to SFTTrainer or DPOTrainer, the trainer code expects a config with those extra fields and fails with this AttributeError.

In short, the error appears when there is a mismatch between the type that the trainer expects and the type you actually pass in as args. SFTTrainer and DPOTrainer call args.model_init_kwargs internally, and a TrainingArguments object from transformers simply does not define that attribute.

Typical Scenarios That Trigger The Error

• Using Old Notebooks With New TRL — Many guides still build training_arguments = TrainingArguments(...) and then pass args=training_arguments into SFTTrainer. With recent TRL versions this pattern fails because training_arguments lacks model_init_kwargs and packing fields.
• Upgrading TRL Only — A pip upgrade that bumps trl while leaving transformers pinned can switch SFTTrainer or DPOTrainer to a build that expects SFTConfig or DPOConfig instead of TrainingArguments.
• Mixed Config Types In One Project — Some scripts mix TrainingArguments for one trainer and SFTConfig or DPOConfig for another. Passing the wrong config variable into a trainer is an easy slip and leads straight to AttributeError: ‘TrainingArguments’ object has no attribute ‘model_init_kwargs’.

Fixing The TrainingArguments model_init_kwargs Error In TRL

The cleanest fix is to pass the config class that the trainer expects. For SFTTrainer this means SFTConfig, and for DPOTrainer this means DPOConfig. These classes mirror many TrainingArguments fields, but they also add the extra attributes that TRL trainers rely on, including model_init_kwargs and packing.

Switching from TrainingArguments to SFTConfig usually involves changing only one line and keeping the same argument names that describe your training run. Here is a minimal sketch that replaces TrainingArguments with SFTConfig for supervised fine tuning:

from trl import SFTTrainer, SFTConfig

sft_config = SFTConfig(
    output_dir="results",
    num_train_epochs=1,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=1,
    learning_rate=2e-4,
    weight_decay=0.001,
    logging_steps=25,
    save_steps=0,
    bf16=False,
    fp16=False,
    max_grad_norm=0.3,
    warmup_ratio=0.03,
    report_to="tensorboard",
)

trainer = SFTTrainer(
    model=model,
    train_dataset=train_dataset,
    peft_config=peft_config,
    tokenizer=tokenizer,
    dataset_text_field="text",
    args=sft_config,
    packing=False,
)

In this pattern, args receives an SFTConfig instance that already contains model_init_kwargs and packing fields with defaults, so the AttributeError: ‘TrainingArguments’ object has no attribute ‘model_init_kwargs’ never fires.

When You Use DPOTrainer Instead Of SFTTrainer

DPOTrainer follows the same pattern, but the matching config class is DPOConfig rather than SFTConfig. The library documentation states that DPOTrainer expects a DPOConfig object and will raise AttributeError: ‘TrainingArguments’ object has no attribute ‘model_init_kwargs’ when a TrainingArguments instance is passed in as args.

from trl import DPOTrainer, DPOConfig

dpo_config = DPOConfig(
    output_dir="results",
    per_device_train_batch_size=2,
    num_train_epochs=1,
    learning_rate=1e-5,
    beta=0.1,
)

trainer = DPOTrainer(
    model=model,
    args=dpo_config,
    beta=dpo_config.beta,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset,
    tokenizer=tokenizer,
)

By aligning each trainer with its matching config class, you keep model_init_kwargs and related attributes in one place and keep your scripts close to the shapes used in the official TRL examples.

Mapping Common Fields From TrainingArguments To TRL Configs

Many scripts already build TrainingArguments with fields such as output_dir, num_train_epochs, per_device_train_batch_size and warmup_ratio. When you switch to SFTConfig or DPOConfig, those same names still work, so the change often feels smaller than the error message suggests.

• Reuse Core Training Settings — Copy output_dir, num_train_epochs, gradient_accumulation_steps, learning_rate and weight_decay straight into SFTConfig or DPOConfig. These fields share the same meaning in TRL configs.
• Carry Over Logging Preferences — Bring logging_steps, save_steps, report_to and gradient_checkpointing across so that runs still log to the same tools and keep the same checkpoint schedule.
• Align Precision Flags — Keep bf16, fp16 and max_grad_norm values the same when you move from TrainingArguments to SFTConfig or DPOConfig so that numerical behaviour does not change by accident.

This direct mapping keeps migration mechanical. You adjust only the config type and leave the core intent of the training run intact.

Handling Related Errors Like ‘TrainingArguments’ Object Has No Attribute ‘Packing’

Some users first see AttributeError traces about a missing packing field on TrainingArguments before they hit the model_init_kwargs variant. The root cause is the same: newer TRL trainers expect config classes that define packing and related flags, while TrainingArguments from transformers does not include them.

The usual pattern looks like this: the script passes packing=False into SFTTrainer while args points at a TrainingArguments instance. Inside the trainer, TRL tries to read args.packing to decide how to build batches or data collators. Since TrainingArguments has no packing attribute, Python raises AttributeError: ‘TrainingArguments’ object has no attribute ‘packing’.

Once you switch args over to SFTConfig or DPOConfig, the packing flag lives on the config object, and that entire class of errors drops away. You also gain access to eval_packing and padding_free settings that give more control over how tokens are arranged into sequences during training.

Extra Debugging Habits That Help With Config Mismatches

• Print Type Information Early — Add a short print(type(args)) line before you build the trainer. Seeing SFTConfig, DPOConfig or TrainingArguments in the console makes it easier to spot mismatches.
• Inspect Available Attributes — In a notebook, dir(args) gives a quick list of attribute names. If model_init_kwargs, packing or dataset_text_field are missing, you know that the config object is not the one the trainer expects.
• Keep Version Pins In One Place — Store transformers, trl and peft version pins in a single requirements cell or file. That way each notebook run uses the same versions and surprise AttributeError messages are less likely after a fresh pip install.

These small habits shorten the time between a failing run and a clear fix, especially when you share notebooks across machines, cloud sessions or teammates.

Step-By-Step Fix Using SFTConfig Or DPOConfig

Use this checklist to turn a failing training script into one that matches the current TRL configuration layout.

1. Scan Imports For TrainingArguments — Search for from transformers import TrainingArguments near the top of the script. If the only trainer you use is SFTTrainer or DPOTrainer, plan to swap that import out for SFTConfig or DPOConfig.
2. Replace The TrainingArguments Block — Find the block that builds training_arguments = TrainingArguments(...). Copy the argument names into a new SFTConfig or DPOConfig call and remove the TrainingArguments creation.
3. Wire The New Config Into The Trainer — Make sure the args field of SFTTrainer or DPOTrainer now points at your new config object, such as args=sft_config or args=dpo_config.
4. Check dataset_text_field And packing — When you build SFTConfig or DPOConfig you can control dataset_text_field, packing, eval_packing and related flags directly on the config. Adjust them so they match how your dataset is prepared.
5. Run A Short Smoke Test — Before launching a long run, start with a tiny dataset slice and one epoch. Confirm that the AttributeError: ‘TrainingArguments’ object has no attribute ‘model_init_kwargs’ no longer shows up and that the trainer starts logging steps.

Quick Reference Table For Config Choices

When Downgrading TRL Or Transformers Still Helps

If you depend on a public notebook or older codebase and cannot change it right away, one short term path is to pin TRL to a version that still accepts TrainingArguments with SFTTrainer. Users who hit AttributeError: ‘TrainingArguments’ object has no attribute ‘model_init_kwargs’ on Gemma or Llama runs have reported that trl==0.7.11 restores the older behaviour.

This route trades new features for quick compatibility. A pinned TRL build may miss bug fixes, new model helpers or fresh trainer options. That is why the long term fix is to migrate scripts onto SFTConfig or DPOConfig, but a version pin can keep a teaching notebook or demo running while you plan the change.

pip install "transformers==4.38.2" "trl==0.7.11"

After a downgrade, run a short training job and confirm that the error clears and that loss values start to print as before.

Small Version-Check Checklist Before Each New Run

A short preflight check can stop AttributeError: ‘TrainingArguments’ object has no attribute ‘model_init_kwargs’ from showing up again when you switch machines or rebuild setups.

1. Freeze Working Versions — After you reach a stable training script, record the exact transformers, trl and peft versions in a requirements file or a comment near the import block.
2. Recreate Setups From That Record — When you start a new run on another machine, build the setup from the same version list instead of installing unnamed latest builds.
3. Skim Release Notes Before Upgrades — When you upgrade transformers or trl, read the most recent changelog entries for mentions of config changes or new trainer arguments.
4. Run A Minimal Sanity Script — Keep a small script that loads a small model, constructs the trainer with your usual config class and runs a couple of steps to check that everything still starts cleanly.

Quick Recap Of AttributeError: ‘TrainingArguments’ Object Has No Attribute ‘Model_Init_Kwargs’

AttributeError: ‘TrainingArguments’ object has no attribute ‘model_init_kwargs’ points at a mismatch between the trainer that runs your fine tuning job and the configuration object you pass to its args field. Newer TRL trainers expect SFTConfig or DPOConfig, while many older guides still pass TrainingArguments from transformers.

For day to day work, the most stable approach is to keep trainers and config objects paired: SFTTrainer with SFTConfig, DPOTrainer with DPOConfig, and the plain transformers Trainer with TrainingArguments. That way model_init_kwargs, packing and related attributes always match what the code inside each trainer tries to read.

When you update TRL or transformers versions, skim the release notes for mentions of new config classes or extra fields in trainer arguments. A small edit to the config object often prevents AttributeError traces and keeps long training runs on track.

Send us your feedbackTell us what you thought about this post

Name * Email *

Your comment *

Quick check — what is …? *

Send feedback