How do I save bnb-4bit model with `merged_16bit`?

[azulika]

This is essentially the same issue as unslothai/unsloth#2749 .

Pull request #254 claims to have fixed this, but the implementation does not seem to solve the core issue. While the pull request adds support for 16-bit base models and mxfp4 native models, it does not address the problem that the original poster in #2749 and others were facing: saving a bnb-4bit model like Qwen3-8B-bnb-4int with merged_16bit.

Currently, it is impossible to use save_pretrained_merged with merged_16bit for bnb-4bit models like unsloth/gemma-3-27b-it-bnb-4bit and unsloth/Mistral-Small-3.2-24B-Instruct-2506-unsloth-bnb-4bit due to the following code in saving_utils.py:

        if base_model_is_quantized and (quant_type == "nf4" or quant_type == "fp4") and save_method == "merged_16bit":
            warnings.warn("Base model should be a 16bits or mxfp4 base model for a 16bit model merge. Use `save_method=forced_merged_4bit` instead")
            return None

Since bnb-4bit models use nf4 quantization, this condition always evaluates to true, causing the function to return None and fail the merge operation.

Does this mean there is currently no way to save bnb-4bit models as 16-bit merged models, or is this feature simply not implemented yet?

[rolandtannous]

load unsloth/Mistral-Small-3.2-24B-Instruct-2506 instead with load_4bit=True (which will load it in 4bit), but will still map the base model to unsloth/Mistral-Small-3.2-24B-Instruct-2506, so when save_pretrained_merged is called it can detect that the model has a base_model in 16bit that can be loaded for merging.

The objective from the code you reference is to stop people from using custom 4bit models as a basis for qlora . Since these custom 4bit models can't be mapped back to a 16bit base model, merging in 16bit using a 4 bit base model will result in very bad performance because the merging with adapters happens in 32 bits` , so in this case you're upcasting from 4bit to 32 bit. This will lose a lot of precision and might make the model unusable.