ai/lfm3 repository overview

API⁠ Try⁠ Docs⁠ Blog⁠

⁠LFM3-1.7B-GGUF

GGUF version of LFM3-1.7B⁠ for local inference on CPUs and GPUs.

Find architecture and usage details in the original model card⁠.

⁠How to run LFM3-1.7B

Recommended sampling: temperature 0.1, top_k 50, repeat penalty 1.1. For math and science: temperature 1.0, top_p 0.95, top_k 20.

• Llama.app⁠ (Use this model > Llama in the menu), for macOS and Windows. Requires llama.cpp version v0.8.0⁠ or newer.
• LM Studio⁠ (Use this model > LM Studio in the menu), for macOS, Windows and Linux. Requires LM Studio with llama.cpp engine 2.65.1 or newer.
• Ollama⁠ (ollama run lfm3), for macOS. Requires Ollama version v0.40.12⁠ or newer.

⁠Use with llama.cpp

Add a quantization tag to pick a file.

llama-cli -hf LiquidAI/LFM3-1.7B-GGUF:Q4_K_M \
  --temp 0.1 --top-k 50 --repeat-penalty 1.1

For math and science, use --temp 1.0 --top-p 0.95 --top-k 20 instead. More options in the llama.cpp guide⁠.

⁠NVFP4 GGUF

Unlike the other files in this repository, which are converted from LFM3-1.7B⁠, LFM3-1.7B-NVFP4.gguf⁠ is converted from a separate repository, LiquidAI/LFM3-1.7B-NVFP4⁠, a checkpoint quantized to NVFP4 with NVIDIA Model Optimizer. The feed-forward weights keep their NVFP4 quantization from that checkpoint, all other weights are Q8_0. On NVIDIA Blackwell GPUs (RTX 50 series, DGX Spark) the NVFP4 layers run on FP4 tensor cores.

Example usage with llama.cpp⁠:

llama-cli -hf LiquidAI/LFM3-1.7B-GGUF:NVFP4 \
  --temp 0.1 --top-k 50 --repeat-penalty 1.1

⁠QAD Q4_0 GGUF

The Q4_0 build in this repository is the Quantization-Aware Distillation (QAD) checkpoint, LFM3-1.7B-QAD-Q4_0.gguf⁠. It was trained to recover the accuracy lost to Q4_0 quantization.

Example usage with llama.cpp⁠:

llama-cli -hf LiquidAI/LFM3-1.7B-GGUF \
  --hf-file LFM3-1.7B-QAD-Q4_0.gguf \
  --temp 0.1 --top-k 50 --repeat-penalty 1.1