When your home LLM feels slow, check data movement before math

In short

If it feels slow, do not poke settings first. Write the wait time for the same prompt three times. Decide whether math is busy or data movement is stuck. If unsure, temporarily try a lighter model and compare.

Who it's for

People running a large language model on a home PC who feel it got heavy lately.

Core

1. Before settings, measure wait time under the same conditions.
2. Next steps differ if math is busy versus data movement is stuck.
3. Change one knob at a time. Write down today’s values so you can undo.

Details

Signs math is busy

GPU stays busy and waits stay long. Shorter prompts barely help. The model itself may be too heavy.

Signs data movement is stuck

GPU looks idle but replies still take time. Longer prompts get much slower. A lighter model does not suddenly fix it.

Tonight’s order

Same prompt three times — note seconds only. Then half the length, three more times. Small gap → suspect math. Big gap → suspect movement.

Field note

On good days, do not add more ritual. On bad days, step back one rung. Outside guides also prefer “one move you can try tonight” over a full toolkit.

Steps

1. Write wait seconds for the same prompt three times.
2. Shorten the prompt and write three more.
3. Glance at GPU use.
4. When the cause shows, change only one knob.

Easy to fail

Rushing at night leads to many setting changes with no record. A three-line seconds memo is enough. Watching only bench scores and skipping your typical task. Changing three settings at once so you never know what helped.

Source articles

• llama.cpp — https://github.com/ggerganov/llama.cpp
• Hugging Face Hub — https://huggingface.co/docs/hub