| Flag |
Description |
Packages |
| amd |
build for AMD GPU, ROCM-based |
2 |
| amd_mae |
enable experimental memory efficient attention on some AMD GPUs |
2 |
| cpu |
build for CPU-generation only |
3 |
| desktop |
Create a "desktop" file (browser launcher) and add an icon. |
3 |
| intel |
build for Intel GPU (XPU)
Compatible Hardware ( https://docs.pytorch.org/docs/main/notes/get_start_xpu.html ):
Intel® Arc A-Series Graphics (CodeName: Alchemist);
Intel® Arc B-Series Graphics (CodeName: Battlemage);
Intel® Core™ Ultra Processors with Intel® Arc™ Graphics (CodeName: Meteor Lake-H);
Intel® Core™ Ultra Desktop Processors (Series 2) with Intel® Arc™ Graphics (CodeName: Lunar Lake);
Intel® Core™ Ultra Mobile Processors (Series 2) with Intel® Arc™ Graphics (CodeName: Arrow Lake-H);
Intel® Data Center GPU Max Series (CodeName: Ponte Vecchio) |
2 |
| ipex |
build for Intel GPU (IPEX)
Compatible Hardware:
Intel® Arc™ A-Series Graphics (Intel® Arc™ A770 [Verified], Intel® Arc™ A750, Intel® Arc™ A580, Intel® Arc™ A770M, Intel® Arc™ A730M, Intel® Arc™ A550M);
Intel® Arc™ B-Series Graphics (Intel® Arc™ B580 [Verified], Intel® Arc™ B570);
Intel® Data Center GPU Max Series [Verified];
For GPUs newer than Intel® Core™ Ultra Processors with Intel® Arc™ Graphics (Meteor Lake) or Intel® Arc™ A-Series Graphics that aren't listed, please check the AOT documentation ( https://www.intel.com/content/www/us/en/docs/dpcpp-cpp-compiler/developer-guide-reference/2025-0/ahead-of-time-compilation.html ) to see if it is supported. If so, follow instructions in the source section above to compile from source. |
2 |
| nvidia |
build for NVidia GPU, cuda-based |
2 |
| rdna2 |
build for AMD GPU (together with amd flag), for 6700, 6600 and maybe other RDNA2 or older |
2 |
| rdna3 |
build for AMD GPU (together with amd flag), for AMD 7600 and maybe other RDNA3 cards |
2 |
| Flag |
Description |
| accelerate |
ggml: enable Accelerate framework |
| blas |
ggml: use BLAS; for using specific vendor check https://wiki.gentoo.org/wiki/Blas-lapack-switch |
| blis |
ggml: use BLIS ( https://github.com/flame/blis ) |
| cann |
ggml: use CANN ; This provides NPU acceleration using the AI cores of your Ascend NPU. |
| cpu-all-variants |
ggml: enable CPU backend; please use together with cpu flag; ggml: build all variants of the CPU backend (requires GGML_BACKEND_DL) |
| cpu-native |
ggml: enable CPU backend; please use together with cpu flag; it will add `-march=native` by itself. CPU_FLAGS_* consistency is expected to be user's problem |
| cpu_flags_loong_lasx |
|
| cpu_flags_loong_lsx |
|
| cpu_flags_riscv_rvv |
|
| cpu_flags_x86_amx_bf16 |
|
| cpu_flags_x86_amx_int8 |
|
| cpu_flags_x86_amx_tile |
|
| cpu_flags_x86_avx |
|
| cpu_flags_x86_avx2 |
|
| cpu_flags_x86_avx512 |
|
| cpu_flags_x86_avx512_bf16 |
|
| cpu_flags_x86_avx512_vbmi |
|
| cpu_flags_x86_avx512_vnni |
|
| cpu_flags_x86_avx_vnni |
|
| cpu_flags_x86_f16c |
|
| cpu_flags_x86_fma |
|
| cpu_flags_x86_sse |
|
| cpu_flags_x86_sse2 |
|
| cpu_flags_x86_sse3 |
|
| cpu_flags_x86_sse4 |
|
| cpu_flags_x86_sse41 |
|
| cpu_flags_x86_sse42 |
|
| cpu_flags_x86_sse4a |
|
| cpu_flags_x86_ssse3 |
|
| cuda |
ggml: use CUDA ; in order for it to compile also tag .... "cpu" has to be selected, don't ask why. Use CMAKE_EXTRA_CACHE_FILE env variable (check https://gitweb.gentoo.org/repo/gentoo.git/tree/eclass/cmake.eclass ) to specify next variables: CMAKE_CUDA_ARCHITECTURES ( use next command to find native architecture: `nvidia-smi --query-gpu=compute_cap --format=csv | tail -n 1 | sed -e 's/\.//g'` ) ; GGML_CUDA_PEER_MAX_BATCH_SIZE ggml: max. batch size for using peer access, default: 128 |
| cuda-f16 |
ggml: use 16 bit floats for some calculations |
| cuda-fa-all-quants |
ggml: compile all quants for FlashAttention |
| cuda-force-cublas |
ggml: always use cuBLAS instead of mmq kernels |
| cuda-force-mmq |
ggml: use mmq kernels instead of cuBLAS |
| cuda-graphs |
ggml: use CUDA graphs (llama.cpp only) |
| cuda-no-peer-copy |
ggml: do not use peer to peer copies |
| cuda-no-vmm |
ggml: do not try to use CUDA VMM |
| cuda-unified-memory |
ggml: CUDA : unified memory: allow CUDA app use unified memory architecture (UMA) to share main memory between the CPU and integrated GPU |
| curl |
|
| disable-arm-neon |
Disable Arm Neon.
Arm Neon is an advanced single instruction multiple data (SIMD) architecture extension for the Arm Cortex-A and Arm Cortex-R series of processors.
Might help in case of CUDA related compilations errors: https://github.com/ggml-org/llama.cpp/issues/12826 |
| dynamic-backends |
In most cases, it is possible to build and use multiple backends at the same time. For example, you can build llama.cpp with both CUDA and Vulkan support by using the -DGGML_CUDA=ON -DGGML_VULKAN=ON options with CMake. At runtime, you can specify which backend devices to use with the --device option. To see a list of available devices, use the --list-devices option.
Backends can be built as dynamic libraries that can be loaded dynamically at runtime. This allows you to use the same llama.cpp binary on different machines with different GPUs. To enable this feature, use the GGML_BACKEND_DL option when building.
GGML_NATIVE ("cpu-native" flag) is not compatible with GGML_BACKEND_DL, if you want to use also native flag, consider using otherwise GGML_CPU_ALL_VARIANTS |
| hbm |
ggml: use memkind for CPU HBM ( High Bandwidth Memory , check wikipedia.) ; a hardware-related feature |
| hip |
ggml: use HIP |
| hip-graphs |
ggml: use HIP graph, experimental, slow |
| hip-no-vmm |
ggml: do not try to use HIP VMM |
| hip-uma |
ggml: use HIP unified memory architecture : from docs/build.md : On Linux it is also possible to use unified memory architecture (UMA) to share main memory between the CPU and integrated GPU by setting -DGGML_HIP_UMA=ON. However, this hurts performance for non-integrated GPUs (but enables working with integrated GPUs). |
| kleidiai |
ggml: use kleidiai optimized kernels if applicable |
| kompute |
ggml: use Kompute |
| llamafile |
ggml: use LLAMAFILE |
| lto |
|
| metal |
ggml: use Metal; use CMAKE_EXTRA_CACHE_FILE env variable (check https://gitweb.gentoo.org/repo/gentoo.git/tree/eclass/cmake.eclass ) to specify next variables: ggml: metal minimum macOS version GGML_METAL_MACOSX_VERSION_MIN ; ggml: metal standard version (-std flag) GGML_METAL_STD |
| metal-embed-library |
ggml: embed Metal library |
| metal-ndebug |
ggml: disable Metal debugging |
| metal-shader-debug |
ggml: compile Metal with -fno-fast-math |
| metal-use-bf16 |
ggml: use bfloat if available |
| msvc |
Enable if you build with MSVC |
| musa |
ggml: use MUSA ; This provides GPU acceleration using a Moore Threads GPU. |
| opencl |
ggml: use OpenCL; This provides GPU acceleration through OpenCL on recent Adreno GPU. |
| opencl-embed-kernels |
ggml: embed kernels |
| opencl-profiling |
ggml: use OpenCL profiling (increases overhead) |
| opencl-use-adreno-kernels |
ggml: use optimized kernels for Adreno |
| rpc |
ggml: use RPC |
| server |
ggml: build examples ; "llama: build server example" |
| static |
static build? |
| utils |
Build also llama.cpp utils (scripts) for performing operations on models. |
| vulkan |
ggml: use Vulkan ; use CMAKE_EXTRA_CACHE_FILE env variable (check https://gitweb.gentoo.org/repo/gentoo.git/tree/eclass/cmake.eclass ) to specify next variables: GGML_VULKAN_SHADERS_GEN_TOOLCHAIN "" "ggml: toolchain file for vulkan-shaders-gen" |
| vulkan-check-results |
ggml: run Vulkan op checks |
| vulkan-debug |
ggml: enable Vulkan debug output |
| vulkan-memory-debug |
ggml: enable Vulkan memory debug output |
| vulkan-perf |
ggml: enable Vulkan perf output |
| vulkan-run-tests |
ggml: run Vulkan tests |
| vulkan-shader-debug-info |
ggml: enable Vulkan shader debug info |
| vulkan-validate |
ggml: enable Vulkan validation |
| webgpu |
The WebGPU backend relies on Dawn: https://dawn.googlesource.com/dawn |