Editing Efficiency (section)

== GPU power limits vs. efficiency ==

Over the last decade or so, there has been several studies regarding setting arbitrary limits on GPU power consumption (possible via nvidia-smi) in order to increase the amount of performance per watt.

The last 5-10% of power fed to a GPU has significantly diminishing returns compared to the first 90%; this [https://www.pugetsystems.com/labs/hpc/nvidia-gpu-power-limit-vs-performance-2296/#GPU_Power_Scaling_vs_Performance_Results set of graphs for A4000s and RTX 3090s] shows this general trend.

As an example, [https://www.pugetsystems.com/labs/hpc/quad-rtx3090-gpu-power-limiting-with-systemd-and-nvidia-smi-1983/ reducing the power limit from 366W to 270~280W on an RTX3090 only loses approx. 5% performance].

This effect is '''more significant on consumer-oriented cards''' (RTX 3090, 4090 etc) due to datacenter cards such as A4000s, A5000s often already being built upon [https://www.techspot.com/review/2342-nvidia-rtx-a4000/#:~:text=Earlier%20this%20year%20Nvidia%20dropped,192%20TAUs%20and%2096%20ROPs. consumer cards that have been power-limited]. For example, the RTX A4000 is a low-power version of an RTX 3070.

Limiting maximum GPU power and using the additional power overhead to add another GPU is one potential option to increase performance where power-limited, or simply to significantly reduce noise/heat without impacting performance.