Posted on Behalf of Michael Schulte, Sr. Fellow Engineer for AMD
Recent research has demonstrated that narrow data formats (e.g., formats using 8 bits or less per number) have the potential to greatly improve the performance and energy efficiency of AI training and inference with a negligible impact on accuracy [1][2]. Narrow data formats also help reduce the amount of memory used for AI models, while increasing the effective memory and network bandwidth, since fewer bits of data need to be stored and transmitted.
more