Update Bing Search Infrastructure: Implementation of Large and Small Language models to improve performance

Microsoft has announced a Bing search infrastructure update, including the introduction of large and small and small language models (LLMS and SLMS), as well as new optimization methods. This update aims to improve performance and reduce search costs.

Using LLMS in search engines can create problems with speed and cost. To solve these problems, Bing taught SLMS, which, according to them, is 100 times faster than LLMS. Bing also uses Nvidia tensorrt-llm to improve SLMS. Tensorrt -LLM is a tool that helps to reduce the time and cost of large models on NVIDIA GPUS.

"LLMS can be expensive and slow. To improve efficiency, we have taught SLM (~ 100x Improvement LLM capacity) that process and understand search queries more precisely."

🚀 According to Microsoft technical report, the integration of NVIDIA Tensorrt-LLM technology has improved the company "Deep Search". "Deep Search" uses SLMS in real time to provide appropriate web results. Before optimizing, the original Bing Transformer model had a 95th percentage of 4.76 seconds per party (20 requests) and a bandwidth of 4.2 requests per second per instance. With Tensorrt-LLM, the delay has decreased to 3.03 seconds per party, and the capacity increased to 6.6 requests per second per instance. This means reducing the delay by 36% and reducing the operative costs by 57%.

• 📌 Bing update leads to faster search results with optimized conclusion and faster reaction time.
• 📌 Improved accuracy due to the increased capabilities of SLM models that provide more contextualized results.
• 📌 Cost efficiency that allows Bing to invest in further innovation and improvement.