Tech Bytes from Intel reveals new tech updates

Robert Hallock, vice president and general manager of AI and technical marketing in the Client Computing Group at Intel, shares insights on OpenVINO 2024.0.0, Intel’s NPU Acceleration Library, and a security innovation in collaboration with CrowdStrike.

Tech Bytes from Intel: OpenVINO 2024 release adds Intel Core Ultra NPU support and deep Gen AI focus

"Debuting in 2018, OpenVINO is the longest-serving and most mature toolkit for optimizing and deploying AI models on the PC. The OpenVINO 2024.0 release brings several important improvements to developers, including: hardware targeting for the Intel Core Ultra NPU, significant performance and feature enhancements for generative AI, and even easier integration with 3rd party model repos like HuggingFace. These advancements demonstrate that OpenVINO is a leader amongst toolkits for AI PC software development," said Robert Hallock.

Major features and benefits:

• New support for the Intel Core Ultra Neural Processing Unit (NPU)
• New validated models, including Mistral, StableLM-tuned-alpha-3b, StableLM-Epoch-3B
• "Activation-aware Weight Quantization" improves speed and accuracy of LLMs quantized on INT4
• Generally improved LLM performance
• Easier optimization and conversion of models from the popular HuggingFace repo
  
   

Intel makes it easier for AI developers to get familiar with the Core Ultra NPU

"The Intel NPU Acceleration Library is a great way for developers to familiarize themselves with offline AI acceleration via NPU. The library is designed to provide an easy on-ramp to utilizing the NPU, which may be unfamiliar territory for developers accustomed to accelerating inferencing with the GPU. By providing a simple on-ramp, developers can quickly leverage existing code to jumpstart their NPU experience on popular models like LLaMA 2-7B, " said Robert Hallock, vice president and general manager of AI and technical marketing in the Client Computing Group at Intel.

Major features and benefits:

• Windows and Linux support
• Support for all Intel Core Ultra processors with integrated NPU
• Support for LLMs/SLMs like TinyLlama, Phi-2, Gemma, etc.
• PyTorch and Transformers integration
• Designed for developer experimentation with NPU acceleration
  
   

Intel and CrowdStrike pioneer use of NPU to improve PC security and performance

This breakthrough unlocks a realm of new possibilities, moving endpoint detection closer to the source while sending relevant data to the cloud for better in-depth analysis. Furthermore, the decision to dispatch additional data to the cloud is now AI-driven, replacing fixed rules. This empowers expanded, selective cloud data collection, ensuring scalability and minimizing network traffic," said Robert Hallock.

Major features and benefits:

• 3-way experiment by Dell, Intel, and CrowdStrike
• Threat detection and isolation for in-memory attacks
• Uses convolutional neural network (CNN) models for analysis, which are computationally costly (~20% CPU impact)
• Offload CNNs to NPU -> restored ~20% CPU performance, improved model accuracy with dedicated AI resources
• Improves threat analysis performance by reducing cloud traffic and latency