High Performance Computing for Endpoints with DigAn®

Introduction

High Performance Computing (HPC) has evolved from being a buzzword to a real-world necessity for hardware and software developers working on application development and deployment across various infrastructures. These infrastructures range from resource-constrained edge and micro-edge devices to on-premises setups, pure cloud, or hybrid environments.

‍Why HPC Infrastructure Matters

HPC infrastructure is crucial for maximizing throughput, optimizing resources (scaling up or down as needed), enhancing duty cycles, and minimizing ownership costs. It also ensures low-power, low-latency, secure environments, especially for HPC-specific applications. But what makes an HPC application work? And what are the enablers for building an HPC environment?

‍The Challenge of HPC in Endpoint Devices

For the software community, enabling an HPC environment on endpoint devices or quantum computers is a significant challenge. On edge devices, developers face resource limitations, with computing power often taking a backseat to other essential tasks. In quantum computing, secondary resources like input sequencing and output handling take precedence, with computation becoming a lesser priority.

In real-time scenarios, both edge and quantum computing environments aim to prove their worth given the substantial investment required. However, traditional cloud or data center infrastructures often provide solutions to these challenges by supporting parallel and distributed processing, memory management, interconnects, security, and monitoring.

‍HPC for Endpoints: Ambient Scientific’s GPX Series

Bringing the focus back to HPC for endpoints, Ambient Scientific’s GPX series AI devices stand out. These devices are architected to handle workloads, particularly AI and ML (mostly vector) tasks, in a predetermined, scalable manner. They also provide the essential development environment needed to support HPC for endpoint devices.

As a fully HPC-compliant infrastructure, Ambient offers cost-efficient, low-latency, minimal power consumption solutions, coupled with a secure and scalable software stack.

‍

‍

Optimized AI/ML Model Development for Edge Devices

One of the key challenges for data scientists (DS) lies in developing applications for resource-constrained edge devices. Pre-trained models such as ResNet, MobileNet, or AlexNet are not designed to run efficiently on edge devices. To overcome this, developers often employ model optimization techniques like Distillation, along with PCQAT (Pruning & Clustering Quantization-Aware Training), PQAT (Pruning Preserved Quantization-Aware Training), or CQAT (Clustering Preserved Quantization-Aware Training). However, even though these state-of-the-art (SOTA) techniques may reduce model size to 10% of original, making deployment on endpoint devices a significant task. This is where Ambient’s toolchain, libraries, and frameworks come into play. They simplify the process for data scientists to build custom AI/ML models that fit within the constraints of edge devices.

‍Ambient’s End-to-End HPC Toolchain

Ambient offers both HPC-compliant hardware and a complete HPC toolchain to enable seamless AI/ML application development. The toolchain includes libraries, frameworks, a compiler, simulators, and tutorials, all optimized for the GPX infrastructure. Developers can create, test, and deploy HPC applications end-to-end using these tools, without the need for external resources.

‍Building HPC Applications: Key Steps and Tools

Developing HPC applications requires a robust software infrastructure. Here's how Ambient supports each phase of the model development lifecycle:

‍

1. Data Collection: Ambient offers a SOTA data collection toolchain that simplifies gathering real-time, authentic data for model building

‍

2. Pre-processing & Featurization: Ambient provides pre-built, optimized libraries like aBLAS, aDNN, and aDSP, which are tailored for MX8 and ARM cores, allowing developers to select libraries based on their use cases.

‍

3. Model Building: Ambient’s comprehensive datasheets guide developers in building models that are compatible with the GPX board. A GPX-compatible simulator enables early testing, offering a go/no-go decision point for developers.

‍

4. Model Testing & Deployment: Ambient's custom compiler toolchain is designed for GPX models. It supports quantization, clustering, scheduling, and other processes, providing full visibility during model execution. Developers can debug and observe the model at each stage to ensure flawless execution.

Additionally, Ambient provides a range of generic libraries, such as drivers, calibrators, auto-tuners, and event monitoring tools, to facilitate the development journey. Sample applications for speech recognition, computer vision, and sensor fusion further assist developers in adapting to the GPX environment.

‍Optimized HPC Experience with Ambient Scientific

Ambient Scientific delivers an unparalleled experience for developers, ensuring ease of adaptation to the GPX environment while building HPC models. The GPX ecosystem is designed to offer optimized libraries, low-power inference capabilities, efficient pipelining, and low-latency execution. With a custom HPC-enabled compiler, the GPX series intelligently utilizes available resources, enabling developers to maximize performance while minimizing power consumption.

‍