7. Machine Learning

7.1. Introduction

The Data Plane Development Kit (DPDK) is an open-source software project managed by the Linux Foundation. It is designed to offload TCP packet processing from the operating system kernel to user-space processes, thereby enhancing computing efficiency and packet throughput.

The dpdk-test-mldev tool is a DPDK application designed to test various machine learning (mldev) use cases. As part of the DAO package, it provides a way for users to run inference operations with specific inputs. The current DAO release provides int8 ( resnet50_int8_t08_b01 ) and fp16 ( resnet50_fp16_t08_b01 ) quantized versions of Resnet50 models, which can be used for running inference operations.

Note

For detailed documentation related to dpdk-test-mldev, refer to documentation

7.2. Preprocessing of Input

Involves converting the input image format to a binary format that the model can accept as an input. This is done using the image2bin.py Python script.

# Convert input in Image format to binary format
python image2bin.py \
--image_file input.jpeg \
--bin_file output.bin

7.3. Model Execution

The preprocessed binary is given as an input to the model. The model processes the input, runs the inference operation, and generates the output in binary format.

# Run inferences with dpdk-test-mldev application
dpdk-test-mldev --lcores=4-23 -a 0000:00:10.0,fw_path=/lib/firmware/mlip-fw.bin -- \
--test inference_ordered \
--filelist model.tar,input.bin,output.bin,reference.bin \
--tolerance 5 \
--stats \
--repetitions 1000

7.4. Postprocessing of Output

The binary output generated by the model is converted into a JSON file for easier interpretation and analysis. This is done using the bin2json.py Python script.

# Convert output in binary format to JSON format
python bin2json.py \
--bin_file output.bin \
--json_file output.json