Skip to content

NHWC 1x1 conv with padding triggers CpuGemmLowpMatrixMultiplyCore::validate mismatch #1253

@alvoron

Description

@alvoron

When using NHWC, 1x1 kernel, stride=1, and non-zero padding, NEConvolutionLayer::validate can fail with:

  "in validate src/cpu/operators/CpuGemmLowpMatrixMultiplyCore.cpp:524: a->dimension(1) != output->dimension(1)"

This happens because the GEMM3D fast path (skip_im2col/skip_col2im) assumes output spatial equals input spatial.
With padding, output W/H increase, violating that assumption.

Reproducer below triggers NEConvolutionLayer::validate to return this error:

#include "arm_compute/core/Error.h"
#include "arm_compute/core/TensorShape.h"
#include "arm_compute/core/Types.h"
#include "arm_compute/runtime/Tensor.h"
#include "arm_compute/runtime/NEON/NEFunctions.h"
#include <iostream>

using namespace arm_compute;

int main() {
    // Reproducer for CpuGemmLowpMatrixMultiplyCore::validate failure when
    // skip_im2col/skip_col2im is used with NHWC 1x1 conv + padding.
    TensorInfo src_info(TensorShape(1, 100, 128, 1), 1, DataType::QASYMM8, DataLayout::NHWC);
    TensorInfo wei_info(TensorShape(1, 1, 1, 64), 1, DataType::QASYMM8, DataLayout::NHWC);
    TensorInfo dst_info(TensorShape(64, 102, 130, 1), 1, DataType::QASYMM8, DataLayout::NHWC);

    src_info.set_quantization_info(QuantizationInfo(1.0f, 0));
    wei_info.set_quantization_info(QuantizationInfo(1.0f, 0));
    dst_info.set_quantization_info(QuantizationInfo(1.0f, 0));

    PadStrideInfo conv_info(1, 1, 1, 1, 1, 1, DimensionRoundingType::FLOOR);
    ActivationLayerInfo act_info;
    WeightsInfo weights_info;

    auto status = NEConvolutionLayer::validate(&src_info, &wei_info, nullptr, &dst_info, conv_info, weights_info, Size2D(1, 1), act_info);
    if (!status) {
        std::cout << "VALIDATE FAILED: " << status.error_description() << std::endl;
        return 1;
    }
    return 0;
}

As a potential fix, I've added padding and dimension check in skip_im2col initialization in CpuGemmConv2d::skip_im_col_info():

    const bool has_padding   = conv_info.pad_left() != 0 || conv_info.pad_right() != 0 ||
                               conv_info.pad_top() != 0 || conv_info.pad_bottom() != 0;
    const bool same_spatial  = (conv_w == src->dimension(idx_width)) && (conv_h == src->dimension(idx_height));
    const bool skip_im2col   = (data_layout == DataLayout::NHWC && kernel_width == 1 && kernel_height == 1 &&
                               conv_info.stride().first == 1 && conv_info.stride().second == 1 &&
                               !has_padding && same_spatial);

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions