Quickstart C++ Wrapper Backend Tutorial - Engine Initialization and Benchmark Description

Table of Contents

• Steps
  • 1. Benchmark description initialization
  • 2. Benchmark description overrides
  • 3. Engine initialization
• Tutorial steps

Steps

1. Benchmark description initialization

The first step for our new benchmark is to fill out the benchmark description. TutorialEltwiseAddBenchmarkDescription defined in tutorial_eltwiseadd_benchmark_seal.h inherits from hebench::cpp::BenchmarkDescription.

In the constructor we should initialize and set the inherited m_descriptor object. We currently have a description already in place from the original example to which we will make some changes for our SEAL-based workload.

TutorialEltwiseAddBenchmarkDescription is declared in tutorial_eltwiseadd_benchmark_seal.cpp.

First, we initialize the descriptor object memory:

    // initialize the descriptor for this benchmark
    std::memset(&m_descriptor, 0, sizeof(hebench::APIBridge::BenchmarkDescriptor));

hebench::APIBridge::BenchmarkDescriptor.workload: specifies which workload this benchmark will perform. Workloads supported by your current HEBench version are defined under hebench::APIBridge::Workload in hebench/api_bridge/types_seal.h. For our example we will set it to hebench::APIBridge::Workload::EltwiseAdd:

    m_descriptor.workload = hebench::APIBridge::Workload::EltwiseAdd;

hebench::APIBridge::BenchmarkDescriptor.data_type: tells the front end what type of data the benchmark is expecting to recieve and return. The possible types are defined in hebench::APIBridge::DataType.

    m_descriptor.data_type = hebench::APIBridge::DataType::Int64;

hebench::APIBridge::BenchmarkDescriptor.category: Specifies one of the benchmark categories defined under hebench::APIBridge::Category. For our first benchmark we are setting it to offline.

    m_descriptor.category = hebench::APIBridge::Category::Offline;

hebench::APIBridge::BenchmarkDescriptor.cat_params.offline: These are parameters which are relevant to tests of the offline category. These specify the supported number of samples (or batch sizes) for each of the operation parameters. These are arbitrary and specific to the benchmark we want to create. See the documentation for more details.

Since our initial workflow forced sample sizes of 2 samples for the first operand and 5 for the second, we are setting them the same here. If our benchmark can support variable sample sizes for any operand, we set the corresponding value to 0. The specific size can be set in a configuration file passed when running this benchmark. For more information regarding benchmark configuration files see Benchmark Configuration File Reference .

    m_descriptor.cat_params.offline.data_count[0] = 2; // 2 data samples for op parameter 0
    m_descriptor.cat_params.offline.data_count[1] = 5; // 5 data samples for op parameter 1

hebench::APIBridge::BenchmarkDescriptor.cipher_param_mask: Specifies which operands will be plain text and which will be encrypted. This is a bit mask where the bit position represents the operation parameter, and its value represents whether it is plain text (0) or encrypted (1). For this tutorial we want first operand plain text and second encrypted.

    m_descriptor.cipher_param_mask = 1 << 1;

Note that we are setting the mask to 1 << 1 or 0000 0010 in binary, indicating that operation parameter in position 0 is plain text, and postion 1 is encrypted. The rest of the bits are ignored if there are not corresponding operation parameters.

hebench::APIBridge::BenchmarkDescriptor.scheme: Specifies the scheme the test will use. It must be one of the schemes registered during engine initialization earlier.

    m_descriptor.scheme = HEBENCH_HE_SCHEME_BFV;

hebench::APIBridge::BenchmarkDescriptor.security: Specifies what security level the benchmark uses. It must be one of the security levels registered during engine initialization earlier.

    m_descriptor.security = TUTORIAL_HE_SECURITY_128;

hebench::APIBridge::BenchmarkDescriptor.other: This is an optional parameter which can be used for a backend defined purpose to differentiate benchmarks in additional backend-specific ways.

    m_descriptor.other = 0; // no extras needed for our purpose:
        // Other backends can use this field to differentiate between
        // benchmarks for which internal parameters, not specified by
        // other fields of this structure, differ.

Finally, a workload that requires flexible workload parameters must specify, at least, one set of default arguments. Workload parameters are flexible enough for backends to specify other parameters beyond the mandatory requirements, such as, but not limited to parameters that could be used for optimization like padding size, extra memory, etc.

According to Vector Element-wise Addition Workload , element-wise addition requires the number of elements for a vector. For this tutorial, the required parameter is enough, so, we do not need to create any extra workload parameters.

We create a single default set with a default value for the vector size:

    // specify default arguments for this workload flexible parameters:
    hebench::cpp::WorkloadParams::EltwiseAdd default_workload_params;
    default_workload_params.n() = 400;
    this->addDefaultParameters(default_workload_params);

Since the value in our original workload was 400, we are setting it the same as default. If our implementation happens to support only vectors of this size, we can enforce it during benchmark initialization itself.

The complete implementation of our Benchmark description is provided below

TutorialEltwiseAddBenchmarkDescription::TutorialEltwiseAddBenchmarkDescription()
{
    // initialize the descriptor for this benchmark
    std::memset(&m_descriptor, 0, sizeof(hebench::APIBridge::BenchmarkDescriptor));
 
    m_descriptor.workload = hebench::APIBridge::Workload::EltwiseAdd;
 
    m_descriptor.data_type = hebench::APIBridge::DataType::Int64;
 
    m_descriptor.category = hebench::APIBridge::Category::Offline;
 
    m_descriptor.cat_params.offline.data_count[0] = 2; // 2 data samples for op parameter 0
    m_descriptor.cat_params.offline.data_count[1] = 5; // 5 data samples for op parameter 1
 
    m_descriptor.cipher_param_mask = 1 << 1;
 
    //
    m_descriptor.scheme = HEBENCH_HE_SCHEME_BFV;
 
    m_descriptor.security = TUTORIAL_HE_SECURITY_128;
 
    m_descriptor.other = 0; // no extras needed for our purpose:
        // Other backends can use this field to differentiate between
        // benchmarks for which internal parameters, not specified by
        // other fields of this structure, differ.
 
    // specify default arguments for this workload flexible parameters:
    hebench::cpp::WorkloadParams::EltwiseAdd default_workload_params;
    default_workload_params.n() = 400;
    this->addDefaultParameters(default_workload_params);
}

2. Benchmark description overrides

We must override methods to create and destroy our actual benchmark. These will be called by C++ wrapper's BaseEngine implementation when the benchmark needs to be instantiated to execute, and disposed of when completed.

Our create method follows:

hebench::cpp::BaseBenchmark *TutorialEltwiseAddBenchmarkDescription::createBenchmark(hebench::cpp::BaseEngine &engine,
                                                                                     const hebench::APIBridge::WorkloadParams *p_params)
{
    if (!p_params)
        throw hebench::cpp::HEBenchError(HEBERROR_MSG_CLASS("Invalid empty workload parameters. This workload requires flexible parameters."),
                                         HEBENCH_ECODE_CRITICAL_ERROR);
 
    TutorialEngine &ex_engine = dynamic_cast<TutorialEngine &>(engine);
    return new TutorialEltwiseAddBenchmark(ex_engine, m_descriptor, *p_params);
}

This is the destroy method:

void TutorialEltwiseAddBenchmarkDescription::destroyBenchmark(hebench::cpp::BaseBenchmark *p_bench)
{
    if (p_bench)
        delete p_bench;
}

There are other methods that we can override to modify the behavior of the description class. Check the documentation on hebench::cpp::BenchmarkDescription for more information.

In this tutorial we are overriding hebench::cpp::BenchmarkDescription::getBenchmarkDescription() to output extra information customized for our example.

std::string TutorialEltwiseAddBenchmarkDescription::getBenchmarkDescription(const hebench::APIBridge::WorkloadParams *p_w_params) const
{
    std::stringstream ss;
    ss << BenchmarkDescription::getBenchmarkDescription(p_w_params);
    ss << ", Encryption parameters" << std::endl
       << ", , HE Library, Microsoft SEAL 3.6" << std::endl
       << ", , Poly modulus degree, " << PolyModulusDegree << std::endl
       << ", , Coefficient Moduli, 60, 60";
    if (p_w_params)
    {
        hebench::cpp::WorkloadParams::EltwiseAdd w_params(*p_w_params);
        ss << std::endl
           << ", Workload parameters" << std::endl
           << ", , n, " << w_params.n();
    } // end if
    return ss.str();
}

3. Engine initialization

At this point, we can start filling out new information for our specific benchmark inside the engine. This is required to make our benchmark visible to the Test Harness since the engine is responsible for managing the benchmarks' lifespans. First we are going to add options to the TutorialEngine::init() method which implements the hebench::cpp::BaseEngine::init() virtual method. The order of initialization in the next sections can be any.

Inside of TutorialEngine::init() we will see the following code already populated from our default example:

void TutorialEngine::init()
{
    // add any new error codes: use
    // this->addErrorCode(code, "generic description");
 
    // add supported schemes
    addSchemeName(HEBENCH_HE_SCHEME_PLAIN, "Plain");
 
    // add supported security
    addSecurityName(HEBENCH_HE_SECURITY_NONE, "None");
 
    // add the all benchmark descriptors
    addBenchmarkDescription(std::make_shared<TutorialEltwiseAddBenchmarkDescription>());
}

For this tutorial, we will make a few changes as we plan to perform our operations homomorphically.

First, we add any new error codes which we want our engine to be capable of returning back to the frontend. For our example we define one new error as shown below. These errors are defined inside of tutorial_error_seal.h.

    // add any new error codes
 
    addErrorCode(TUTORIAL_ECODE_SEAL_ERROR, "SEAL error.");

Next, we add the scheme names we want to support. For our simple example we will just add BFV scheme which is predefined as part of hebench::APIBridge namespace. Any other schemes can be custom defined as well.

    // add supported schemes
 
    //addSchemeName(HEBENCH_HE_SCHEME_PLAIN, "Plain");
    addSchemeName(HEBENCH_HE_SCHEME_BFV, "BFV");

We then add our supported security levels. For our new backend we specify a flag for 128-bit security inside of tutorial_engine_seal.h and then pass it here.

    // add supported security
 
    //addSecurityName(HEBENCH_HE_SECURITY_NONE, "None");
    addSecurityName(TUTORIAL_HE_SECURITY_128, "128 bits");

Last, in init() we add all of the supported benchmarks to the engine. For our example we will be adding just the single example benchmark but as we expand our backend and add more benchmarks, we would add them to this section in the same fashion as shown.

    // add the all benchmark descriptors
    addBenchmarkDescription(std::make_shared<TutorialEltwiseAddBenchmarkDescription>());

The full code for our engine init function looks like this:

void TutorialEngine::init()
{
    // add any new error codes
 
    addErrorCode(TUTORIAL_ECODE_SEAL_ERROR, "SEAL error.");
 
    // add supported schemes
 
    //addSchemeName(HEBENCH_HE_SCHEME_PLAIN, "Plain");
    addSchemeName(HEBENCH_HE_SCHEME_BFV, "BFV");
 
    // add supported security
 
    //addSecurityName(HEBENCH_HE_SECURITY_NONE, "None");
    addSecurityName(TUTORIAL_HE_SECURITY_128, "128 bits");
 
    // add the all benchmark descriptors
    addBenchmarkDescription(std::make_shared<TutorialEltwiseAddBenchmarkDescription>());
}

Tutorial steps

Tutorial Home
Preparation
Engine Initialization and Benchmark Description
Benchmark Implementation
File References