hebench::APIBridge Namespace Reference

Namespaces
	WorkloadParamType

Classes
struct	WorkloadParam
	Defines a single workload parameter. More...
struct	WorkloadParams
	Specifies the parameters for a workload. More...
struct	CategoryParams
	Specifies parameters for a category. More...
struct	BenchmarkDescriptor
	Defines a benchmark test. More...
struct	_FlexibleData
	Structure to contain flexible data. More...
struct	DataPack
	Defines a data package for an operation. More...
struct	DataPackCollection
	Defines a collection of data packs. More...
struct	ParameterIndexer
	Indexes a DataPack. More...
union	WorkloadParam.__unnamed5__
union	CategoryParams.__unnamed7__
struct	CategoryParams.__unnamed7__.latency
struct	CategoryParams.__unnamed7__.offline

Typedefs
typedef std::int32_t	ErrorCode
	Return value for API bridge functions. More...
typedef std::int32_t	Scheme
	Open-ended homomorphic encryption scheme ID. More...
typedef std::int32_t	Security
	Open-ended homomorphic encryption scheme security ID. More...
typedef _FlexibleData	NativeDataBuffer
	Represents a native data buffer maintained by the Test Harness. More...
typedef _FlexibleData	Handle
	Encapsulates an opaque handle to backend data. More...

Enumerations
enum	Workload {   MatrixMultiply = 1 , EltwiseAdd , EltwiseMultiply , DotProduct ,   LogisticRegression , LogisticRegression_PolyD3 , LogisticRegression_PolyD5 , LogisticRegression_PolyD7 ,   SimpleSetIntersection , Generic }
	Defines all possible workloads. More...
enum	DataType { Int32 = 1 , Int64 , Float32 , Float64 }
	Defines data types for a workload. More...
enum	Category { Latency = 1 , Offline }
	Defines all possible categories for each workload. More...

Functions
ErrorCode	destroyHandle (Handle h)
	Releases resources held by the specified handle. More...
ErrorCode	initEngine (Handle *h_engine, const int8_t *p_buffer, uint64_t size)
	Initializes the backend engine. More...
ErrorCode	subscribeBenchmarksCount (Handle h_engine, std::uint64_t *p_count)
	Retrieves the number of benchmarks for which the backend is registering to perform. More...
ErrorCode	subscribeBenchmarks (Handle h_engine, Handle *p_h_bench_descs, std::uint64_t count)
	Retrieves handles to the benchmark descriptions for which the backend is registering to perform. More...
ErrorCode	getWorkloadParamsDetails (Handle h_engine, Handle h_bench_desc, std::uint64_t *p_param_count, std::uint64_t *p_default_count)
	Retrieves details about the flexible parameters supported by this workload. More...
ErrorCode	describeBenchmark (Handle h_engine, Handle h_bench_desc, BenchmarkDescriptor *p_bench_desc, WorkloadParams *p_default_params, std::uint64_t default_count)
	Retrieves the concrete description for a benchmark registered by backend. More...
ErrorCode	createBenchmark (Handle h_engine, Handle h_bench_desc, const WorkloadParams *p_params, Handle *h_benchmark)
	Instantiates a benchmark on the backend. More...
ErrorCode	initBenchmark (Handle h_benchmark, const BenchmarkDescriptor *p_concrete_desc)
	Allows the benchmark to perform initialization steps based on the finalized, concrete description. More...
ErrorCode	encode (Handle h_benchmark, const DataPackCollection *p_parameters, Handle *h_plaintext)
	Given a pack of parameters in raw, native data format, encodes them into plain text suitable for backend encryption or operation. More...
ErrorCode	decode (Handle h_benchmark, Handle h_plaintext, DataPackCollection *p_native)
	Decodes plaintext data into the appropriate raw, native format. More...
ErrorCode	encrypt (Handle h_benchmark, Handle h_plaintext, Handle *h_ciphertext)
	Encrypts a plain text into a cipher text. More...
ErrorCode	decrypt (Handle h_benchmark, Handle h_ciphertext, Handle *h_plaintext)
	Decrypts a cipher text into corresponding plain text. More...
ErrorCode	load (Handle h_benchmark, const Handle *h_local_packed_params, std::uint64_t local_count, Handle *h_remote)
	Loads the specified data from the local host into the remote backend to use as parameter during a call to operate(). More...
ErrorCode	store (Handle h_benchmark, Handle h_remote, Handle *h_local_packed_params, std::uint64_t local_count)
	Retrieves the specified data from the backend. More...
ErrorCode	operate (Handle h_benchmark, Handle h_remote_packed_params, const ParameterIndexer *p_param_indexers, uint64_t indexers_count, Handle *h_remote_output)
	Performs the workload operation of the benchmark. More...
std::uint64_t	getSchemeName (Handle h_engine, Scheme s, char *p_name, std::uint64_t size)
	Retrieves the name of a specified scheme ID from the backend. More...
std::uint64_t	getSchemeSecurityName (Handle h_engine, Scheme s, Security sec, char *p_name, std::uint64_t size)
	Retrieves the name of the specified security for the scheme ID from the backend. More...
std::uint64_t	getBenchmarkDescriptionEx (Handle h_engine, Handle h_bench_desc, const hebench::APIBridge::WorkloadParams *p_w_params, char *p_description, std::uint64_t size)
	Retrieves backend specific text description for a benchmark descriptor. More...
std::uint64_t	getErrorDescription (Handle h_engine, ErrorCode code, char *p_description, std::uint64_t size)
	Retrieves the general error description of an error code. More...
std::uint64_t	getLastErrorDescription (Handle h_engine, char *p_description, std::uint64_t size)
	Retrieves the detailed description of the last error that occurred during an operation on the engine. More...

---

Class Documentation

hebench::APIBridge::WorkloadParam

struct hebench::APIBridge::WorkloadParam

Defines a single workload parameter.

This struct holds the value and meta data for a workload parameter. The correct type is workload specific and specified per workload.

See also

WorkloadParams

Definition at line 317 of file types.h.

Class Members
union WorkloadParam	__unnamed__
WorkloadParamType	data_type	Type of the parameter data.
char	name[HEBENCH_MAX_BUFFER_SIZE]	Null-terminated string containing the name for the parameter. This name if for description purposes only and should not be used as identifier or any other specific purpose.

hebench::APIBridge::WorkloadParams

struct hebench::APIBridge::WorkloadParams

Specifies the parameters for a workload.

A workload can have 0 or more flexible parameters that can be specified during benchmark initialization; for example, vector element-wise operations have the number of elements in a vector as a parameter.

The number of parameters is workload specific.

Definition at line 362 of file types.h.

Collaboration diagram for hebench::APIBridge::WorkloadParams:

Class Members
uint64_t	count	Number of workload parameters.
WorkloadParam *	params	Parameters for the workload.

hebench::APIBridge::CategoryParams

struct hebench::APIBridge::CategoryParams

Specifies parameters for a category.

This type contains a union to accomodate parameters for all categories. Clients must be mindful that writing data to parameters for a category will overwrite data for all other categories in the same union, thus, separate instaces of this type should be used for different categories.

Definition at line 421 of file types.h.

Class Members
union CategoryParams	__unnamed__
uint64_t	min_test_time_ms	Specifies the minimum time, in milliseconds, to run the test. Latency benchmark will submit an operation with the same set of inputs as many times as needed until the time elapsed during the test is, at least, the number of milliseconds specified. Offline benchmark will submit the complete dataset for the test in a single operation. Test Harness will continue to submit operation requests on the whole dataset until the time elapsed during the test is, at least, the number of milliseconds specified. A value of 0, indicates that the minimum test time is user-specified at run-time via configuration files. Regardless of the time specified, for latency category, Test Harness will submit, at least, two iterations; and for offline category, Test Harness will submit, at least, one iteration. It is clear that the full test will take, at least, as much time as specified by this field, since the last operation request that satisfies the elapsed time will not be interrupted until it completes.

hebench::APIBridge::BenchmarkDescriptor

struct hebench::APIBridge::BenchmarkDescriptor

Defines a benchmark test.

A benchmark test is defined by a Workload, a Category for said workload, a scheme for such Category, which inputs are plain text of cipher text, the security level for the scheme, and an extra parameter that is backend specific.

cipher_param_mask is a bit mask that defines which operation parameters are cipher or plain text. If bit i is set, then parameter i is cipher text, otherwise, it is plain text. Only the least significant n bits of this field will be used for an operation that supports n operands. The rest are ignored.

Test Harness will request the list of BenchmarkDescriptor objects for which a backend is subscribing. When Test Harness is about to execute a benchmark, the BenchmarkDescriptor is passed down to the backend with the same values used during subscription.

Fields workload, data_type, category, cat_params and cypher_param_mask specify the actual operation for the benchmark that Test Harness will execute. Different combination of values specified by scheme, security, and other fields do not affect the type of benchmark, but further describe it.

A benchmark is defined by the combination of the values for all these fields, thus, two benchmarks are considered the same if all fields have the same value. So, if we are running two benchmark with workload matrix multiplication, latency category, CKKS scheme with 128 bits security and BFV scheme with 128 bits security, then scheme field for both benchmarks must be different. Note that two benchmarks with the same descriptor can differ in the workload parameters, effectively performing the same benchmark on a workload with different parameter sets.

See also

WorkloadParams

Definition at line 526 of file types.h.

Collaboration diagram for hebench::APIBridge::BenchmarkDescriptor:

Class Members
CategoryParams	cat_params	Parameters for the category.
Category	category	Category for the benchmark.
uint32_t	cipher_param_mask	Input mask to define which operation parameters for the computation are plain text or cipher text.
DataType	data_type	Data type for the workload.
int64_t	other	Backend specific extra parameter.
Scheme	scheme	Scheme for the benchmark.
Security	security	Security for the scheme.
Workload	workload	Workload for the benchmark.

hebench::APIBridge::_FlexibleData

struct hebench::APIBridge::_FlexibleData

Structure to contain flexible data.

What all of this structure's fields actually represent, is custom-defined by the structure user. Field names are just provided as a guideline. It is the client's responsibility to define the meaning of each field based on where is this structure used and to adhere to this definition.

Definition at line 551 of file types.h.

Class Members
void *	p	Pointer to underlying data. While this field does not necessarily need to point to any data (or contain a valid pointer value), it usually points to underlying data buffer(s).
uint64_t	size	Size of underlying data. Usually specifies the size, in bytes, of the underlying data represented by this structure.
int64_t	tag	Optional tag. Flexible extra field to allow further customization of this structure.

hebench::APIBridge::DataPack

struct hebench::APIBridge::DataPack

Defines a data package for an operation.

This data pack contains a list of samples (one sample per NativeDataBuffer) for the same component.

Field param_position specifies the zero-based position of the data as a parameter in the call order for the operation to execute, or the zero-based componenet in the result. For example, in (R0, R1) = op(A, B, C), based on the input components: A is at position 0, B is at position 1, and C is at 2; and for the output components: R0 is at position 0, and R1 is at position 1.

Definition at line 610 of file types.h.

Collaboration diagram for hebench::APIBridge::DataPack:

Class Members
uint64_t	buffer_count	Number of data buffers in p_buffers.
NativeDataBuffer *	p_buffers	Array of data buffers for parameter.
uint64_t	param_position	The 0-based position of this parameter in the corresponding function call.

hebench::APIBridge::DataPackCollection

struct hebench::APIBridge::DataPackCollection

Defines a collection of data packs.

This struct is used to encode/decode a collection of data packs, in a single call instead of encoding each data pack at a time. This is useful for backends to optimize large amounts of data.

Definition at line 623 of file types.h.

Collaboration diagram for hebench::APIBridge::DataPackCollection:

Class Members
DataPack *	p_data_packs	Collection of data packs.
uint64_t	pack_count	Number of data packs in the collection.

hebench::APIBridge::ParameterIndexer

struct hebench::APIBridge::ParameterIndexer

Indexes a DataPack.

In a call to operate(), a list of ParameterIndexer is passed, where each element of that list corresponds to an input parameter position in the call order. For that call to operate(), the i-th ParameterIndexer specifies the index for the first component sample to use from the corresponding DataPack and how many values in the batch to use for the operation.

Typically, for a latency benchmark, the indexer for each input component will be value_index = 0 and batch_size = 1, indicating the first element of the dataset for that component.

For an offline benchmark, unless otherwise specified, each input component will be value_index = 0 and batch_size = all_samples_for_component, indicating the complete dataset.

For details on the ordering of results based on the operation parameter sample index see Ordering of Results Based on Input Batch Sizes .

See also

DataPack, operate(), Ordering of Results Based on Input Batch Sizes

Definition at line 649 of file types.h.

Class Members
uint64_t	batch_size	Number of values to use, starting from index.
uint64_t	value_index	Index of parameter value inside the data pack.

hebench::APIBridge::WorkloadParam.__unnamed5__

union hebench::APIBridge::WorkloadParam.__unnamed5__

Definition at line 329 of file types.h.

Class Members
double	f_param	64 bits double precision floating point argument value for workload parameter. Only one of i_param, u_param, f_param must be set for the workload parameter, based on data_type.
int64_t	i_param	64 bits signed integer argument value for workload parameter. Only one of i_param, u_param, f_param must be set for the workload parameter, based on data_type.
uint64_t	u_param	64 bits unsigned integer argument value for workload parameter. Only one of i_param, u_param, f_param must be set for the workload parameter, based on data_type.

hebench::APIBridge::CategoryParams.__unnamed7__

union hebench::APIBridge::CategoryParams.__unnamed7__

Definition at line 448 of file types.h.

Class Members
__unnamed7__	latency	Specifies the parameters for the latency category.
__unnamed7__	offline	Specifies the parameters for the offline category.
uint64_t	reserved[HEBENCH_MAX_CATEGORY_PARAMS]	Generic placeholder for categories. This field is used to allocate memory for the whole union. It is advised to access the parameters through the respective category field instead.

hebench::APIBridge::CategoryParams.__unnamed7__.latency

struct hebench::APIBridge::CategoryParams.__unnamed7__.latency

Definition at line 457 of file types.h.

Class Members
uint64_t	warmup_iterations_count	Specifies the number of warmup iterations to perform for latency test. Latency benchmark will run the operation this many iterations with the same set of inputs before doing any actual measurement. The operation results and measured times for the operation during warmup are not counted under the actual operation timing, although may be reported under warmup.

hebench::APIBridge::CategoryParams.__unnamed7__.offline

struct hebench::APIBridge::CategoryParams.__unnamed7__.offline

Definition at line 469 of file types.h.

Class Members
uint64_t	data_count[HEBENCH_MAX_OP_PARAMS]	Specifies the number of data samples for each parameter for Category::Offline. This specifies the the number of samples submitted by Test Harness for each parameter for the offline operation. A value larger than available input samples will see the whole dataset of inputs be used for that parameter in a single operation. Note that in case that there is not enough data samples for a parameter, the number of samples will be lower than requested. A value of 0 indicates to Test Harness that number of samples for the corresponding operation parameter accepts any value. In this case, sample size is user-specified at run-time via configuration files. If value specified by Test user is also 0, the sample size for the operation parameter is defined in the workload specification. Regardless of the sample sizes (specified or default), the actual number of samples may be limited by the dataset for the workload, if any.

Typedef Documentation

ErrorCode

typedef std::int32_t hebench::APIBridge::ErrorCode

Return value for API bridge functions.

A value of HEBENCH_ECODE_SUCCESS means success.

An error value of ECODE_CRITICAL_ERROR will prompt the Test Harness to terminate. This usually indicates an unrecoverable error that indicates that the backend engine encounter an unstable state and it affects the backend engine capacity to recover, cleanup and/or create new benchmarks. On this error, Test Harness will stop benchmarking and attempt to clean up any open handles before exiting. Any further errors are ignored.

Backend should custom-define and return appropriate error codes for all required operations through the API bridge. Custom error codes values must be different than those pre-defined in the API bridge.

See also

getErrorDescription()

Definition at line 34 of file types.h.

Handle

typedef _FlexibleData hebench::APIBridge::Handle

Encapsulates an opaque handle to backend data.

All fields of this struct are managed by the backend. Test harness will not modify any of them, unless otherwise specified. It is up to the backend to set these fields to match its needs.

Definition at line 595 of file types.h.

NativeDataBuffer

typedef _FlexibleData hebench::APIBridge::NativeDataBuffer

Represents a native data buffer maintained by the Test Harness.

This structure is created and maintained by the Test Harness and is passed to the backend through the API bridge.

The buffer pointed to by the p field is formatted to contain data in the format expected for the respective parameter for which the buffer is used, and thus, it is safe to cast to the propper data type pointer.

Note that the size field contains the size in bytes of the buffer, not the number of items of the corresponding data type.

The tag field should not be modified outside of the Test Harness.

Definition at line 588 of file types.h.

Scheme

typedef std::int32_t hebench::APIBridge::Scheme

Open-ended homomorphic encryption scheme ID.

Name and details of the scheme can be obtained using helper functions from the API bridge.

Definition at line 406 of file types.h.

Security

typedef std::int32_t hebench::APIBridge::Security

Open-ended homomorphic encryption scheme security ID.

Name and details of the scheme security can be obtained using helper functions from the API bridge.

Definition at line 412 of file types.h.

Enumeration Type Documentation

Category

enum hebench::APIBridge::Category

Defines all possible categories for each workload.

Note that some workloads may not support certain categories.

Enumerator
Latency	Test Harness sends the same single data sample repeatedly to backend.
Offline	Test Harness loads the whole dataset to backend before requesting all the results in a single operation. For multiple samples on each input, the results are ordered in a row-major fashion. For more information, see Ordering of Results Based on Input Batch Sizes .

Definition at line 390 of file types.h.

DataType

enum hebench::APIBridge::DataType

Defines data types for a workload.

A workload may not support all data types.

Enumerator
Int32	32 bits signed integers.
Int64	64 bits signed integers.
Float32	32 bits IEEE 754 standard floating point real numbers.
Float64	64 bits IEEE 754 standard floating point real numbers.

Definition at line 378 of file types.h.

Workload

enum hebench::APIBridge::Workload

Defines all possible workloads.

See extended documentation for detailed definition, description, data layout, parameters, results, etc.

Enumerator
MatrixMultiply	Matrix Multiplication workload. Workload Params: 0: uint64 - number of rows in matrix M0: rows0. 1: uint64 - number of columns in matrix M0: cols0. 2: uint64 - number of columns in matrix M1: cols1. Operation Params: M0, M1 0: M0 - matrix with rows0 rows and cols0 columns. 1: M1 - matrix with cols0 rows and cols1 columns. Result: 0: M - matrix with rows0 rows and cols1 columns. M = M0 x M1 For details see Matrix Multiplication Workload .
EltwiseAdd	Vector Element-wise Addition workload. Workload Params: 0: uint64 - number of elements in vector: n. Operation Params: A, B 0: A - vector with n contiguous elements in memory. 1: B - vector with n contiguous elements in memory. Result: 0: C - vector with n contiguous elements in memory. C[i] = A[i] + B[i] For details see Vector Element-wise Addition Workload .
EltwiseMultiply	Vector Element-wise Multiplication workload. Workload Params: 0: uint64 - number of elements in vector: n. Operation Params: A, B 0: A - vector with n contiguous elements in memory. 1: B - vector with n contiguous elements in memory. Result: 0: C - vector with n contiguous elements in memory. C[i] = A[i] * B[i] For details see Vector Element-wise Multiplication Workload .
DotProduct	Vector Dot Product workload. Workload Params: 0: uint64 - number of elements in vector: n. Operation Params: A, B 0: A - vector with n contiguous elements in memory. 1: B - vector with n contiguous elements in memory. Result: 0: c - scalar. c = A[0] * B[0] + A[1] * B[1] + ... + A[n - 1] * B[n - 1] For details see Vector Dot Product Workload .
LogisticRegression	Logistic Regression Inference workload. Workload Params: 0: uint64 - n: number of features in feature vector. Operation Params: W, b, X 0: W - feature vector. 1: b - scalar bias. 2: X - input vector with n elements. Result: 0: P(X) - scalar. P(X) = sigmoid(W . X + b) For details see Logistic Regression Inference Workload .
LogisticRegression_PolyD3	Logistic Regression Inference workload with polynomial approximation. Workload Params: 0: uint64 - n: number of features in feature vector. Operation Params: W, b, X 0: W - feature vector. 1: b - scalar bias. 2: X - input vector with n elements. Result: 0: P(X) - scalar. P(X) = sigmoid_pd3(W . X + b) where sigmoid_pd3 is the sigmoid function computed using polynomial approximation of degree 3 around x = 0. For details see Logistic Regression Inference Workload .
LogisticRegression_PolyD5	Logistic Regression Inference workload with polynomial approximation. Workload Params: 0: uint64 - n: number of features in feature vector. Operation Params: W, b, X 0: W - feature vector. 1: b - scalar bias. 2: X - input vector with n elements. Result: 0: P(X) - scalar. P(X) = sigmoid_pd5(W . X + b) where sigmoid_pd5 is the sigmoid function computed using polynomial approximation of degree 5 around x = 0. For details see Logistic Regression Inference Workload .
LogisticRegression_PolyD7	Logistic Regression Inference workload with polynomial approximation. Workload Params: 0: uint64 - n: number of features in feature vector. Operation Params: W, b, X 0: W - feature vector. 1: b - scalar bias. 2: X - input vector with n elements. Result: 0: P(X) - scalar. P(X) = sigmoid_pd7(W . X + b) where sigmoid_pd7 is the sigmoid function computed using polynomial approximation of degree 7 around x = 0. For details see Logistic Regression Inference Workload .
SimpleSetIntersection	Simple Set Intersection workload. Workload Params: 0: uint64 - Size of dataset X (number of items in set): n. 1: uint64 - Size of dataset Y (number of items in set): m. 2: uint64 - Number of elements in an item of a dataset. Must be greater than 0: k. Operation Params: X, Y 0: X - set containing n items. 1: Y - set containing m items. Result: 0: Z - set containing, at most, min(n, m) items. `Z` = {`z_0`, ..., `z_n`, where `z_i` in `X` and `z_i` in `Y`} For details see Simple Set Intersection Workload .
Generic	Generic workload. Workload Params: 0: uint64 - Number of inputs to the operation: n. 1: uint64 - Number of outputs for the operation: m. 2: uint64 - Number of components in first input vector: length_InputParam0. ... n + 1: uint64 - Number of components in last input vector: length_InputParamn-1. n + 2: uint64 - Number of components in first output vector: length_ResultComponent0. ... m + n + 1: uint64 - Number of components in last output vector: length_ResultComponentm-1. Operation Params: InputParam[i] for 0 <= i < n i: InputParam[i] - vector with length_InputParami contiguous elements in memory. Result: ResultComponent[i] for 0 <= i < m i: ResultComponent[i] - vector with length_ResultComponenti contiguous elements in memory. (ResultComponent[0], ResultComponent[1], ..., ResultComponent[m - 1]) = op(InputParam[0], InputParam[1], ..., InputParam[n - 1]) where op is a bijective function defined by a user-provided mapping of inputs into outputs. For details see Generic Workload .

Definition at line 82 of file types.h.

Function Documentation

createBenchmark()

ErrorCode hebench::APIBridge::createBenchmark	(	Handle	h_engine,
		Handle	h_bench_desc,
		const WorkloadParams *	p_params,
		Handle *	h_benchmark
	)

Instantiates a benchmark on the backend.

Parameters

[in]	h_engine	Handle to the backend engine to perform the benchmark.
[in]	h_bench_desc	Handle to the description of the benchmark to instantiate.
[in]	p_params	A set of parameters for the benchmark to be instantiated. It is ignored if workload does not support parameters.
[out]	h_benchmark	Points to a handle to be filled with the initialized benchmark.

Returns

Error code HEBENCH_ECODE_SUCCESS on success, or any other error code if an error occurred during operation, including, unsupported workload parameters.

The returned benchmark handle will be used to reference this benchmark while executing the benchmark operations by the Test Harness. Destroying or modifying h_bench_desc or p_params must not affect the benchmark instantiation.

Backend can assume that only a single benchmark handle will be initialized at all times for the specified backend engine.

Parameter h_bench_desc is a handle to a description previously registered by the backend during the call to subscribeBenchmarks() of the benchmark to be performed.

If workload for the benchmark supports parameters p_params cannot be null; otherwise, it is ignored and can be set to null.

See also

subscribeBenchmarks()

decode()

ErrorCode hebench::APIBridge::decode	(	Handle	h_benchmark,
		Handle	h_plaintext,
		DataPackCollection *	p_native
	)

Decodes plaintext data into the appropriate raw, native format.

Parameters

[in]	h_benchmark	Handle to the initialized benchmark to perform.
[in]	h_plaintext	Handle to encoded data.
[out]	p_native	Pre-allocated buffers to contain the decoded output.

Returns

Error code.

The data buffers in p_native will be passed into this function pre-allocated. The param_position field will be already set to the required component for each data pack. The buffer sizes are determined by the Test harness based on the expected shape for each component of the data pack (representing either operation parameter or result of the operation performed). These shapes are operation-dependent and are documented for each operation. It is the responsibility of the backend to keep track of these shapes (through the opaque handle) so that they can be retrieved appropriately.

Backend must decode the data from h_plaintext handle and fill out as much as it can into each data buffer. If some data buffer's size if smaller than needed for that data entry, the rest of the encoded data entry must be discarded.

This function is the inverse of encode(), so, decoding the result of encode(), must produce the exact same bits in p_native 's data buffers.

See also

encode()

decrypt()

ErrorCode hebench::APIBridge::decrypt	(	Handle	h_benchmark,
		Handle	h_ciphertext,
		Handle *	h_plaintext
	)

Decrypts a cipher text into corresponding plain text.

Parameters

[in]	h_benchmark	Handle to the initialized benchmark to perform.
[in]	h_ciphertext	Handle to the encrypted data to decrypt.
[out]	h_plaintext	Opaque handle to the decrypted data.

Returns

Error code.

The resulting plain text handle represents the encoded plain text. This may be passed to a call to decode() in order to obtain the raw native data.

Destroying or re-utilizing handle h_ciphertext after this call completes shall not affect the resulting plain text.

See also

encrypt()

describeBenchmark()

ErrorCode hebench::APIBridge::describeBenchmark	(	Handle	h_engine,
		Handle	h_bench_desc,
		BenchmarkDescriptor *	p_bench_desc,
		WorkloadParams *	p_default_params,
		std::uint64_t	default_count
	)

Retrieves the concrete description for a benchmark registered by backend.

Parameters

[in]	h_engine	Handle to the backend engine to perform the benchmark.
[in]	h_bench_desc	Handle to the benchmark description to query for information.
[out]	p_bench_desc	Points to the benchmark descriptor to receive the description of the benchmark.
[out]	p_default_params	Array of WorkloadParams that will receive the sets of default parameters supported by this workload. It can be null if caller does not need this information.
[in]	default_count	Number of elements available in the array pointed by p_default_params . This is will be as many default flexible parameter sets as returned by getWorkloadParamsDetails(), or ignored if p_default_params is null.

Returns

Error code.

The array p_default_params is null if caller does not need the information regarding default arguments for workload, otherwise, it must have enough capacity to hold, at least, as many default sets as specified by default_count . The number of default sets in default_count should be the same as returned by getWorkloadParamsDetails(). If workload does not support parameters, p_default_params is ignored.

See also

getWorkloadParamsDetails()

destroyHandle()

ErrorCode hebench::APIBridge::destroyHandle

(

Handle

h

)

Releases resources held by the specified handle.

Parameters

[in]

h

Handle to destroy.

Returns

Error code.

The specified handle is no longer valid for any calls requiring a handle after this function is called. This function releases resources held by the handle in the backend and destroys the handle.

Test Harness will call this method to clean up a handle it no longer needs. Test Harness will take care to destroy handles in reverse order in which it obtained them.

encode()

ErrorCode hebench::APIBridge::encode	(	Handle	h_benchmark,
		const DataPackCollection *	p_parameters,
		Handle *	h_plaintext
	)

Given a pack of parameters in raw, native data format, encodes them into plain text suitable for backend encryption or operation.

Parameters

[in]	h_benchmark	Handle to the initialized benchmark to perform.
[in]	p_parameters	Points to a DataPackCollection instance containing the information for the pack of parameters to encode.
[out]	h_plaintext	Opaque handle representing the resulting encoded data by the backend.

Returns

Error code.

The size and shape of each parameter are determined by the Test Harness based on expected input shapes of the operation performed. These shapes are operation-dependent and are documented for each operation. It is the responsibility of the backend to keep track of these shapes (through the opaque handle) so that they can be retrieved appropriately when needed.

Unless otherwise specified by a workload or category, in general, Test Harness will encode all parameters that ought to be encrypted first in a single call to encode, and all the plain text in another call.

Destroying and cleaning up p_parameters after this call completes shall not affect the resulting encoding.

This function is the inverse of decode(), and thus, decoding the resulting h_plaintext should result in the exact same raw, native data passed during encode.

See also

decode()

encrypt()

ErrorCode hebench::APIBridge::encrypt	(	Handle	h_benchmark,
		Handle	h_plaintext,
		Handle *	h_ciphertext
	)

Encrypts a plain text into a cipher text.

Parameters

[in]	h_benchmark	Handle to the initialized benchmark to perform.
[in]	h_plaintext	Handle to the plain text to encrypt.
[out]	h_ciphertext	Opaque handle to the encrypted data.

Returns

Error code.

The resulting cipher text handle is commonly used in a call to the load() function to load the cipher text into the backend for execution of the operation to benchmark.

Destroying or re-utilizing handle h_plaintext after this call completes shall not affect the resulting cipher text.

See also

decrypt()

getBenchmarkDescriptionEx()

std::uint64_t hebench::APIBridge::getBenchmarkDescriptionEx	(	Handle	h_engine,
		Handle	h_bench_desc,
		const hebench::APIBridge::WorkloadParams *	p_w_params,
		char *	p_description,
		std::uint64_t	size
	)

Retrieves backend specific text description for a benchmark descriptor.

Parameters

[in]	h_engine	Handle to the backend engine.
[in]	h_bench_desc	Handle to benchmark description for which to obtain the text description.
[in]	p_w_params	Workload parameters for this benchmark. May be null if no workload parameters are needed or their description is not required.
[out]	p_description	Buffer to store C-string description of the benchmark.
[in]	size	Size in bytes of p_description buffer.

Returns

Number of bytes needed in buffer to store the complete description, including the C-string null terminator, or zero if error.

If p_description is null, this function performs no copy. Use this behavior to obtain the size, in bytes, required to store the complete description, including the C-string null terminator.

If p_description is not null, this function copies as many bytes as specified in size from the benchmark text description into the buffer pointed to by p_description, including the C-string null terminator.

This description is used along with other text descriptions about the workload, data types, encryption schemes, etc, to produce the benchmark report. It is intended to supply any extra description that is specific to the general-purpose fields in BenchmarkDescriptor structure (such as other) that is not specified by getSchemeName(), getSchemeSecurityName() and other similar functions.

getErrorDescription()

std::uint64_t hebench::APIBridge::getErrorDescription	(	Handle	h_engine,
		ErrorCode	code,
		char *	p_description,
		std::uint64_t	size
	)

Retrieves the general error description of an error code.

Parameters

[in]	h_engine	Handle to the backend engine. Can be a null handle (especially if there is no engine when the error occurred).
[in]	code	Error code to describe.
[out]	p_description	Buffer to store C-string description of the error.
[in]	size	Size in bytes of p_description buffer.

Returns

Number of bytes needed in buffer to store the complete description, including the C-string null terminator, or zero if error.

If p_description is null, this function performs no copy. Use this behavior to obtain the size, in bytes, required to store the full description of the error, including the C-string null terminator.

If p_description is not null, this function copies as many bytes as specified in size from the error description into the buffer pointed to by p_description, including the C-string null terminator.

getLastErrorDescription()

std::uint64_t hebench::APIBridge::getLastErrorDescription	(	Handle	h_engine,
		char *	p_description,
		std::uint64_t	size
	)

Retrieves the detailed description of the last error that occurred during an operation on the engine.

Parameters

[in]	h_engine	Handle to the backend engine.
[out]	p_description	Buffer to store C-string description of the error.
[in]	size	Size in bytes of p_description buffer.

Returns

Number of bytes needed in buffer to store the complete description, including the C-string null terminator, or zero if error.

If p_description is null, this function performs no copy. Use this behavior to obtain the size, in bytes, required to store the full detailed description of the error, including the C-string null terminator.

If p_description is not null, this function copies as many bytes as specified in size from the error description into the buffer pointed to by p_description, including the C-string null terminator.

Detailed description is backend implementation defined. If no detailed implementation, returning the general error description is suggested. If no error has occurred, some "no error" message is suggested.

getSchemeName()

std::uint64_t hebench::APIBridge::getSchemeName	(	Handle	h_engine,
		Scheme	s,
		char *	p_name,
		std::uint64_t	size
	)

Retrieves the name of a specified scheme ID from the backend.

Parameters

[in]	h_engine	Handle to the backend engine.
[in]	s	Scheme ID.
[out]	p_name	Buffer to store C-string name of the scheme.
[in]	size	Size in bytes of p_name buffer.

Returns

Number of bytes needed in buffer to store the complete name, including the C-string null terminator, or zero if error.

If p_name is null, this function performs no copy. Use this behavior to obtain the size, in bytes, required to store the complete name of the scheme, including the C-string null terminator.

If p_name is not null, this function copies as many bytes as specified in size from the scheme's full name into the buffer pointed to by p_name, including the C-string null terminator.

This function is used for report purposes.

getSchemeSecurityName()

std::uint64_t hebench::APIBridge::getSchemeSecurityName	(	Handle	h_engine,
		Scheme	s,
		Security	sec,
		char *	p_name,
		std::uint64_t	size
	)

Retrieves the name of the specified security for the scheme ID from the backend.

Parameters

[in]	h_engine	Handle to the backend engine.
[in]	s	Scheme ID.
[in]	sec	Security ID for scheme.
[out]	p_name	Buffer to store C-string name of the security.
[in]	size	Size in bytes of p_name buffer.

Returns

Number of bytes needed in buffer to store the complete name, including the C-string null terminator, or zero if error.

If p_name is null, this function performs no copy. Use this behavior to obtain the size, in bytes, required to store the complete name of the security scheme, including the C-string null terminator.

If p_name is not null, this function copies as many bytes as specified in size from the security scheme's full name into the buffer pointed to by p_name, including the C-string null terminator.

This function is used for report purposes.

getWorkloadParamsDetails()

ErrorCode hebench::APIBridge::getWorkloadParamsDetails	(	Handle	h_engine,
		Handle	h_bench_desc,
		std::uint64_t *	p_param_count,
		std::uint64_t *	p_default_count
	)

Retrieves details about the flexible parameters supported by this workload.

Parameters

[in]	h_engine	Handle to the backend engine to perform the benchmark.
[in]	h_bench_desc	Handle to the benchmark description to query for information.
[out]	p_param_count	Points to variable to receive the number of workload parameters required by the backend.
[out]	p_default_count	Points to variable to receive the number of sets of default workload parameters supported by the backend.

Returns

Error code.

Value returned in p_param_count is 0 if and only if p_default_count is also 0.

See also

describeBenchmark()

initBenchmark()

ErrorCode hebench::APIBridge::initBenchmark	(	Handle	h_benchmark,
		const BenchmarkDescriptor *	p_concrete_desc
	)

Allows the benchmark to perform initialization steps based on the finalized, concrete description.

Parameters

[in]	h_benchmark	Handle to a benchmark created from createBenchmark().
[in]	p_concrete_desc	Benchmark descriptor filled out with all the concrete information about the requests for the benchmark to run.

Returns

Error code HEBENCH_ECODE_SUCCESS on success, or any other error code if an error occurred during operation, including, unsupported description values.

A subscribed benchmark description may include fields that accept user-specified values, such as sample sizes in an offline test, for example. Test Harness calls this function to indicate the concrete user-specified values for all fileds in the benchmark description.

During benchmark creation, a backend only has access to the workload parameters and its own version of the description. A backend may use this function to perform extra initialization steps using the actual values for the user-specified fields in the benchmark description, if any.

See also

createBenchmark()

initEngine()

ErrorCode hebench::APIBridge::initEngine	(	Handle *	h_engine,
		const int8_t *	p_buffer,
		uint64_t	size
	)

Initializes the backend engine.

Parameters

[out]	h_engine	Points to a handle to fill with the initialized backend engine handle descriptor. Must not be null.
[in]	p_buffer	Input buffer of bytes with extra information for engine initialization. May be null.
[in]	size	Number of bytes pointed by p_buffer .

Returns

Error code.

Use this method to initialize backend engine, such as drivers, hardware, etc. required to perform operations.

Data stored in p_buffer is specified by frontend user through benchmark configuration files. Test Harness will forward this information to backend using this buffer.

load()

ErrorCode hebench::APIBridge::load	(	Handle	h_benchmark,
		const Handle *	h_local_packed_params,
		std::uint64_t	local_count,
		Handle *	h_remote
	)

Loads the specified data from the local host into the remote backend to use as parameter during a call to operate().

Parameters

[in]	h_benchmark	Handle to the initialized benchmark to perform.
[in]	h_local_packed_params	Collection of handles to the data of packed parameters to load into backend remote.
[in]	local_count	Number of handles in array h_local_packed_params.
[out]	h_remote	Handle representing the loaded data into the backend.

Returns

Error code.

This function exists to allow backends to transfer the data from the host into the actual device (accelerator, remote server, etc.) that will perform the operations.

Each handle in h_local_packed_params represents an encoded/encrypted DataPackCollection object in the local host.

In general, plain text parameters will be packed together under a single DataPackCollection during encoding, and encrypted parameters will be packed together under another single DataPackCollection. Handles for each DataPackCollection are generated during encode() and encrypt() calls, and thus, h_local_packed_params collection allows for all of these handles to be loaded at the same time. If not specified in the workload description, the order for DataPackCollection handles is encrypted first, followed by plain text when both types of parameters, encrypted and plain text, are present; otherwise, Test harness will use a single DataPackCollection for all parameters as such:

All parameters encrypted, or all parameters plain text:

h_local_packed_params[0] <- handle to DataPackCollection containing all parameters.
local_count = 1

Mix of encrypted and plain text parameters:

h_local_packed_params[0] <- handle to DataPackCollection containing encrypted parameters.
h_local_packed_params[1] <- handle to DataPackCollection containing plain text parameters.
local_count = 2

Destroying or re-utilizing collection and/or handles in h_local_packed_params after this call completes shall not affect the resulting h_remote handle.

See also

operate(), store()

operate()

ErrorCode hebench::APIBridge::operate	(	Handle	h_benchmark,
		Handle	h_remote_packed_params,
		const ParameterIndexer *	p_param_indexers,
		uint64_t	indexers_count,
		Handle *	h_remote_output
	)

Performs the workload operation of the benchmark.

Parameters

[in]	h_benchmark	Handle to the initialized benchmark to perform.
[in]	h_remote_packed_params	Handle representing the parameters for the operation previously loaded into the backend remote by a call to load() to use for the operation. The data represented by this handle must not be changed since it may be reused in several calls to operate().
[in]	p_param_indexers	Indexers for the parameters of the operation.
[in]	indexers_count	Number of indexrs in p_param_indexers .
[out]	h_remote_output	Handle representing the result of the operation stored in the backend remote.

Returns

Error code.

This is the function that Test Harness will benchmark. Backend MUST ensure that the complete operation runs on all specified inputs and that the corresponding outputs are generated before this function returns.

Destroying or re-using handle or values in h_remote_packed_params and p_param_indexers after this call completes shall not affect the resulting h_remote_output.

The actual collection of inputs to the operation that backend must use is loaded into the backend remote using a single call of the load() function. Parameter p_param_indexers contains the indexers for the parameter packs, and thus the number of elements in array p_param_indexers is, at least, the same as the number of parameters required for the operation. indexers_count specifies the actual number of elements in p_param_indexers . The actual configuration for the call to operate() issued by Test Harness is defined for each workload.

Once decoded, the workload results generated by this function must be indexed starting at 0, regardless of starting input sample index specified by p_param_indexers. Any offset in the input sample index will be taken into account by Test Harness when validating the results. For more information, see Ordering of Results Based on Input Batch Sizes .

The following example shows a typical flow followed by Test Harness when performing a benchmark. Here, Test Harness is benchmarking offline an operation of the form

result = op(A, B)

where parameter A has a single sample, and B has 10 samples. There is a result for each pair of values. Both parameters are encrypted. The pseudo-code submits all 10 values for B. Benchmarking statements are omitted for clarity.

constexpr int A_count = 1;
constexpr int B_count = 10;
NativeDataBuffer p_raw_inputs_A[A_count];
NativeDataBuffer p_raw_inputs_B[B_count];
// fill up the inputs
//...
//
 
Handle h_engine, h_benchmark;
// initialize the engine and benchmark
//...
//
 
// prepare the parameters for encoding
 
constexpr int op_params_count = 2; // operation has 2 parameters
 
DataPack param_packs[op_params_count];
param_packs[0].p_buffers = p_raw_inputs_A;
param_packs[0].buffer_count = A_count;
param_packs[0].param_position = 0; // A is the first parameter, so, position 0
param_packs[1].p_buffers = p_raw_inputs_B;
param_packs[1].buffer_count = B_count;
param_packs[1].param_position = 1; // B is the second parameter, so, position 1
 
DataPackCollection packed_parameters;
packed_parameters.p_data_packs = param_packs;
packed_parameters.pack_count = op_params_count;
 
// encode the raw parameters data
Handle h_encoded_inputs;
encode(h_benchmark, &packed_parameters, &h_encoded_inputs);
 
// encrypt the encoded data
Handle h_cipher_inputs;
encrypt(h_benchmark, h_encoded_inputs, &h_cipher_inputs);
 
// load encrypted data into backend's remote to use as input to the operation
Handle h_remote_inputs;
load(h_benchmark,
     &h_cipher_inputs, 1, // only 1 DataPackCollection
     &h_remote_inputs);
 
// clean up data we no longer need (in reverse order of creation)
destroyHandle(h_cipher_inputs);
destroyHandle(h_encoded_inputs);
 
// prepare parameter indexers for operation
 
ParameterIndexer params[op_params_count]; // one indexer per parameter
params[0].batch_size = 1;  // for parameter A we will be using one value
params[0].value_index = 0; // starting with the first value
 
params[1].batch_size = 10; // for parameter B we will be using 10 values
params[1].value_index = 0; // starting with the first.
 
// operate
Handle h_remote_result;
while (!elapsed(min_test_time))
{
    operate(h_benchmark,
            h_remote_inputs, params,
            &h_remote_result);
}
 
// at this point, the operation must be completed in remote backend
 
// clean up data we no longer need
destroyHandle(h_remote_inputs);
 
// postprocess output
Handle h_cipher_output;
 
// retrieve data from backend's remote and store in host
store(h_benchmark, h_remote_result,
      &h_cipher_output, 1 // Only 1 local DataPackCollection for result expected for this operation.
     );
 
// clean up data we no longer need
destroyHandle(h_remote_result);
 
// decrypt results
Handle h_plain_result;
decrypt(h_benchmark, h_cipher_output, &h_plain_result);
 
// clean up data we no longer need
destroyHandle(h_cipher_output);
 
// allocate space for decoded results
 
NativeDataBuffer p_raw_results[A_count * B_count]; // allocate space for all 10 outputs
// allocate memory for raw outputs
// ...
//
 
DataPack results_pack;
results_pack.p_buffers = p_raw_results;
results_pack.buffer_count = A_count * B_count;
results_pack.param_position = 0; // we are retrieving results in the first position into this data pack
 
DataPackCollection packed_results;
packed_results.p_data_packs = &results_pack;
packed_results.pack_count = 1; // result shape is [1, 10]
 
// allocate memory for raw outputs
// ...
//
 
// decode the results
decode(Handle h_benchmark, h_plain_result, &packed_results);
 
// clean up data we no longer need
destroyHandle(h_plain_result);
 
// benchmark completed
destroyHandle(h_benchmark);
 
// validate output
// ...
//
 
// clean up packed_results
// ...
//
 
// when done with all the benchmarks, clean up backend
destroyHandle(h_engine);

See also

load(), store(), Ordering of Results Based on Input Batch Sizes

store()

ErrorCode hebench::APIBridge::store	(	Handle	h_benchmark,
		Handle	h_remote,
		Handle *	h_local_packed_params,
		std::uint64_t	local_count
	)

Retrieves the specified data from the backend.

Parameters

[in]	h_benchmark	Handle to the initialized benchmark to perform.
[in]	h_remote	Handle to the data to retrieve from backend remote and store into h_local.
[out]	h_local_packed_params	Collection of opaque handles to represent the data stored in the local host retrieved from the backend.
[in]	local_count	Number of available handles to receive data.

Returns

Error code.

This function is the counterpart to load().

Function operate() receives an opaque handle that represents the result of the operation in the backend. Because this result may live in a device other than the host (accelerator, remote server, etc.), a call to store() must be issued to retrieve the data from the backend remote and store it in the host.

Each handle in h_local_packed_params will represent an encoded/encrypted DataPackCollection object after stored in the local host. If there are not enough handles in h_local_packed_params to store all the data from h_remote, extra remote data will be ignored. Any extra handles will be padded with zeroes.

Destroying or re-using handle h_remote after this call completes shall not affect the resulting h_local_packed_params handle.

See also

load(), operate()

subscribeBenchmarks()

ErrorCode hebench::APIBridge::subscribeBenchmarks	(	Handle	h_engine,
		Handle *	p_h_bench_descs,
		std::uint64_t	count
	)

Retrieves handles to the benchmark descriptions for which the backend is registering to perform.

Parameters

[in]	h_engine	Backend engine handle.
[out]	p_h_bench_descs	Pointer to array where to store the handles to the benchmark descriptions. Array has capacity for count handles. Cannot be null.
[in]	count	Number of handles available in array pointed by p_h_bench_descs . This is will be as many handles as returned by subscribeBenchmarksCount().

Returns

Error code.

The handles in p_h_bench_descs are pointed to the internal representation of the description of each benchmark that the backend is registering to perform. These benchmark description handles shall be used to query information about the registered benchmarks without having to instantiate the represented benchmark proper, and to instantiate the benchmark once it is time to execute it by calling the appropriate API bridge functions.

See also

subscribeBenchmarksCount()

subscribeBenchmarksCount()

ErrorCode hebench::APIBridge::subscribeBenchmarksCount	(	Handle	h_engine,
		std::uint64_t *	p_count
	)

Retrieves the number of benchmarks for which the backend is registering to perform.

Parameters

[in]	h_engine	Backend engine handle.
[out]	p_count	Points to variable to receive the result. Cannot be null.

Returns

Error code.

The number of benchmarks returned in p_count for which backend is subscribing must be greater than 0.