bbc1.core.data_handler module¶

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

class bbc1.core.data_handler.DataHandler(networking=None, config=None, workingdir=None, domain_id=None, loglevel='all', logname=None)[source]¶

Bases: object

DB and storage handler

NOTIFY_INSERTED = b'\x00\x04'¶

REPAIR_TRANSACTION_DATA = b'\x00\x05'¶

REPLICATION_ALL = 0¶

REPLICATION_CROSS_REF = b'\x00\x06'¶

REPLICATION_EXT = 2¶

REPLICATION_P2P = 1¶

REQUEST_REPLICATION_INSERT = b'\x00\x00'¶

REQUEST_SEARCH = b'\x00\x02'¶

RESPONSE_REPLICATION_INSERT = b'\x00\x01'¶

RESPONSE_SEARCH = b'\x00\x03'¶

count_domain_in_cross_ref(outer_domain_id)[source]¶: Count the number of domains in the cross_ref table

count_transactions(asset_group_id=None, asset_id=None, user_id=None, start_from=None, until=None, db_num=0)[source]¶

Count transactions that matches the given conditions

When Multiple conditions are given, they are considered as AND condition.

Parameters:	asset_group_id (bytes) – asset_group_id that target transactions should have asset_id (bytes) – asset_id that target transactions should have user_id (bytes) – user_id that target transactions should have start_from (int) – the starting timestamp to search until (int) – the end timestamp to search db_num (int) – index of DB if multiple DBs are used
Returns:	the number of transactions
Return type:	int

exec_sql(db_num=0, sql=None, args=(), commit=False, fetch_one=False, return_cursor=False)[source]¶

Execute sql sentence

Parameters:	db_num (int) – index of DB if multiple DBs are used sql (str) – SQL string args (list) – Args for the SQL commit (bool) – If True, commit is performed fetch_one (bool) – If True, fetch just one record return_cursor (bool) – If True (and fetch_one is False), return db_cur (iterator)
Returns:	list of records
Return type:	list

get_asset_info(txobj)[source]¶

Retrieve asset information from transaction object

Parameters:	txobj (BBcTransaction) – transaction object to analyze
Returns:	list of list [asset_group_id, asset_id, user_id, file_size, file_digest]
Return type:	list

get_in_storage(asset_group_id, asset_id)[source]¶

Get the asset file with the asset_id from local storage

Parameters:	asset_group_id (bytes) – asset_group_id of the asset asset_id (bytes) – asset_id of the asset
Returns:	the file content
Return type:	bytes or None

insert_cross_ref(transaction_id, outer_domain_id, txid_having_cross_ref, no_replication=False)[source]¶

Insert cross_ref information into cross_ref_table

Parameters:	transaction_id (bytes) – target transaction_id outer_domain_id (bytes) – domain_id that holds cross_ref about the transaction_id txid_having_cross_ref (bytes) – transaction_id in the outer_domain that includes the cross_ref no_replication (bool) – If False, the replication is sent to other nodes in the domain

insert_transaction(txdata, txobj=None, fmt_type=0, asset_files=None, no_replication=False)[source]¶

Insert transaction data and its asset files

Either txdata or txobj must be given to insert the transaction.

Parameters:	txdata (bytes) – serialized transaction data txobj (BBcTransaction) – transaction object to insert fmt_type (int) – 2-byte value of BBcFormat type asset_files (dict) – asset files in the transaction
Returns:	set of asset_group_ids in the transaction
Return type:	set

process_message(msg)[source]¶

Process received message

Parameters:	msg (dict) – received message

remove(transaction_id, txobj=None, db_num=-1)[source]¶

Delete all data regarding the specified transaction_id

This method requires either transaction_id or txobj.

Parameters:	transaction_id (bytes) – target transaction_id txobj (BBcTransaction) – transaction object to remove db_num (int) – index of DB if multiple DBs are used

restore_transaction_data(db_num, transaction_id, txobj)[source]¶: Remove and insert a transaction

search_domain_having_cross_ref(transaction_id=None)[source]¶

Search domain_id that holds cross_ref about the specified transaction_id

Parameters:	transaction_id (bytes) – target transaction_id
Returns:	records of cross_ref_tables [“id”,”transaction_id”, “outer_domain_id”, “txid_having_cross_ref”]
Return type:	list

search_transaction(transaction_id=None, asset_group_id=None, asset_id=None, user_id=None, start_from=None, until=None, direction=0, count=1, db_num=0)[source]¶

Search transaction data

When Multiple conditions are given, they are considered as AND condition.

Parameters:	transaction_id (bytes) – target transaction_id asset_group_id (bytes) – asset_group_id that target transactions should have asset_id (bytes) – asset_id that target transactions should have user_id (bytes) – user_id that target transactions should have start_from (int) – the starting timestamp to search until (int) – the end timestamp to search direction (int) – 0: descend, 1: ascend count (int) – The maximum number of transactions to retrieve db_num (int) – index of DB if multiple DBs are used
Returns:	mapping from transaction_id to serialized transaction data dict: dictionary of {asset_id: content} for the transaction
Return type:	dict

search_transaction_topology(transaction_id, traverse_to_past=True)[source]¶

Search in topology info

Parameters:	transaction_id (bytes) – base transaction_id traverse_to_past (bool) – True: search backward (to past), False: search forward (to future)
Returns:	list of records of topology table
Return type:	list

store_in_storage(asset_group_id, asset_id, content, do_overwrite=False)[source]¶

Store asset file in local storage

Parameters:	asset_group_id (bytes) – asset_group_id of the asset asset_id (bytes) – asset_id of the asset content (bytes) – the content of the asset file do_overwrite (bool) – If True, file is overwritten
Returns:	True if successful
Return type:	bool

class bbc1.core.data_handler.DataHandlerDomain0(networking=None, config=None, workingdir=None, domain_id=None, loglevel='all', logname=None)[source]¶

Bases: bbc1.core.data_handler.DataHandler

Data handler for domain_global_0

exec_sql(sql, *args)[source]¶

Execute sql sentence

Parameters:	db_num (int) – index of DB if multiple DBs are used sql (str) – SQL string args (list) – Args for the SQL commit (bool) – If True, commit is performed fetch_one (bool) – If True, fetch just one record return_cursor (bool) – If True (and fetch_one is False), return db_cur (iterator)
Returns:	list of records
Return type:	list

get_asset_info(txobj)[source]¶

Retrieve asset information from transaction object

Parameters:	txobj (BBcTransaction) – transaction object to analyze
Returns:	list of list [asset_group_id, asset_id, user_id, file_size, file_digest]
Return type:	list

get_in_storage(asset_group_id, asset_id)[source]¶

Get the asset file with the asset_id from local storage

Parameters:	asset_group_id (bytes) – asset_group_id of the asset asset_id (bytes) – asset_id of the asset
Returns:	the file content
Return type:	bytes or None

insert_transaction(txdata, txobj=None, asset_files=None, no_replication=False)[source]¶

Insert transaction data and its asset files

Either txdata or txobj must be given to insert the transaction.

Parameters:	txdata (bytes) – serialized transaction data txobj (BBcTransaction) – transaction object to insert fmt_type (int) – 2-byte value of BBcFormat type asset_files (dict) – asset files in the transaction
Returns:	set of asset_group_ids in the transaction
Return type:	set

process_message(msg)[source]¶

Process received message

Parameters:	msg (dict) – received message

remove(transaction_id)[source]¶

Delete all data regarding the specified transaction_id

This method requires either transaction_id or txobj.

Parameters:	transaction_id (bytes) – target transaction_id txobj (BBcTransaction) – transaction object to remove db_num (int) – index of DB if multiple DBs are used

search_transaction(transaction_id=None, asset_group_id=None, asset_id=None, user_id=None, count=1)[source]¶

Search transaction data

When Multiple conditions are given, they are considered as AND condition.

Parameters:	transaction_id (bytes) – target transaction_id asset_group_id (bytes) – asset_group_id that target transactions should have asset_id (bytes) – asset_id that target transactions should have user_id (bytes) – user_id that target transactions should have start_from (int) – the starting timestamp to search until (int) – the end timestamp to search direction (int) – 0: descend, 1: ascend count (int) – The maximum number of transactions to retrieve db_num (int) – index of DB if multiple DBs are used
Returns:	mapping from transaction_id to serialized transaction data dict: dictionary of {asset_id: content} for the transaction
Return type:	dict

search_transaction_topology(transaction_id, reverse_link=False)[source]¶

Search in topology info

Parameters:	transaction_id (bytes) – base transaction_id traverse_to_past (bool) – True: search backward (to past), False: search forward (to future)
Returns:	list of records of topology table
Return type:	list

store_in_storage(asset_group_id, asset_id, content)[source]¶

Store asset file in local storage

Parameters:	asset_group_id (bytes) – asset_group_id of the asset asset_id (bytes) – asset_id of the asset content (bytes) – the content of the asset file do_overwrite (bool) – If True, file is overwritten
Returns:	True if successful
Return type:	bool

class bbc1.core.data_handler.DbAdaptor(handler=None, db_name=None, db_num=0, loglevel='all', logname=None)[source]¶

Bases: object

Base class for DB adaptor

check_table_existence(tblname)[source]¶: Check whether the table exists or not

create_table(tbl, tbl_definition, primary_key=0, indices=[])[source]¶: Create a table

create_ver2_column()[source]¶: Create column for version 2 meta table (add timestamp in asset_info_table)

get_version()[source]¶

get_version of the DB

Returns:	version string
Return type:	str

open_db()[source]¶: Open the DB

update_table_def(from_ver)[source]¶: Update table definition

class bbc1.core.data_handler.MysqlAdaptor(handler=None, db_name=None, db_num=None, server_info=None, loglevel='all', logname=None)[source]¶

Bases: bbc1.core.data_handler.DbAdaptor

DB adaptor for MySQL

check_table_existence(tblname)[source]¶: Check whether the table exists or not

create_table(tbl, tbl_definition, primary_key=0, indices=[])[source]¶

Create a table

Parameters:	tbl (str) – table name tbl_definition (list) – schema of the table [[“column_name”, “data type”],[“colmun_name”, “data type”],,] primary_key (int) – index (column) of the primary key of the table indices (list) – list of indices to create index

open_db()[source]¶: Open the DB

class bbc1.core.data_handler.SqliteAdaptor(handler=None, db_name=None, loglevel='all', logname=None)[source]¶

Bases: bbc1.core.data_handler.DbAdaptor

DB adaptor for SQLite3

check_table_existence(tblname)[source]¶: Check whether the table exists or not

create_table(tbl, tbl_definition, primary_key=0, indices=[])[source]¶

Create a table

Parameters:	tbl (str) – table name tbl_definition (list) – schema of the table [[“column_name”, “data type”],[“colmun_name”, “data type”],,] primary_key (int) – index (column) of the primary key of the table indices (list) – list of indices to create index

open_db()[source]¶: Open the DB (create DB file if not exists)