Sync Your Data From Edge-to-Cloud With Actian Zen EasySync

Welcome back to the world of Actian Zen, a versatile and powerful edge data management solution designed to help you build low-latency embedded apps. This is Part 3 of the quickstart blog series that focuses on helping embedded app developers get started with Actian Zen.

Establishing consistency and consolidating data across different devices and servers are essential for most edge-to-cloud solutions. Syncing data is necessary for almost every mobile, edge, or IoT application, and developers are familiar with the basic concepts and challenges. That’s why many experienced developers value efficient solutions. The Actian Zen EasySync tool is a new utility specifically designed for this purpose.

This blog will guide you through the steps for setting up and running EasySync.

What is EasySync?

Zen EasySync is a versatile data synchronization tool that automates the synchronization of newly created or updated records from one Zen database server to another. This tool transfers data across multiple servers, whether you’re working on the edge or within a centralized network. Key features of EasySync include:

• Flexible Syncing Schedule: Sync data can be scheduled to poll for changes on a defined interval or can be used as a batch transfer tool, depending on your needs.
• Logging: Monitor general activity, detect errors, and troubleshoot unexpected results with logging capabilities.

Prerequisites

Before using EasySync, ensure the following in your Zen installation:

• System Data: The files must have system data v2 enabled, with file format version 13 or version 16.
• ZEN 16.0  installed.
• Unique Key: Both source and destination files must have a user-defined unique key.

EasySync Usage Scenarios

EasySync supports various data synchronization scenarios, making it a flexible tool for different use cases. Here are some common usage scenarios depicted in the diagram below:

1. Push to Remote: Synchronize data from a local database to a remote database.
2. Pull from Remote: Synchronize data from a remote database to a local database.
3. Pull and Push to Remotes: Synchronize data between multiple remote databases.
4. Aggregate Data from Edge: Collect data from multiple edge databases and synchronize it to a central database.
5. Disseminate Data to Edge: Distribute data from a central database to multiple edge databases.

Getting Started With EasySync

To demonstrate how to use EasySync, we will create a Python application that simulates sensor data and synchronizes it using EasySync. This application will create a sensor table on your edge device and remote server, insert random sensor data, and sync the data with a remote database. The remote database can contain various sets of data from several edge devices.

Step 1: Create the Configuration File

First, we need to create a JSON configuration file (config.json). This file will define the synchronization settings and the files to be synchronized, where files are stored in a source (demodata) and destination (demodata) folders.

Here is an example of what the configuration file might look like:

{
  "version": 1,
  "settings": {
    "polling_interval_sec": 10,
    "log_file": " C:/ProgramData/Actian/Zen/logs/datasync.log",
    "record_err_log": " C:/ProgramData/Actian/Zen/logs/recorderrors.log",
    "resume_on_error": true
  },
  "files": [
    {
      "id": 1,
      "source_file": "btrv://localhost/demodata?dbfile= sensors.mkd",
      "source_username": "",
      "source_password": "",
      "destination_file": "btrv://<Destination Server>/demodata?dbfile= sensors.mkd",
      "destination_username": "",
      "destination_password": "",
      "unique_key": 0
    },
    {
      "id": 2,
      "source_file": "btrv://localhost/demodata?dbfile=bookstore.mkd",
      "destination_file": "btrv://<Destination Server>/demodata?dbfile=bookstore.mkd",
      "create_destination": true,
      "unique_key": 1
    }
  ]
}

Step 2: Write the Python Script

Next, we create a Python script that simulates sensor data, creates the necessary database table, and inserts records into the database. 

Save the following Python code in a file named run_easysync.py. Run the script to create the sensors table on your local edge device and server, and to insert data on your edge device.

import pyodbc
import random
import time
from time import sleep
random.seed()
def CreateSensorTable(server, database):
    try:
db_connection_string = f"Driver={{Pervasive ODBC Interface}};
ServerName={server};
DBQ={database};"
        conn = pyodbc.connect(db_connection_string, autocommit=True)
        cursor = conn.cursor()
       # cursor.execute("DROP TABLE IF EXISTS sensors;")
        cursor.execute("""
            CREATE TABLE sensors SYSDATA_KEY_2(
                id IDENTITY,
                ts DATETIME NOT NULL,
                temperature INT NOT NULL,
                pressure FLOAT NOT NULL,
                humidity INT NOT NULL
            );
        """)
        print(f"Table 'sensors' created successfully on {server}")
     except pyodbc.DatabaseError as err:
         print(f"Failed to create table on {server} with error: {err}")
def GetTemperature():
     return random.randint(70, 98)
def GetPressure():
     return round(random.uniform(29.80, 30.20), 3)
def GetHumidity():
     return random.randint(40, 55)
def InsertSensorRecord(server, database):
     temp = GetTemperature()
     press = GetPressure()
     hum = GetHumidity()
     try:
      insert = 'INSERT INTO sensors (id, ts, temperature, pressure, humidity) VALUES (0, NOW(), ?, ?, ?)'
        db_connection_string = f"Driver={{Pervasive ODBC Interface}};ServerName={server};DBQ={database};"
        conn = pyodbc.connect(db_connection_string, autocommit=True)
        cursor = conn.cursor()
        cursor.execute(insert, temp, press, hum)
        print(f"Inserted record [Temperature {temp}, Pressure {press}, Humidity {hum}] on {server}")
    except pyodbc.DatabaseError as err:
        print(f"Failed to insert record on {server} with error: {err}")
# Main
local_server = "localhost"
local_database = "Demodata"
remote_server = "remote-server_name"
remote_database = "demodata"

# Create sensor table on both local and remote servers
CreateSensorTable(local_server, local_database)
CreateSensorTable(remote_server, remote_database)

while True:
    InsertSensorRecord(local_server, local_database)
    sleep(0.5)

Syncing Data from IoT Device to Remote Server

Now, let’s incorporate the data synchronization process using the EasySync tool to ensure the sensor data from the IoT device is replicated to a remote server.

Step 3: Run EasySync

To synchronize the data using EasySync, follow these steps:

1. Ensure the easysync utility is installed and accessible from your command line.
2. Run the Python script to start generating and inserting sensor data.
3. Execute the EasySync command to start the synchronization process.

Open your command line and navigate to the directory containing your configuration file and Python script. Then, run the following command:

easysync -o config.json

This command runs the EasySync utility with the specified configuration file and ensures that the synchronization process begins.

Conclusion

Actian Zen EasySync is a simple but effective tool for automating data synchronization across Zen database servers. By following the steps outlined in this blog, you can easily set up and run EasySync. EasySync provides the flexibility and reliability you need to manage your data on the edge. Remember to ensure your files are in the correct format, have system data v2 enabled, and possess a user-defined unique key for seamless synchronization. With EasySync, you can confidently manage data from IoT devices and synchronize it to remote servers efficiently.

For further details and visual guides, refer to the Actian Academy and the comprehensive documentation. Happy coding!

About Johnson Varughese

Johnson Varughese is the Manager of Support Engineering at Actian, where he supports developers creating applications using ZEN interfaces such as Btrieve, ODBC, JDBC, ADO.NET, Btrieve2, and others. His role involves providing technical guidance, troubleshooting, and ensuring developers can effectively utilize these interfaces to build robust and efficient applications in different programming environments. Johnson's expertise in these technologies helps streamline development processes and enhance the performance and functionality of the applications being developed.