It’s Back to the Future for Flat Files – Part 2

Why Embedded Software Application Developers Loath Databases

Last week I wrote an initial blog on flat files and why embedded software application developers readily adopted them. Simply put, they’re always there – if you’ve got an operating system that’s more than a bare-bones executive kernel it’s there and it’s free. Also, flat files are really simple to use, and, while there may be underlying differences in various file formats, encryption techniques strengths, and other underlying characteristics, from a developer’s standpoint they’re the same at the API level. But, it’s the alternatives to flat files that developers are reluctant to use, first and foremost, databases.

Is Choosing Flat Files or Databases Kind of Like Choosing Between Two Political Candidates You Don’t Like?

We’re in the election season so I couldn’t resist the analogy. In answer to the question: Yes. For most developers, choosing between a flat file and a database is similar to choosing between two candidates you just rather do without. On the one hand, you’ve got Candidate A, “tried and true”, but they don’t measure up to the challenges you expect them to face going forward. On the other hand, you’ve got Candidate B, a choice who makes lots of promises, but you don’t think they can deliver, and you expect the investments to make their initiatives real to be astronomical. If you think I’m crazy enough to apply any real-life politicians to this comparison – sorry, I’m going to play politician myself and pass on that. But you get the general point (and hopefully a good chuckle).

Upon Further Inspection of the Plan, Can I Go Back and Vote for the Other Guy?

The fact is, in the past, Databases really have been Candidate B in this scenario; let me give you a bit more detail and it will be crystal clear why Developers have historically shunned them.

1. I’m a Developer Not a Database Administrator and My End-Users Are Business Analysts.

Developers know that a database will offer them more functionality than a flat file, everything from built-in indexing, networking, administrative and underlying at-rest and in-transit security, and far more. Just undertaking a do-it-yourself project for indexing or writing search and sort routines gives most developers enough of a headache to know a database offers more. However, with a flat file, all of that extra code is directly under their control and they can manage the design of it and it’s completely invisible to end-users and configurable by them before anything is ever deployed. Databases have historically needed direct and periodic management to meet the needs of their end-users and their workloads, with tuning and configuration through command-line interfaces, not embedded in programming APIs.

2. You Expect Me to Stuff This Gigantic Square Peg in This Little Round Hole?

As I mentioned in the last flat file blog, most operational technology has very limited compute resources. Although modern embedded platforms are relatively resource-rich, they are generally not capable of running your typical enterprise database which can easily be half a gigabyte to several gigabytes as a deployed footprint (obviously, this doesn’t include the application/user-created data tables, schemas, etc.). File systems management presents very little overhead, often in the tens of kilobyte ranges and there’s no additional configuration needed.

3. Am I Better Off Than I Was With the Prior Regime, at Least They Understood Me?

The size and constituency are only part of the problem, indicators of what the traditional database was designed to do and run on, namely as a relational database running in a data center, supporting either transactional operations on data in real-time or analytical operations or larger sets of data in hindsight.  Most developers don’t want relational mapping, they want a simple API to directly access the data they are managing – generally, real-time but for more than discrete transactions or after-the-fact analytics.  Since they’re not interested in relational mapping they’re generally also uninterested in picking up SQL skills.

Respect the Legacy, But Move Towards the Future

I get it, as I said in the last blog on flat files, I used to be one of these OT engineers myself and I hated databases but that was in the late 1980s and early 1990s. There are far more choices out there today, some of them, like Actian Zen, are purpose-built for these environments and cater to the preferences of modern developers.

Actian is the industry leader in operational data warehouse and edge data management solutions for modern businesses. With a complete set of connected solutions to help you manage data on-premises, in the cloud, and at the edge with mobile and IoT. Actian can help you develop the technical foundation needed to support true business agility.

To learn more, visit www.actian.com.

About Actian Corporation

Actian makes data easy. We deliver cloud, hybrid, and on-premises data solutions that simplify how people connect, manage, and analyze data. We transform business by enabling customers to make confident, data-driven decisions that accelerate their organization’s growth. Our data platform integrates seamlessly, performs reliably, and delivers at industry-leading speeds.