We have been building a data synchronization tool that can determine record changes in three different ways:
- Reading the archive log.
- Leveraging the DataExchange PDC tables.
- Creating and using our own change tracking keys.
Of the three solutions, none are perfect.
- The first requires the use of the Archive Logging feature of the database. However, this also requires that you have downtime to clear and release the archive logs whenever you want to synchronize changes. This method is also prone to some really bad problems if the archive log gets out of sync, such that you have to re-sync the entire table. I really don't recommend this, but it IS possible to use this metadata.
- The second option requires the additional purchase and implementation of DataExchange, which is just as costly as AuditMaster. Once our solution is finalized, we hope to come up with a less-expensive option to collect the same data, but it would still require the use of the DX PDC tables to maintain the metadata, which can then be easily synchronized with the external system.
- The third option has the advantage that it doesn't require any additional "tracking" components, since we build our own change log based on CRC's of each record, but effective delta synchronization still requires examining EVERY record in the data file every time through.
Once the changes are exported to CSV files, they can then be read into a target database as a list of changes only. Our eventual plan (not yet completed) is to enable ODBC access on the target database, such that we could synchronize directly with any ODBC-compatible target system.
We are currently looking for a few beta testers to play with the current version. If you are interested, please let me know.
Of course, if you are ONLY interested in trying to find out if a file has changed, then you can simply check the datestamp. This is always updated when the file is closed, and it is updating on EVERY disk write when the "Use System Cache" setting is checked.