SyncFramework 3.0: .NET 10, an in-memory client and an official MySQL provider

SyncFramework is my C# library for synchronizing data with delta encoding. Each client records its changes as
deltas, pushes them to a server, and pulls the deltas the other clients pushed. Only the changes travel, never
the whole dataset.
Today I published a new release to nuget.org. It's the first one built for .NET 10, and it's tagged v3.0.0 in
the repository.
The packages
| Package | Previous | Now |
|---|---|---|
| BIT.Data.Sync | 2.34.0 | 3.0.0 |
| BIT.Data.Sync.AspNetCore | 2.34.0 | 3.0.0 |
| BIT.Data.Sync.EfCore | 9.2.33 | 10.0.0 |
| BIT.Data.Sync.EfCore.Sqlite | 9.2.33 | 10.0.0 |
| BIT.Data.Sync.EfCore.SqlServer | 9.2.33 | 10.0.0 |
| BIT.Data.Sync.EfCore.Npgsql | 9.2.33 | 10.0.0 |
| BIT.Data.Sync.EfCore.MySql | new | 10.0.0 |
The previous releases targeted net9.0. Everything now targets .NET 10 only.
The version numbers look odd side by side, so here is the rule. The core goes to 3.0.0 because it drops net9.0
and has one binary break. The EF Core packages follow EF Core's major version, which is 10.
What's new
1. An in-memory client
SyncFrameworkInMemoryClient (namespace BIT.Data.Sync.Client) is an ISyncFrameworkClient that talks to an
ISyncFrameworkServer in the same process. There is no HTTP in between. It answers Push, Fetch and HandShake the
way the HTTP controller does.
It also copies every delta at the boundary, the way serialization over HTTP would. That detail matters. The server
re-indexes the deltas it saves, and without the copy it would renumber the client's own stored deltas. With the
copy, it never does. There is a test that fails if you remove the copy.
I wrote it for tests, simulations and apps that host their own server. This is the whole setup, taken from the
repository's tests:
var server = new SyncFrameworkServer(new SyncServerNode(new MemoryDeltaStore(), null, "node"));
var db = new SimpleDatabase("A", new SyncFrameworkInMemoryClient(server, "node"));
await db.PushAsync();
await db.PullAsync();
The tests run three databases through one in-memory server and check that they converge, that updates and
deletes propagate, and that an unknown node fails with a message.
2. An official MySQL provider
BIT.Data.Sync.EfCore.MySql is a new package. It uses Oracle's MySql.EntityFrameworkCore 10.0.12.
The old package, BIT.Data.Sync.EfCore.Pomelo.MySql, used Pomelo's provider. Pomelo's provider failed at runtime
on EF Core 10, with a MissingMethodException. The Pomelo package gets no further releases. If you use MySQL,
move to BIT.Data.Sync.EfCore.MySql.
The MySqlPomelo updater alias stays, so deltas stored by Pomelo clients can still be read.
3. Current .NET 10 dependencies
The packages now use EF Core 10.0.14, Npgsql EF 10.0.3 and ASP.NET Core 10.0.14.
BIT.Data.Sync.AspNetCore now uses the shared ASP.NET Core framework instead of the old
Microsoft.AspNetCore.Mvc.Core 2.2.5 package, which was from 2019.
4. Work from April, finally released
In April I wrote two things that never made it to a release: the SyncFramework.Dashboard, a Blazor Server admin
UI, and the MySqlDeltaGenerator for the official MySQL driver. Both are in this release. The dashboard is not a
NuGet package; it lives in the repository under src/Tools/SyncFramework.Dashboard. It lists
the nodes with their delta counts, adds new nodes with a memory or SQLite store, and replays a node's deltas into
a SQLite database so you can inspect them. The details are in the 10.0.0 entry of CHANGELOG.md.
Bug fixes
Five bugs, fixed in two pull requests.
#27, MemoryDeltaStore: a client that pulled before its first push got the server's index as its push
cursor. The server's index counts ahead of the client's own, so the client's deltas were never pushed. The push
still reported success, with the message "Nothing to send". Now a pull sets only the pull cursor, and an identity
with no push yet reads as nothing pushed. The test ADatabaseThatPullsBeforeItsFirstPushStillPushesAllItsDeltas
covers it: it got 3 records instead of 5 before the fix.
#26 fixed four more:
SyncFrameworkServer: the explicitISyncFrameworkServer.RegisterNodeFunctionthrew
NotImplementedException, so the server failed when used through its interface.SyncFrameworkHttpClient.PushAsyncreturnednullon a non-success HTTP status. It now returns a failed
response with the status and body.MemoryDeltaStore: a push moved the pull cursor to the client's own push index, so after pushing, a client
skipped other clients' deltas with lower server indexes. The pull cursor also came backnullfor a status
created by a push.SyncControllerBase.PushreturnedSuccess = falsewith no message for three kinds of exception.
The EF Core delta store wasn't affected by the cursor bugs. It already kept the two cursors apart.
Breaking changes
- .NET 10 only. No package targets net9.0 any more.
SimpleDatabase.UpdateandDeletenow returnTaskinstead ofasync void, so you can await them and
see their exceptions. Your source still compiles, but anything compiled against 2.x that calls these two
methods must be rebuilt.
Where it came from
I found the pull-before-push bug while using SyncFramework's core as the transport for a signed work ledger in
Creatura, an agent framework I'm building. Each signed entry travels as one delta, and every client checks the
signatures of what it pulls.
Upgrading
Pick the packages you use:
dotnet add package BIT.Data.Sync --version 3.0.0
dotnet add package BIT.Data.Sync.AspNetCore --version 3.0.0
dotnet add package BIT.Data.Sync.EfCore --version 10.0.0
dotnet add package BIT.Data.Sync.EfCore.Sqlite --version 10.0.0
dotnet add package BIT.Data.Sync.EfCore.SqlServer --version 10.0.0
dotnet add package BIT.Data.Sync.EfCore.Npgsql --version 10.0.0
dotnet add package BIT.Data.Sync.EfCore.MySql --version 10.0.0
If you used BIT.Data.Sync.EfCore.Pomelo.MySql, remove it and add BIT.Data.Sync.EfCore.MySql.
Links
- BIT.Data.Sync on NuGet: https://www.nuget.org/packages/BIT.Data.Sync
- BIT.Data.Sync.EfCore.MySql on NuGet: https://www.nuget.org/packages/BIT.Data.Sync.EfCore.MySql
- Source: https://github.com/egarim/SyncFramework
- Pull request #26: https://github.com/egarim/SyncFramework/pull/26
- Pull request #27: https://github.com/egarim/SyncFramework/pull/27