Entity Framework Core & lazy loading

Entity Framework Core & lazy loading

In Entity Framework 7, lazy loading is a technique used to delay the loading of related entities until they are actually needed. This can help to improve the performance of an application by reducing the amount of data that is retrieved from the database upfront.

To implement lazy loading in EF7, the “virtual” modifier is used on the navigation properties of an entity class. Navigation properties are used to represent relationships between entities, such as one-to-many or many-to-many.

For example, consider the following code snippet for a “Course” entity class and a “Student” entity class in EF7, with a one-to-many relationship between them:

public class Course
{
    public int CourseId { get; set; }
    public string CourseName { get; set; }
    public virtual ICollection<Student> Students { get; set; }
}

public class Student
{
    public int StudentId { get; set; }
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public int CourseId { get; set; }
    public virtual Course Course { get; set; }
}

 

In this example, the “Students” navigation property in the “Course” class and the “Course” navigation property in the “Student” class are both marked as “virtual”. This allows EF7 to override these properties with a proxy at runtime, enabling lazy loading for the related entities.

To use lazy loading in an EF7 application, the “DbContext.LazyLoadingEnabled” property must be set to “true”. When lazy loading is enabled, related entities will not be loaded from the database until they are actually accessed.

For example, the following code demonstrates how lazy loading can be used to retrieve a list of courses and their students:

using (var context = new SchoolContext())
{
    context.LazyLoadingEnabled = true;
    var courses = context.Courses.ToList();
    foreach (var course in courses)
    {
        Console.WriteLine("Course: " + course.CourseName);
        foreach (var student in course.Students)
        {
            Console.WriteLine("Student: " + student.FirstName + " " + student.LastName);
        }
    }
}

In this code, the list of courses is retrieved from the database and stored in the “courses” variable. The students for each course are not retrieved until they are accessed in the inner loop. This allows the application to retrieve only the data that is needed, improving performance and reducing the amount of data transferred from the database.

Lazy loading can be a useful tool for optimizing the performance of an EF7 application, but it is important to consider the trade-offs and use it appropriately. Lazy loading can increase the number of database queries and the overall complexity of an application, and it may not always be the most efficient approach.

 

5 Good Practices for Integration Testing with NUnit

5 Good Practices for Integration Testing with NUnit

Integration tests are a crucial part of any software development process, as they help ensure that different parts of a system are working together correctly. When writing integration tests, it is important to follow best practices in order to ensure that your tests are effective and maintainable. Here are a few good practices for integration testing using NUnit:

  1. Use test fixtures: NUnit provides a concept called “test fixtures,” which allow you to set up and tear down common resources that are needed by multiple test cases. This can help reduce duplication and make your tests more maintainable.
    [TestFixture]
    public class DatabaseTests
    {
        private Database _database;
    
        [SetUp]
        public void SetUp()
        {
            _database = new Database();
        }
    
        [TearDown]
        public void TearDown()
        {
            _database.Dispose();
        }
    
        [Test]
        public void Test1()
        {
            // test code goes here
        }
    
        [Test]
        public void Test2()
        {
            // test code goes here
        }
    }
    

     

  2. Use setup and teardown methods: In addition to test fixtures, NUnit also provides setup and teardown methods that can be used to perform common tasks before and after each test case. This can be helpful for setting up test data or cleaning up after a test.
    [TestFixture]
    public class DatabaseTests
    {
        private Database _database;
    
        [SetUp]
        public void SetUp()
        {
            _database = new Database();
        }
    
        [TearDown]
        public void TearDown()
        {
            _database.Dispose();
        }
    
        [SetUp]
        public void TestSetup()
        {
            // setup code goes here
        }
    
        [TearDown]
        public void TestTeardown()
        {
            // teardown code goes here
        }
    
        [Test]
        public void Test1()
        {
            // test code goes here
        }
    
        [Test]
        public void Test2()
        {
            // test code goes here
        }
    }
    

     

  3. Use test cases: NUnit allows you to specify multiple test cases for a single test method using the TestCase attribute. This can help reduce duplication and make it easier to test different scenarios.
    [TestFixture]
    public class CalculatorTests
    {
        [TestCase(1, 2, 3)]
        [TestCase(10, 20, 30)]
        [TestCase(-1, -2, -3)]
        public void TestAdd(int x, int y, int expected)
        {
            Calculator calculator = new Calculator();
            int result = calculator.Add(x, y);
            Assert.AreEqual(expected, result);
        }
    }
    

     

  4. Use the Assert class: NUnit provides a variety of assertions that can be used to verify the behavior of your code. It is important to use these assertions rather than manually checking for expected results, as they provide better error messages and make it easier to debug test failures.
    [TestFixture]
    public class CalculatorTests
    {
        [Test]
        public void TestAdd()
        {
            Calculator calculator = new Calculator();
            int result = calculator.Add(1, 2);
            Assert.AreEqual(3, result);
        }
    
        [Test]
        public void TestSubtract()
        {
            Calculator calculator = new Calculator();
            int result = calculator.Subtract(10, 5);
            Assert.AreEqual(5, result);
        }
    }
    

     

  5. Use test categories: NUnit allows you to categorize your tests using the Category attribute. This can be helpful for organizing your tests and selectively running only certain categories of tests.
    [TestFixture]
    public class DatabaseTests
    {
        [Test]
        [Category("Database")]
        public void Test1()
        {
            // test code goes here
        }
    
        [Test]
        [Category("Database")]
        public void Test2()
        {
            // test code goes here
        }
    
        [Test]
        [Category("API")]
        public void Test3()
        {
            // test code goes here
        }
    }
       
    

     

By following these best practices, you can write integration tests that are effective, maintainable, and easy to understand. This will help you ensure that your code is working correctly and reduce the risk of regressions as you continue to develop and evolve your system.

Moving to apple silicon as a DotNet Developer

Moving to apple silicon as a DotNet Developer

ARM (Advanced RISC Machine) is a popular architecture for mobile devices and other low-power devices. Microsoft has supported ARM architectures in the .NET framework for many years, and this support has continued with the release of .NET 6 and .NET 7.

In .NET 6 and 7, support for ARM architectures has been improved and expanded in several ways. One of the key changes is the introduction of ARM64 JIT (Just-In-Time) compilation, which allows .NET applications to take advantage of the performance improvements offered by the ARM64 architecture. This means that .NET applications can now be compiled and run natively on ARM64 devices, providing better performance and a more seamless experience for users.

Another important change in .NET 6 and 7 is the support for ARM32 and ARM64 for ASP.NET and ASP.NET Core. This means that developers can now build and deploy web applications on ARM devices, making it easier to create cross-platform applications that can run on a wide range of devices.

In addition to these changes, .NET 6 and 7 also include support for ARM64 in the .NET Native toolchain, which allows developers to build native applications for ARM devices using C# and .NET. This makes it easier to create high-performance, native applications for ARM devices without having to write code in a different language.

In conclusion, the support for ARM architectures in .NET 6 and 7 is an important development for developers who are looking to create and deploy applications on devices such as Apple’s M1 and M2. With this support, developers can take advantage of the performance and capabilities of the ARM architecture to create powerful and efficient applications that can run on a variety of devices. This will make it easier for developers to create and deploy applications on a wide range of devices, including mobile devices and other low-power devices. Overall, the support for ARM architectures in .NET 6 and 7 is a major improvement that will help developers create and deploy high-quality applications on a variety of devices.

How to wrap your synchronous implementation in an asynchronous implementation.

How to wrap your synchronous implementation in an asynchronous implementation.

In this article, we will be discussing why it is sometimes useful to wrap your synchronous implementation in an asynchronous implementation.

Introduction

Async programming is an important paradigm in modern software development, allowing you to perform long-running tasks without blocking the calling thread. Async programming is particularly useful in scenarios where the operation may take a long time to complete, or when the operation is interacting with a slow resource, such as a network or a database.

One common scenario where you may need to wrap your synchronous implementation in an asynchronous implementation is when you are working with an API or a library that does not provide async versions of its methods. In these cases, you can use the Task.Run method to wrap the synchronous methods in a task, allowing you to use the await keyword to asynchronously wait for the operation to complete.

Example: Wrapping a Synchronous Data Processor

To illustrate this concept, let’s consider the following synchronous IDataProcessor interface:

public interface IDataProcessor
{
    void ProcessData(IEnumerable<IData> data);
}

This interface has a single method, ProcessData, which takes an IEnumerable of IData objects as input and processes the data.

Now let’s say that you want to use this IDataProcessor interface in an async context, but the interface does not provide an async version of the ProcessData method. To use this interface asynchronously, you can create an async wrapper class that wraps the synchronous implementation in an async implementation.

Here is an example of how you can wrap the synchronous IDataProcessor implementation in an asynchronous implementation:

public class AsyncDataProcessor : IDataProcessor
{
    private readonly IDataProcessor _dataProcessor;

    public AsyncDataProcessor(IDataProcessor dataProcessor)
    {
        _dataProcessor = dataProcessor;
    }

    public Task ProcessDataAsync(IEnumerable<IData> data)
    {
        return Task.Run(() => _dataProcessor.ProcessData(data));
    }
}

This implementation has a single method, ProcessDataAsync, which takes an IEnumerable of IData objects as input and asynchronously processes the data. The implementation uses the Task.Run method to wrap the synchronous ProcessData method in a task, allowing it to be called asynchronously using the await keyword.

To use this implementation, you can simply create an instance of AsyncDataProcessor and call the ProcessDataAsync method, passing in the list of data as an argument. For example:

var dataProcessor = new AsyncDataProcessor(new DataProcessor());
await dataProcessor.ProcessDataAsync(data);

This code creates an instance of the AsyncDataProcessor class and calls the ProcessDataAsync method, passing in the data object as an argument. The await keyword is used to asynchronously wait for the data processing to complete.

Conclusion

In this article, we discussed why it is sometimes useful to wrap your synchronous implementation in an asynchronous implementation. We used the Task.Run method to wrap a synchronous IDataProcessor implementation in an async implementation, allowing us to use the await keyword

 

Implementing asynchronous interfaces

Implementing asynchronous interfaces

In this article, we will be discussing how to define an async interface based on a synchronous interface example.

Introduction

Async interfaces are useful when you need to perform asynchronous operations within your application. Async interfaces allow you to define methods that return a Task object instead of a value, allowing you to use the await keyword to asynchronously wait for the operation to complete.

One important aspect of async programming is the ability to cancel an async operation. This is particularly useful in scenarios where the async operation may take a long time to complete, or when the operation is no longer needed. To support cancellation in async operations, you can use a cancellation token.

Step 1: Define the Synchronous Interface

The first step in defining an async interface is to define the synchronous version of the interface. This will serve as the basis for the async interface.

Here is an example of a synchronous interface:

public interface IDataProcessor
{
    void ProcessData(IEnumerable<IData> data);
}

This interface has a single method, ProcessData, which takes an IEnumerable of IData objects as input and processes the data.

Step 2: Define the Async Interface

Now that we have defined the synchronous interface, we can define the async version of the interface. To do this, we simply need to modify the ProcessData method to return a Task object instead of void, and add the async keyword to the method.

Here is the async version of the IDataProcessorAsync interface:

public interface IDataProcessorAsync
{
    Task ProcessDataAsync(IEnumerable<IData> data, CancellationToken cancellationToken);
}

In this version of the interface, we have added a cancellationToken parameter to the ProcessDataAsync method. This parameter is of type CancellationToken, which is a struct that represents a cancellation request.

Step 3: Implement the Async Interface

Now that we have defined the async version of the interface, we can implement it in a class. To implement the async interface, we simply need to define a method that matches the signature of the ProcessDataAsync method and uses the await keyword to asynchronously perform the data processing.

Here is an example of an async implementation of the IDataProcessorAsync interface:

public class DataProcessor : IDataProcessorAsync
{
    public async Task ProcessDataAsync(IEnumerable<IData> data, CancellationToken cancellationToken)
    {
        // Process the data asynchronously
        await Task.Delay(1000, cancellationToken);
    }
}

This implementation has a single method, ProcessDataAsync, which takes an IEnumerable of IData objects and a cancellationToken as input and asynchronously processes the data. In this example, the data processing is simulated using the Task.Delay method, which causes the task to wait for a specified amount of time before completing. The cancellationToken is passed to the Task.Delay method as an argument, allowing the task to be cancelled if a cancellation request is made.

Step 4: Use the Async Interface

Now that we have defined and implemented the async interface, we can use it in our application. To use the async interface, we simply need to create an instance of the implementing class and call the async method using the await keyword.

Here is an example of how to use the async IDataProcessor interface:

var cts = new CancellationTokenSource();
var dataProcessor = new DataProcessor();
await dataProcessor.ProcessDataAsync(data, cts.Token);

This code creates a CancellationTokenSource object, which is used to create and manage a cancellation token. The cts.Token property is then passed to the ProcessDataAsync method as a cancellation token. The await keyword is used to asynchronously wait for the data processing to complete.

To cancel the async operation, you can call the Cancel method on the CancellationTokenSource object. This will trigger a cancellation request, which will cause the ProcessDataAsync method to throw a TaskCanceledException when the await keyword is encountered.


cts.Cancel();

Conclusion

In this article, we discussed how to define an async interface based on a synchronous interface example, including the use of a cancellation token. We defined the synchronous version of the interface, modified it to include a cancellation token and return a Task object, implemented the async interface in a class, and demonstrated how to use your new async interface implementation.

AspNetCore Kestrel: change your application port using a configuration file

AspNetCore Kestrel: change your application port using a configuration file

Modern AspNetCore applications use the built-in web server kestrel,this server is usually bound to the localhost address using the ports 5000 and 5001 for http and https.

But what if you want to run 2 applications in the same server? then you have a problem because if you use the default ports one of the applications will not start correctly.

This can easily be solved by changing the default ports in your WebHostBuilder as shown below

public static IHostBuilder CreateHostBuilder(string[] args) =>
    Host.CreateDefaultBuilder(args)
   .ConfigureWebHostDefaults(webBuilder => {
       webBuilder.UseUrls("http://0.0.0.0:8016");
       webBuilder.UseStartup<Startup>();
   });

The problem with the example above is that the URLs are hardcoded, so here is a better solution

public static IHostBuilder CreateHostBuilder(string[] args) =>
  Host.CreateDefaultBuilder(args)
  .ConfigureWebHostDefaults(webBuilder => {
      var config = new ConfigurationBuilder()
      .SetBasePath(Directory.GetCurrentDirectory())
      .AddJsonFile("hosting.json", optional: true)
      .AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
      .AddCommandLine(args)
      .AddEnvironmentVariables()
      .Build();

      webBuilder.UseUrls(config["server.urls"]);
      webBuilder.UseStartup<Startup>();
  });

the example above uses a configuration builder to merge the appsettings.json and the hosting.json in a single configuration object, then with use the value of the property “server.urls” as base URL/port for our application

Here is the content of the hosting.json file

{
    "server.urls": "http://0.0.0.0:8016" 
}

 

 

 

Blazor: Use tag helpers to include java scripts and CSS in your html header

Blazor: Use tag helpers to include java scripts and CSS in your html header

Sometime we want to reuse our Blazor components in another apps, the best way to do this is to create a razor library, this process of create a razor library is not different from create a normal class library to share code. There is only one exception, razor components might need to reference JavaScript or CSS files. This problem can be easily solve in 2 steps as shown below.

1) Create a class that inherits from TagHelperComponent,,this class should include the tags that you want to include in the html header section of your app

using Microsoft.AspNetCore.Html;
using Microsoft.AspNetCore.Razor.TagHelpers;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;


namespace MyBlazorApp
{
  [HtmlTargetElement("head")]
  public class MyTagHelper: TagHelperComponent
  {
    private string Tags= 
@"
        <!-- ZXingBlazor -->
        <script src=""_content/ZXingBlazor/lib/barcodereader/zxing.js""></script>
        <script src = ""_content/ZXingBlazor/lib/barcodereader/barcode.js"" ></ script >
        < !--ZXingBlazor-- >
        < !--Signature Pad  -->
        <script src = ""_content/Mobsites.Blazor.SignaturePad/bundle.js"" ></ script >
        < link href=""_content/Mobsites.Blazor.SignaturePad/bundle.css"" rel=""stylesheet"" />
        < link href=""_content/Ultra.PropertyEditors.Module.Blazor/js/signaturepropertyeditor.js""/>
        <!-- Signature Pad  -->
        <!-- HTML Editor  -->
        <link href = ""//cdn.quilljs.com/1.3.6/quill.snow.css"" rel=""stylesheet"">
        <link href = ""//cdn.quilljs.com/1.3.6/quill.bubble.css"" rel=""stylesheet"">
        <script src = ""https://cdn.quilljs.com/1.3.6/quill.js"" ></ script >
        <script src=""_content/Blazored.TextEditor/quill-blot-formatter.min.js""></script>
        <script src = ""_content/Blazored.TextEditor/Blazored-BlazorQuill.js"" ></ script >
        < !--HTML Editor  -->
";
    public override Task ProcessAsync(TagHelperContext context, TagHelperOutput output)
    {
      if (string.Equals(context.TagName, "head", StringComparison.OrdinalIgnoreCase))
      {
        output.PostContent.AppendHtml(Tags).AppendLine();
      }
      return Task.CompletedTask;
    }
  }
}

*Note: to reference JavaScript or CSS from any razor library you can use the following syntax,please notice the doble quotes.

<script src=""_content/MyAssemblyName/PathToMyJavaScript/MyJavaScriptFile.js""></script>

 

2) Create an extension method in the “Microsoft.Extensions.DependencyInjection” namespace so you can easily add your tag helper to the service collection

 

using Microsoft.AspNetCore.Razor.TagHelpers;
using Microsoft.Extensions.DependencyInjection;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace Microsoft.Extensions.DependencyInjection
{
  public static class StartupExtensions
  {
    public static IServiceCollection AddMyHtmlTags(this IServiceCollection services)
    {
      services.AddTransient<ITagHelperComponent, MyTagHelper>();
      return services;
    }
  }
}

 

Here is an example on how to use your new extension in your startup class

 public void ConfigureServices(IServiceCollection services
 {
   services.AddMyHtmlTags();
 }

 

Log XPO queries in a Netcore app (3,5,6)

Log XPO queries in a Netcore app (3,5,6)

The LogLevel section in the appsettings.json file does not affect the .NET Framework tracing mechanism, which is used by XPO to log the queries, still we have a few work arounds for a Netcore app

  1. We can implement our own logger as shown here https://docs.devexpress.com/XPO/403928/best-practices/how-to-log-sql-queries#xpo-logger-net
  2. We can set the value of the logging switch by reflection using the following snippet
private static void EnableXpoDebugLog()
{
    FieldInfo xpoSwitchF = typeof(ConnectionProviderSql).GetField("xpoSwitch", System.Reflection.BindingFlags.Static | System.Reflection.BindingFlags.NonPublic);
    TraceSwitch xpoSwitch = (TraceSwitch)xpoSwitchF.GetValue(null);
    xpoSwitch.Level = TraceLevel.Info;
}

I actually use the second work around in my own projects.

If you want to learn more about the logging mechanism of XPO you can take a look to the following articles

https://supportcenter.devexpress.com/ticket/details/t1033081/how-to-log-the-sql-queries-made-by-xpo-in-net-5-applications

https://supportcenter.devexpress.com/ticket/details/t913939/enable-logging-in-xaf-blazor

Implementing database synchronization with entity framework core

Implementing database synchronization with entity framework core

Ok, so far, our synchronization framework is only implemented for an in-memory database that we use for testing purposes.

Now let’s implement a different use case, lets add synchronization functionality to an entity framework core DbContext.

As I explained before, the key part of synchronizing data using delta encoding is to be able to track the differences that happen to a data object, in this case, a relational database.

these are the task that we need to do to accomplish our goal

  1. Find out how entity framework converts the changes that happen to the objects to SQL commands
  2. Decide what information we need to track and save as a delta
  3. Create the infrastructure to save deltas (IDeltaStore)
  4. Create the infrastructure to process deltas (IDeltaProcessor)
  5. Implement the synchronization node functionality in an Entity Framework DbContext(ISyncClientNode)
  6. Create a test scenario

 

1 Find out how entity framework converts the changes that happen to the objects to SQL commands

In our companies (BitFrameworks & Xari) we have been working in data synchronization for a while, but all this work has been done in the XPO realm.

We know that in most ORMs frameworks there is a layer of the ORM that is in charge of translating the changes made to objects into SQL commands, the trick is to locate this layer. So while I was trapped in Mexico waiting for a flight back to Phoenix, I decided to dig into entity framework’s core GitHub report, this is what I found https://github.com/dotnet/efcore/blob/b18a7efa7c418e43184db08c6d1488d6600054cb/src/EFCore.Relational/Update/Internal/BatchExecutor.cs#L161

public virtual async Task<int> ExecuteAsync(
           IEnumerable<ModificationCommandBatch> commandBatches,
           IRelationalConnection connection,
           CancellationToken cancellationToken = default)

As you can see one of the parameters is an IEnumerable of ModificationCommandBatch https://github.com/dotnet/efcore/blob/main/src/EFCore.Relational/Update/ModificationCommandBatch.cs this command batch exposes a read-only list of modification commands (ModificationCommand)

https://github.com/dotnet/efcore/blob/cc53b3e80755e5d882bb21ef10e0e0e33194d9bd/src/EFCore.Relational/Update/ModificationCommandBatch.cs#L30

public abstract class ModificationCommandBatch
{
    /// <summary>
    ///     The list of conceptual insert/update/delete <see cref="ModificationCommands" />s in the batch.
    /// </summary>
    public abstract IReadOnlyList<IReadOnlyModificationCommand> ModificationCommands { get; }

now let’s take look into the ModificationCommand https://github.com/dotnet/efcore/blob/main/src/EFCore.Relational/Update/ModificationCommand.cs this class provides all the information about the changes that will be converted into SQL commands, which means that if we serialize this object and save it as a delta we can then send it to another node and replicate the changes…VOILA!!!

Now here is a stone in our path, the class https://github.com/dotnet/efcore/blob/main/src/EFCore.Relational/Update/ModificationCommand.cs is not serializable or to say it in a better way NOT easily serializable, so let’s stop here for a moment and move to a different task

So now we know where the changes that we need to keep track of are, now let’s try to understand how those changes are converted into SQL commands and then executed into the database.

2 Decide what information we need to track and save as a delta

Entity framework core uses dependency injection to be able to handle different database engines so the idea here is that there are a lot of small services that can be replaced in other to create a different implementation, for example, SQLite, SqlServer, Postgres, etc …

After a lot of digging, I found that the service that is in charge of generating the update commands (insert, update and delete) UpdateSqlGenerator

https://github.com/dotnet/efcore/blob/main/src/EFCore.Relational/Update/UpdateSqlGenerator.cs

this class implements IUpdateSqlGenerator https://github.com/dotnet/efcore/blob/main/src/EFCore.Relational/Update/IUpdateSqlGenerator.cs and as you can see all methods receive a string builder and a ModificationCommand so this is the service  in charge of translating the ModificationCommand into SQL commands and SQL commands are easy to serialize because they are just text, so this is what we are going to serialize and save as a delta

    public interface IUpdateSqlGenerator
    {
        /// <summary>
        ///     Generates SQL that will obtain the next value in the given sequence.
        /// </summary>
        /// <param name="name">The name of the sequence.</param>
        /// <param name="schema">The schema that contains the sequence, or <see langword="null" /> to use the default schema.</param>
        /// <returns>The SQL.</returns>
        string GenerateNextSequenceValueOperation(string name, string? schema);

        /// <summary>
        ///     Generates a SQL fragment that will get the next value from the given sequence and appends it to
        ///     the full command being built by the given <see cref="StringBuilder" />.
        /// </summary>
        /// <param name="commandStringBuilder">The builder to which the SQL fragment should be appended.</param>
        /// <param name="name">The name of the sequence.</param>
        /// <param name="schema">The schema that contains the sequence, or <see langword="null" /> to use the default schema.</param>
        void AppendNextSequenceValueOperation(
            StringBuilder commandStringBuilder,
            string name,
            string? schema);

        /// <summary>
        ///     Appends a SQL fragment for the start of a batch to
        ///     the full command being built by the given <see cref="StringBuilder" />.
        /// </summary>
        /// <param name="commandStringBuilder">The builder to which the SQL fragment should be appended.</param>
        void AppendBatchHeader(StringBuilder commandStringBuilder);

        /// <summary>
        ///     Appends a SQL command for deleting a row to the commands being built.
        /// </summary>
        /// <param name="commandStringBuilder">The builder to which the SQL should be appended.</param>
        /// <param name="command">The command that represents the delete operation.</param>
        /// <param name="commandPosition">The ordinal of this command in the batch.</param>
        /// <returns>The <see cref="ResultSetMapping" /> for the command.</returns>
        ResultSetMapping AppendDeleteOperation(
            StringBuilder commandStringBuilder,
            IReadOnlyModificationCommand command,
            int commandPosition);

        /// <summary>
        ///     Appends a SQL command for inserting a row to the commands being built.
        /// </summary>
        /// <param name="commandStringBuilder">The builder to which the SQL should be appended.</param>
        /// <param name="command">The command that represents the delete operation.</param>
        /// <param name="commandPosition">The ordinal of this command in the batch.</param>
        /// <returns>The <see cref="ResultSetMapping" /> for the command.</returns>
        ResultSetMapping AppendInsertOperation(
            StringBuilder commandStringBuilder,
            IReadOnlyModificationCommand command,
            int commandPosition);

        /// <summary>
        ///     Appends a SQL command for updating a row to the commands being built.
        /// </summary>
        /// <param name="commandStringBuilder">The builder to which the SQL should be appended.</param>
        /// <param name="command">The command that represents the delete operation.</param>
        /// <param name="commandPosition">The ordinal of this command in the batch.</param>
        /// <returns>The <see cref="ResultSetMapping" /> for the command.</returns>
        ResultSetMapping AppendUpdateOperation(
            StringBuilder commandStringBuilder,
            IReadOnlyModificationCommand command,
            int commandPosition);
    }

3 Create the infrastructure to save deltas (Implementing IDeltaStore)

Now is time to create a delta store, this is an easy one since we only need to inherit from our delta store base and save the information in an entity framework DbContext, so here is the implementation

https://github.com/egarim/SyncFramework/blob/main/src/EntityFrameworkCore/BIT.Data.Sync.EfCore/EFDeltaStore.cs

if you want to compare it with other delta store implementations you can take a look at the in-memory version here

https://github.com/egarim/SyncFramework/blob/main/src/BIT.Data.Sync/Imp/MemoryDeltaStore.cs

4 Create the infrastructure to process deltas (implementing IDeltaProcessor)

So far, we know what we need to store in the deltas which basically is SQL commands and their parameters so it means to process those SQL Commands our delta processor needs to create a database connection and execute SQL commands

https://github.com/egarim/SyncFramework/blob/main/src/EntityFrameworkCore/BIT.Data.Sync.EfCore/EFDeltaProcessor.cs

public EFDeltaProcessor(DbContext dBContext) 
{
    _dBContext = dBContext;
  
}
public EFDeltaProcessor(string connectionstring, string DbEngineAlias, string ProviderInvariantName)
{

    this.CurrentDbEngine = DbEngineAlias;
    this.connectionString = connectionstring;

    try
    {
        factory = DbProviderFactories.GetFactory(ProviderInvariantName);
    }
    catch (Exception ex)
    {
        Debug.WriteLine(ex.Message);
        throw new Exception("There was a problem creating the database connection using DbProviderFactories.GetFactory. Please your make sure the DbProviderFactory for your database is registered https://docs.microsoft.com/en-us/dotnet/api/system.data.common.dbproviderfactories.registerfactory?view=net-5.0", ex);
    }
    //TODO check provider registration later

    //DbProviderFactories.RegisterFactory("Microsoft.Data.SqlClient", SqlClientFactory.Instance);
}

there are a few things to notice in that class, first, it has 2 constructors because we need 2 different ways to create the connection to the database, one using the entity framework DbContext and one using ADO.NET DbProviderFactory

All the magic happens in the ProcessDeltas method, this method is in charge of, extract the content of the deltas and transforming them into SQL commands and parameters, and then executing the command.

please notice that the content of each delta is an instance of ModificationCommandData

https://github.com/egarim/SyncFramework/blob/main/src/EntityFrameworkCore/BIT.Data.Sync.EfCore/Data/ModificationCommandData.cs

which is a class that allows us to store multiple SQL commands (for different database engines) and their parameters

5 Implement the synchronization node functionality in an Entity Framework DbContext(ISyncClientNode)

At the moment we are able to produce and process deltas for entity framework relational, so the next step is to implement the functionality of synchronization client node by implementing the following interface

https://github.com/egarim/SyncFramework/blob/main/src/BIT.Data.Sync/Client/ISyncClientNode.cs

namespace BIT.Data.Sync.Client
{
    public interface ISyncClientNode
    {
        IDeltaProcessor DeltaProcessor { get; }
        IDeltaStore DeltaStore { get; }
        ISyncFrameworkClient SyncFrameworkClient { get; }
        string Identity { get;  }

    }
}

https://github.com/egarim/SyncFramework/blob/main/src/EntityFrameworkCore/BIT.Data.Sync.EfCore/SyncFrameworkDbContext.cs

The server-side

I’m not going to show the implementation of the server since that implementation is generic and uses the same delta store and delta processor that we created at the beginning of this article. for more information check the following links

Adding network support

https://www.jocheojeda.com/2021/10/17/syncframework-adding-network-support/

Testing network support

https://www.youtube.com/watch?v=mSl0n0O5QIg&t=4s

 

The next post its going to be a video testing a simple synchronization scenario, see you in the next post!!!

 

 

SyncFramework – Adding network support

SyncFramework – Adding network support

So far, all our test exists inside the same process, so they communicate through variables, in real-life scenarios nodes won’t exist in the same process and most of the time not even in the same location.

The easiest and most standard way to implement client-server communication in the dotnet world is a rest API, so let’s define an API client and a server service, lets start with the API client

ISyncFrameworkClient

https://github.com/egarim/SyncFramework/tree/main/src/BIT.Data.Sync/Client/ISyncFrameworkClient.cs

public interface ISyncFrameworkClient
{
    Task<List<Delta>> FetchAsync(Guid startindex, string identity, CancellationToken cancellationToken);
    Task PushAsync(IEnumerable<IDelta> Deltas, CancellationToken cancellationToken);
}

As you can see in the code above, this interface is really simple because the client only implements 2 operations

  • Fetch: downloads the deltas from the server
  • Push: upload the deltas from our delta store to the server

Now we should create an implementation for this interface, see the code below

https://github.com/egarim/SyncFramework/tree/main/src/BIT.Data.Sync/Client/SyncFrameworkHttpClient.cs

 

public class SyncFrameworkHttpClient : ISyncFrameworkClient
{
    HttpClient _httpClient;
    public string DeltaStoreId { get; }
    public SyncFrameworkHttpClient(HttpClient httpClient,string NodeId)
    {
        this.DeltaStoreId = NodeId;
        _httpClient = httpClient;
        _httpClient.DefaultRequestHeaders.Add("NodeId", NodeId);
      
        this.DeltaStoreId = NodeId;
    }
    public SyncFrameworkHttpClient(string BaseAddress, string DeltaStoreId):this(new HttpClient() { BaseAddress=new Uri(BaseAddress)},DeltaStoreId)
    {
       
    }
    public virtual async Task PushAsync(IEnumerable<IDelta> Deltas, CancellationToken cancellationToken  = default)
    {
       
        try
        {
            List<Delta> toserialzie = new List<Delta>();
            foreach (IDelta delta in Deltas)
            {
                toserialzie.Add(new Delta(delta));
            }
            cancellationToken.ThrowIfCancellationRequested();

            DataContractJsonSerializer js = new DataContractJsonSerializer(typeof(List<Delta>));
            MemoryStream msObj = new MemoryStream();
            js.WriteObject(msObj, toserialzie);
            msObj.Position = 0;
            StreamReader sr = new StreamReader(msObj);
            string jsonDeltas = sr.ReadToEnd();

            var data = new StringContent(jsonDeltas, Encoding.UTF8, "application/json");
            await _httpClient.PostAsync("/Sync/Push", data, cancellationToken).ConfigureAwait(false);
        }
        catch (Exception ex)
        {
            var message = ex.Message;
            throw;
        }
      
    }
    public virtual async Task<List<Delta>> FetchAsync(Guid startindex, string identity, CancellationToken cancellationToken = default)
    {
        var QueryParams = new Dictionary<string, string>();
        QueryParams.Add(nameof(startindex), startindex.ToString());
        QueryParams.Add(nameof(identity), identity);
        cancellationToken.ThrowIfCancellationRequested();
        var query = HttpUtility.ParseQueryString("");
        foreach (KeyValuePair<string, string> CurrentParam in QueryParams)
        {
            query[CurrentParam.Key] = CurrentParam.Value;
        }
        var reponse = await _httpClient.GetStringAsync($"/Sync/Fetch?{query.ToString()}").ConfigureAwait(false);

        using (var ms = new MemoryStream(Encoding.Unicode.GetBytes(reponse)))
        {

            DataContractJsonSerializer deserializer = new DataContractJsonSerializer(typeof(List<Delta>));
            List<Delta> Deltas = (List<Delta>)deserializer.ReadObject(ms);

            return Deltas;
        }

        //List<Delta> Deltas = JsonConvert.DeserializeObject<List<Delta>>(reponse);
        return null;

    }

}

 

it’s an implementation of the ISyncFrameworkClient interface using HTTP communication

  • Fetch: uses an HTTP get request
  • Push: uses an HTTP post request

Also, the “nodeid” header is added to the request, you will understand why when we implement the server part.

Now that we have defined the contract for the client and also provided the base implementation using an HTTP client, its time to define what a client node is, please take a look at the code below

ISyncClientNodeExtensions

https://github.com/egarim/SyncFramework/tree/main/src/BIT.Data.Sync/Client/ISyncClientNodeExtensions.cs

    public static class ISyncClientNodeExtensions
    {

        public static async Task<List<Delta>> FetchAsync(this ISyncClientNode instance, CancellationToken cancellationToken = default)
        {
            cancellationToken.ThrowIfCancellationRequested();
            var LastDetalIndex = await instance.DeltaStore.GetLastProcessedDeltaAsync(cancellationToken).ConfigureAwait(false);
            return await instance.SyncFrameworkClient.FetchAsync(LastDetalIndex, instance.Identity, cancellationToken).ConfigureAwait(false);
        }
        public static async Task PullAsync(this ISyncClientNode instance, CancellationToken cancellationToken = default)
        {
            cancellationToken.ThrowIfCancellationRequested();
            var Deltas = await instance.FetchAsync(cancellationToken).ConfigureAwait(false);
            if (Deltas.Any())
            {
                await instance.DeltaProcessor.ProcessDeltasAsync(Deltas, cancellationToken).ConfigureAwait(false);
                Guid index = Deltas.Max(d => d.Index);
                await instance.DeltaStore.SetLastProcessedDeltaAsync(index, cancellationToken).ConfigureAwait(false);
            }

        }
        public static async Task PushAsync(this ISyncClientNode instance, CancellationToken cancellationToken = default)
        {
            cancellationToken.ThrowIfCancellationRequested();
            var LastPushedDelta = await instance.DeltaStore.GetLastPushedDeltaAsync(cancellationToken).ConfigureAwait(false);
            var Deltas = await instance.DeltaStore.GetDeltasAsync(LastPushedDelta,cancellationToken).ConfigureAwait(false);
            if (Deltas.Any())
            {
                var Max = Deltas.Max(d => d.Index);
                await instance.SyncFrameworkClient.PushAsync(Deltas, cancellationToken).ConfigureAwait(false);
                await instance.DeltaStore.SetLastPushedDeltaAsync(Max,cancellationToken).ConfigureAwait(false);
            }



        }
    }

so, this is how the SyncClientNode is structured

Let’s move to the server-side now, here the idea is to be able to host multiple delta store and delta processors and also to be able to introduce custom logic either saving the deltas into the delta store or processing the deltas into a data object

ISyncServerNode

https://github.com/egarim/SyncFramework/blob/main/src/BIT.Data.Sync/Server/ISyncServerNode.cs

public interface ISyncServerNode
{
    string NodeId { get; set; }
    Task SaveDeltasAsync(IEnumerable<IDelta> deltas, CancellationToken cancellationToken);
    Task<IEnumerable<IDelta>> GetDeltasAsync(Guid startindex, string identity, CancellationToken cancellationToken);
    Task ProcessDeltasAsync(IEnumerable<IDelta> deltas, CancellationToken cancellationToken);
}

Now we need to define a server, so here is the interface for the SyncServer

ISyncServer

https://github.com/egarim/SyncFramework/blob/main/src/BIT.Data.Sync/Server/ISyncServer.cs

public interface ISyncServer
{
    IEnumerable<ISyncServerNode> Nodes { get; }
    Task<IEnumerable<IDelta>> GetDeltasAsync(string name, Guid startindex, string identity, CancellationToken cancellationToken);
    Task ProcessDeltasAsync(string Name, IEnumerable<IDelta> deltas, CancellationToken cancellationToken);
    Task SaveDeltasAsync(string name, IEnumerable<IDelta> deltas, CancellationToken cancellationToken);
}

As you can see, the members are almost the same as the sync node, this design allows us to have more than one node on the server-side

Here is the implementation of the SyncServer

SyncServer

https://github.com/egarim/SyncFramework/blob/main/src/BIT.Data.Sync/Server/SyncServer.cs

public class SyncServer: ISyncServer
{


    IEnumerable<ISyncServerNode> _Nodes;
   
    public SyncServer(params ISyncServerNode[] Nodes)
    {
        this._Nodes = Nodes;
    }

    public IEnumerable<ISyncServerNode> Nodes => _Nodes;

   

    public async Task<IEnumerable<IDelta>> GetDeltasAsync(string NodeId, Guid startindex, string identity, CancellationToken cancellationToken)
    {
        cancellationToken.ThrowIfCancellationRequested();
        ISyncServerNode Node = GetNode(NodeId);
        if (Node != null)
        {
            return await Node.GetDeltasAsync(startindex, identity, cancellationToken).ConfigureAwait(false);
        }

        IEnumerable<IDelta> result = new List<IDelta>();
        return result;
    }
    public Task ProcessDeltasAsync(string NodeId, IEnumerable<IDelta> deltas, CancellationToken cancellationToken)
    {
        cancellationToken.ThrowIfCancellationRequested();
        ISyncServerNode Node = GetNode(NodeId);
        if (Node != null)
        {
            return Node.ProcessDeltasAsync(deltas, cancellationToken);
        }
        return null;

    }

    private ISyncServerNode GetNode(string NodeId)
    {
        return Nodes.FirstOrDefault(node => node.NodeId == NodeId);
    }

    public Task SaveDeltasAsync(string NodeId, IEnumerable<IDelta> deltas, CancellationToken cancellationToken)
    {
        cancellationToken.ThrowIfCancellationRequested();

        ISyncServerNode Node = GetNode(NodeId);

        if (Node != null)
        {
            return Node.SaveDeltasAsync(deltas, cancellationToken);
        }
        return Task.CompletedTask;
    }
}

the following pictures show the 2 possible server implementations

 

and that’s it for this post, in the next post I will show the test cases for this implementation