Using the IMemoryStore Interface and DevExpress XPO ORM to Implement a Custom Memory Store for Semantic Kernel

Using the IMemoryStore Interface and DevExpress XPO ORM to Implement a Custom Memory Store for Semantic Kernel

In today’s AI-driven world, the ability to quickly and efficiently store, retrieve, and manage data is crucial for developing sophisticated applications. One tool that helps facilitate this is the Semantic Kernel, a lightweight, open-source development kit designed for integrating AI models into C#, Python, or Java applications. It enables rapid enterprise-grade solutions by serving as an effective middleware.

One of the key concepts in Semantic Kernel is memory—a collection of records, each containing a timestamp, metadata, embeddings, and a key. These memory records can be stored in various ways, depending on how you implement the interfaces. This flexibility allows you to define the storage mechanism, which means you can choose any database solution that suits your needs.

In this blog post, we’ll walk through how to use the IMemoryStore interface in Semantic Kernel and implement a custom memory store using DevExpress XPO, an ORM (Object-Relational Mapping) tool that can interact with over 14 database engines with a single codebase.

Why Use DevExpress XPO ORM?

DevExpress XPO is a powerful, free-to-use ORM created by DevExpress that abstracts the complexities of database interactions. It supports a wide range of database engines such as SQL Server, MySQL, SQLite, Oracle, and many others, allowing you to write database-independent code. This is particularly helpful when dealing with a distributed or multi-environment system where different databases might be used.

By using XPO, we can seamlessly create, update, and manage memory records in various databases, making our application more flexible and scalable.

Implementing a Custom Memory Store with DevExpress XPO

To integrate XPO with Semantic Kernel’s memory management, we’ll implement a custom memory store by defining a database entry class and a database interaction class. Then, we’ll complete the process by implementing the IMemoryStore interface.

Step 1: Define a Database Entry Class

Our first step is to create a class that represents the memory record. In this case, we’ll define an XpoDatabaseEntry class that maps to a database table where memory records are stored.


public class XpoDatabaseEntry : XPLiteObject {
    private string _oid;
    private string _collection;
    private string _timestamp;
    private string _embeddingString;
    private string _metadataString;
    private string _key;

    [Key(false)]
    public string Oid { get; set; }
    public string Key { get; set; }
    public string MetadataString { get; set; }
    public string EmbeddingString { get; set; }
    public string Timestamp { get; set; }
    public string Collection { get; set; }

    protected override void OnSaving() {
        if (this.Session.IsNewObject(this)) {
            this.Oid = Guid.NewGuid().ToString();
        }
        base.OnSaving();
    }
}

This class extends XPLiteObject from the XPO library, which provides methods to manage the record lifecycle within the database.

Step 2: Create a Database Interaction Class

Next, we’ll define an XpoDatabase class to abstract the interaction with the data store. This class provides methods for creating tables, inserting, updating, and querying records.


internal sealed class XpoDatabase {
    public Task CreateTableAsync(IDataLayer conn) {
        using (Session session = new(conn)) {
            session.UpdateSchema(new[] { typeof(XpoDatabaseEntry).Assembly });
            session.CreateObjectTypeRecords(new[] { typeof(XpoDatabaseEntry).Assembly });
        }
        return Task.CompletedTask;
    }

    // Other database operations such as CreateCollectionAsync, InsertOrIgnoreAsync, etc.
}

This class acts as a bridge between Semantic Kernel and the database, allowing us to manage memory entries without having to write complex SQL queries.

Step 3: Implement the IMemoryStore Interface

Finally, we implement the IMemoryStore interface, which is responsible for defining how the memory store behaves. This includes methods like UpsertAsync, GetAsync, and DeleteCollectionAsync.


public class XpoMemoryStore : IMemoryStore, IDisposable {
    public static async Task ConnectAsync(string connectionString) {
        var memoryStore = new XpoMemoryStore(connectionString);
        await memoryStore._dbConnector.CreateTableAsync(memoryStore._dataLayer).ConfigureAwait(false);
        return memoryStore;
    }

    public async Task CreateCollectionAsync(string collectionName) {
        await this._dbConnector.CreateCollectionAsync(this._dataLayer, collectionName).ConfigureAwait(false);
    }

    // Other methods for interacting with memory records
}

The XpoMemoryStore class takes advantage of XPO’s ORM features, making it easy to create collections, store and retrieve memory records, and perform batch operations. Since Semantic Kernel doesn’t care where memory records are stored as long as the interfaces are correctly implemented, you can now store your memory records in any of the databases supported by XPO.

Advantages of Using XPO with Semantic Kernel

  • Database Independence: You can switch between multiple databases without changing your codebase.
  • Scalability: XPO’s ability to manage complex relationships and large datasets makes it ideal for enterprise-grade solutions.
  • ORM Abstraction: With XPO, you avoid writing SQL queries and focus on high-level operations like creating and updating objects.

Conclusion

In this blog post, we’ve demonstrated how to integrate DevExpress XPO ORM with the Semantic Kernel using the IMemoryStore interface. This approach allows you to store AI-driven memory records in a wide variety of databases while maintaining a flexible, scalable architecture.

In future posts, we’ll explore specific use cases and how you can leverage this memory store in real-world applications. For the complete implementation, you can check out my GitHub fork.

Stay tuned for more insights and examples!

Creating XAF Property Editors in a Unified Way for Windows Forms and Blazor Using WebView

Creating XAF Property Editors in a Unified Way for Windows Forms and Blazor Using WebView

Introduction

The eXpressApp Framework (XAF) from DevExpress is a versatile application framework that supports multiple UI platforms, including Windows Forms and Blazor. Maintaining separate property editors for each platform can be cumbersome. This article explores how to create unified property editors for both Windows Forms and Blazor by leveraging WebView for Windows Forms and the Monaco Editor, the editor used in Visual Studio Code.

Blazor Implementation

 

 

Windows forms Implementation

 

Prerequisites

Before we begin, ensure you have the following installed:

  • Visual Studio 2022 or later
  • .NET 8.0 SDK or later
  • DevExpress XAF 22.2 or later

Step 1: Create a XAF Application for Windows Forms and Blazor

  1. Create a New Solution:
    • Open Visual Studio and create a new solution.
    • Add two projects to this solution:
      • A Windows Forms project.
      • A Blazor project.
  2. Set Up XAF:
    • Follow the DevExpress documentation to set up XAF in both projects. Official documentation here

Step 2: Create a Razor Class Library

  1. Create a Razor Class Library:
    • Add a new Razor Class Library project to the solution.
    • Name it XafVsCodeEditor.
  2. Design the Monaco Editor Component:

We are done with the shared library that we will reference in both Blazor and Windows projects.

Step 3: Integrate the Razor Class Library into Windows Forms

  1. Add NuGet References:
    • In the Windows Forms project, add the following NuGet packages:
      • Microsoft.AspNetCore.Components.WebView.WindowsForms
      • XafVsCodeEditor (the Razor Class Library created earlier).
    • You can see all the references in the csproj file.
  2. Change the Project Type: In order to add the ability to host Blazor components, we need to change the project SDK from Microsoft.NET.Sdk to Microsoft.NET.Sdk.Razor.
  3. Add Required Files:
    1. wwwroot: folder to host CSS, JavaScript, and the index.html.
    2. _Imports.razor: this file adds global imports. Source here.
    3. index.html: one of the most important files because it hosts a special blazor.webview.js to interact with the WebView. See here.

Official Microsoft tutorial is available here.

Step 4: Implementing the XAF Property Editors

I’m not going to show the full steps to create the property editors. Instead, I will focus on the most important parts of the editor. Let’s start with Windows.

In Windows Forms, the most important method is when you create the instance of the control, in this case, the WebView. As you can see, this is where you instantiate the services that will be passed as a parameter to the component, in our case, the data model. You can find the full implementation of the property editor for Windows here and the official DevExpress documentation here.

protected override object CreateControlCore()
{
    control = new BlazorWebView();
    control.Dock = DockStyle.Fill;
    var services = new ServiceCollection();
    services.AddWindowsFormsBlazorWebView();
    control.HostPage = "wwwroot\\index.html";
    var tags = MonacoEditorTagHelper.AddScriptTags;
    control.Services = services.BuildServiceProvider();
    parameters = new Dictionary<string, object>();
    if (PropertyValue == null)
    {
        PropertyValue = new MonacoEditorData() { Language = "markdown" };
    }
    parameters.Add("Value", PropertyValue);
    control.RootComponents.Add<MonacoEditorComponent>("#app", parameters);
    control.Size = new System.Drawing.Size(300, 300);
    return control;
}

Now, for the property editor for Blazor, you can find the full source code here and the official DevExpress documentation here.

protected override IComponentModel CreateComponentModel()
{
    var model = new MonacoEditorDataModel();
    model.ValueChanged = EventCallback.Factory.Create<IMonacoEditorData>(this, value => {
        model.Value = value;
        OnControlValueChanged();
        WriteValue();
    });
    return model;
}

One of the most important things to notice here is that in version 24 of XAF, Blazor property editors have been simplified so they require fewer layers of code. The magical databinding happens because in the data model there should be a property of the same value and type as one of the parameters in the Blazor component.

Step 5: Running the Application

Before we run our solution, we need to add a domain object that implements a property of type IMonacoData, which is the interface we associated with our property editor. Here is a sample domain object that has a property of type MonacoEditorData:

[DefaultClassOptions]
public class DomainObject1 : BaseObject, IXafEntityObject
{
    public DomainObject1(Session session) : base(session) { }
    public override void AfterConstruction()
    {
        base.AfterConstruction();
    }

    MonacoEditorData monacoEditorData;
    string text;
    
    public MonacoEditorData MonacoEditorData
    {
        get => monacoEditorData;
        set => SetPropertyValue(nameof(MonacoEditorData), ref monacoEditorData, value);
    }

    [Size(SizeAttribute.DefaultStringMappingFieldSize)]
    public string Text
    {
        get => text;
        set => SetPropertyValue(nameof(Text), ref text, value);
    }

    public void OnCreated()
    {
        this.MonacoEditorData = new MonacoEditorData("markdown", "");
        MonacoEditorData.PropertyChanged += SourceEditor_PropertyChanged;
    }

    void IXafEntityObject.OnSaving()
    {
        this.Text = this.MonacoEditorData.Code;
    }

    void IXafEntityObject.OnLoaded()
    {
        this.MonacoEditorData = new MonacoEditorData("markdown", this.Text);
        MonacoEditorData.PropertyChanged += SourceEditor_PropertyChanged;
    }

    private void SourceEditor_PropertyChanged(object sender, PropertyChangedEventArgs e)
    {
        this.Text = this.MonacoEditorData.Code;
    }
}

As you can see, DomainObject1 implements the interface IXafEntityObject. We are using the interface events to load and save the content of the editor to the text property.

Now, build and run the solution. You should now have a Windows Forms application that hosts a Blazor property editor using WebView and the Monaco Editor, as well as a Blazor application using the same property editor.

You can find a working example here.

Conclusion

By leveraging WebView and the Monaco Editor, you can create unified property editors for both Windows Forms and Blazor in XAF applications. This approach simplifies maintenance and provides a consistent user experience across different platforms. With the flexibility of Blazor and the robustness of Windows Forms, you can build powerful and versatile property editors that cater to a wide range of user needs.

The Shift Towards Object Identifiers (OIDs):Why Compound Keys in Database Tables Are No Longer Valid

The Shift Towards Object Identifiers (OIDs):Why Compound Keys in Database Tables Are No Longer Valid

Why Compound Keys in Database Tables Are No Longer Valid

 

Introduction

 

In the realm of database design, compound keys were once a staple, largely driven by the need to adhere to normalization forms. However, the evolving landscape of technology and data management calls into question the continued relevance of these multi-attribute keys. This article explores the reasons why compound keys may no longer be the best choice and suggests a shift towards simpler, more maintainable alternatives like object identifiers (OIDs).

 

The Case Against Compound Keys

 

Complexity in Database Design

 

  • Normalization Overhead: Historically, compound keys were used to satisfy normalization requirements, ensuring minimal redundancy and dependency. While normalization is still important, the rigidity it imposes can lead to overly complex database schemas.
  • Business Logic Encapsulation: When compound keys include business logic, they can create dependencies that complicate data integrity and maintenance. Changes in business rules often necessitate schema alterations, which can be cumbersome.

Maintenance Challenges

 

  • Data Integrity Issues: Compound keys can introduce challenges in maintaining data integrity, especially in large and complex databases. Ensuring the uniqueness and consistency of multi-attribute keys can be error-prone.
  • Performance Concerns: Queries involving compound keys can become less efficient, as indexing and searching across multiple columns can be more resource-intensive compared to single-column keys.

 

The Shift Towards Object Identifiers (OIDs)

 

Simplified Design

 

  • Single Attribute Keys: Using OIDs as primary keys simplifies the schema. Each row can be uniquely identified by a single attribute, making the design more straightforward and easier to understand.
  • Decoupling Business Logic: OIDs help in decoupling the business logic from the database schema. Changes in business rules do not necessitate changes in the primary key structure, enhancing flexibility.

 

Easier Maintenance

 

  • Improved Data Integrity: With a single attribute as the primary key, maintaining data integrity becomes more manageable. The likelihood of key conflicts is reduced, simplifying the validation process.
  • Performance Optimization: OIDs allow for more efficient indexing and query performance. Searching and sorting operations are faster and less resource-intensive, improving overall database performance.

 

Revisiting Normalization

 

Historical Context

 

  • Storage Constraints: Normalization rules were developed when data storage was expensive and limited. Reducing redundancy and optimizing storage was paramount.
  • Modern Storage Solutions: Today, storage is relatively cheap and abundant. The strict adherence to normalization may not be as critical as it once was.

Balancing Act

 

  • De-normalization for Performance: In modern databases, a balance between normalization and de-normalization can be beneficial. De-normalization can improve performance and simplify query design without significantly increasing storage costs.
  • Practical Normalization: Applying normalization principles should be driven by practical needs rather than strict adherence to theoretical models. The goal is to achieve a design that is both efficient and maintainable.

ORM Design Preferences

 

Object-Relational Mappers (ORMs)

 

  • Design with OIDs in Mind: Many ORMs, such as XPO from DevExpress, were originally designed to work with OIDs rather than compound keys. This preference simplifies database interaction and enhances compatibility with object-oriented programming paradigms.
  • Support for Compound Keys: Although these ORMs support compound keys, their architecture and default behavior often favor the use of single-column OIDs, highlighting the practical advantages of simpler key structures in modern application development.

Conclusion

 

The use of compound keys in database tables, driven by the need to fulfill normalization forms, may no longer be the best practice in modern database design. Simplifying schemas with object identifiers can enhance maintainability, improve performance, and decouple business logic from the database structure. As storage becomes less of a constraint, a pragmatic approach to normalization, balancing performance and data integrity, becomes increasingly important. Embracing these changes, along with leveraging ORM tools designed with OIDs in mind, can lead to more robust, flexible, and efficient database systems.

Querying external data sources in XAF

Querying external data sources in XAF

I will explain what XAF is just for the sake of the consistency of this article, XAF is a low code application framework for line of business applications that runs on NET framework (windows forms and web forms) and in dotnet (windows forms, Blazor and Web API)

XAF is laser focus on productivity, DevExpress team has created several modules that encapsulate design patterns and common tasks needed on L.O.B apps.

The starting point in XAF is to provide a domain model using an ORMs like XPO or Entity framework and then XAF will create an application for you using the target platform of choice.

It’s a common misunderstanding that you need to use and ORM in order to provide a domain model to XAF

DevExpress team has created ObjectSpace abstraction so XAF can be extended to use different data access technologies ( you can read more about it here https://docs.devexpress.com/eXpressAppFramework/DevExpress.ExpressApp.BaseObjectSpace)

Out of the box XAF provide 3 branches of object spaces as show is the graph below.

XPObjectSpace: this is the object space that allows you to use XPO as a data access technology.

EfCoreObjectSpace: this is the object space that allows you to use Microsoft Entity Framework as a data access technology.

NonPersistenObjectSpace: this object space is interesting as it provides the domain model needed for XAF to generate the views and interact with the data is not attached to an ORM technology so it’s up to us to provide the data, also this type of object space can be used in combination with XPObjectSpace and EfCoreObjectSpace

 

When querying external data sources, you also need to solve the problem of filtering and sorting data in order to provide a full solution, for that reason DevExpress team provide us with the DynamicCollection class, that is a proxy collection that allows you to filter and sort an original collection without changing it.

Now that we know the parts involved in presenting data in a XAF application, we can define the required flow.

 

    [DefaultClassOptions]
    [DefaultProperty(nameof(Article.Title))]
    [DevExpress.ExpressApp.ConditionalAppearance.Appearance("", Enabled = false, TargetItems = "*")]
    [DevExpress.ExpressApp.DC.DomainComponent]
    public class Article : NonPersistentObjectBase {
        internal Article() { }

 

    public override void Setup(XafApplication application) {
        base.Setup(application);
        // Manage various aspects of the application UI and behavior at the module level.
        application.SetupComplete += Application_SetupComplete;
    }

 

  • Wire the application object space created event.
private void Application_SetupComplete(object sender, EventArgs e) {
    Application.ObjectSpaceCreated += Application_ObjectSpaceCreated;
}
private void Application_ObjectSpaceCreated(object sender, ObjectSpaceCreatedEventArgs e) {
    var npos = e.ObjectSpace as NonPersistentObjectSpace;
    if (npos != null) {
        new ArticleAdapter(npos);
        new ContactAdapter(npos);
    }
}

 

public ArticleAdapter(NonPersistentObjectSpace npos) {
     this.objectSpace = npos;
     objectSpace.ObjectsGetting += ObjectSpace_ObjectsGetting;
 }

 

private void ObjectSpace_ObjectsGetting(object sender, ObjectsGettingEventArgs e) {
     if(e.ObjectType == typeof(Article)) {
         var collection = new DynamicCollection(objectSpace, e.ObjectType, e.Criteria, e.Sorting, e.InTransaction);
         collection.FetchObjects += DynamicCollection_FetchObjects;
         e.Objects = collection;
     }
 }
 private void DynamicCollection_FetchObjects(object sender, FetchObjectsEventArgs e) {
     if(e.ObjectType == typeof(Article)) {
         e.Objects = articles;
         e.ShapeData = true;
     }
 }

Full source code here

In conclusion the ObjectSpace abstraction ensures that different data access technologies can be employed, while the DynamicCollection class allows for seamless filtering and sorting of data from external sources. By following the outlined steps, developers can create robust, adaptable, and efficient applications with XAF, ultimately saving time and effort while maximizing application performance.

How to fix “The type initializer for ‘Gdip’ threw an exception” caused by the netcore framework depencency,  when you run a Xaf Blazor App on ubuntu linux 18.04

How to fix “The type initializer for ‘Gdip’ threw an exception” caused by the netcore framework depencency, when you run a Xaf Blazor App on ubuntu linux 18.04

If you are running Xaf Blazor in ubuntu 18.04 you might have seen the following exception

The type initializer for ‘Gdip’ threw an exception.
at DevExpress.ExpressApp.Actions.ActionBase.OnHandleException(Exception e) at DevExpress.ExpressApp.Actions.ActionBase.ExecuteCore(Delegate handler, ActionBaseEventArgs eventArgs) at DevExpress.ExpressApp.Actions.PopupWindowShowAction.DoExecute(Window window) at DevExpress.ExpressApp.Actions.PopupWindowShowAction.DialogController_Accepting(Object sender, DialogControllerAcceptingEventArgs e) at DevExpress.ExpressApp.SystemModule.DialogController.Accept(SimpleActionExecuteEventArgs args) at DevExpress.ExpressApp.SystemModule.DialogController.acceptAction_OnExecute(Object sender, SimpleActionExecuteEventArgs e) at DevExpress.ExpressApp.Actions.SimpleAction.RaiseExecute(ActionBaseEventArgs eventArgs) at DevExpress.ExpressApp.Actions.ActionBase.ExecuteCore(Delegate handler, ActionBaseEventArgs eventArgs)

The error is caused by missing dependency, so the DotNet runtime itself will throw that exception. Also, I want to highlight that the exception is not related to XAF, you can read more about this problem here https://github.com/dotnet/runtime/issues/27200

To get the missing dependency just open a console and run the following commands

sudo apt-get update -y

sudo apt-get install -y libgdiplus