DirectSaver – EfCore shaman

DirectSaver can be used for quick data insert and update bypassing Entity Framework time consuming operations. In order to use it follow steps below:

Inherit your DataContext class from ShamanDbContext instead of DbContext, and set necessary options in constructor. For example

C#

public class ProductionContext : ShamanDbContext { public ProductionContext() { ShamanOptions.WithSqlServer(); } }

1
2
3
4
5
6
7

public class ProductionContext : ShamanDbContext
{
    public ProductionContext()
    {
        ShamanOptions.WithSqlServer();
    }
}

ShamanDbContext provides GetDirectSaver method that is factory method for DirectSaver objects.
Obrain direct saver object related for entity type

C#

var personDirectSaver = ctx.GetDirectSaver<Person>();

1

var personDirectSaver = ctx.GetDirectSaver<Person>();
Use Insert or Update method depending on situation, i.e.

C#

if (isInsert) personDirectSaver.Insert(dbContext, person); else personDirectSaver.Update(dbContext, person);

1
2
3
4

if (isInsert)
personDirectSaver.Insert(dbContext, person);
else
personDirectSaver.Update(dbContext, person);

Supported features

Works only with MsSql Server, but can be extended for other databases when necessary.
Identity fields and other fields marked with DatabaseGeneratedAttribute are readed back from database after insert operation.

Demo

Direct saver has been tested with small program for data migration. Data from XML file was persisted in MsSQL database.

Option 1, classic Ef usage

At the beginning I’ve used simple code template:

int SaveXmlObject(XmlRecipeNode xmlObject)
{
    // check if object already exists i.e. previous migration
    var dbObject = TryGetExisting(xmlObject);
    var isNewRecord = dbObject == null;
    var needToBeSaved = isNewRecord || CheckDifferences(dbObject, xmlObject);
    if (!needToBeSaved)
        return dbObject.Id;
    if (isNewRecord)
    {
        dbObject = MakeFromXml(xmlObject);
        context.RecipeNodes.Add(dbObject);
    }
    else
    {
        UpdateFromXml(dbObject, xmlObject);
    }
    context.SaveChanges();
    return dbObject.Id;
}

int SaveXmlObject(XmlRecipeNode xmlObject)

{

// check if object already exists i.e. previous migration

var dbObject = TryGetExisting(xmlObject);

var isNewRecord = dbObject == null;

var needToBeSaved = isNewRecord || CheckDifferences(dbObject, xmlObject);

if (!needToBeSaved)

return dbObject.Id;

if (isNewRecord)

{

dbObject = MakeFromXml(xmlObject);

context.RecipeNodes.Add(dbObject);

}

else

{

UpdateFromXml(dbObject, xmlObject);

}

context.SaveChanges();

return dbObject.Id;

}

I’ve spend some time for optimizing performance but finally gave up. I was totally disappointed obtaining speed between 15-40 saved elements per second. My biggest dataset consists of 105.000 records so estimated time was around hour.

Option 2, DirectSaver

I’ve replaced context.RecipeNodes.Add and context.SaveChanges() with Insert and Update, so my code template looks now like this.

int SaveXmlObject(XmlRecipeNode xmlObject)
{
    var context = new ProductionContext();
    // check if object already exists i.e. previous migration
    var dbObject = TryGetExisting(xmlObject);
    var isNewRecord = dbObject == null;
    var needToBeSaved = isNewRecord || CheckDifferences(dbObject, xmlObject);
    if (!needToBeSaved)
        return dbObject.Id;
    if (isNewRecord)
    {
        dbObject = MakeFromXml(xmlObject);
        _directSaver.Insert(context, dbObject);
    }
    else
    {
        UpdateFromXml(dbObject, xmlObject);
        _directSaver.Update(context, dbObject);
    }
    return dbObject.Id;
}

int SaveXmlObject(XmlRecipeNode xmlObject)

{

var context = new ProductionContext();

// check if object already exists i.e. previous migration

var dbObject = TryGetExisting(xmlObject);

var isNewRecord = dbObject == null;

var needToBeSaved = isNewRecord || CheckDifferences(dbObject, xmlObject);

if (!needToBeSaved)

return dbObject.Id;

if (isNewRecord)

{

dbObject = MakeFromXml(xmlObject);

_directSaver.Insert(context, dbObject);

}

else

{

UpdateFromXml(dbObject, xmlObject);

_directSaver.Update(context, dbObject);

}

return dbObject.Id;

}

Performance was quite good, between 1600 and 2400 records per second.

Conclusion

DirectSaver can be used for mass data insert or update with great performance and without effort related to using SqlCommand. Exactly the same, simple code can be used after model migration, because DirectSaver uses ‘code first’ annotation for obtaining data model.
Disadvantage of this approach is that programmer must care about data relational consistency and object changes detection by himself.