与 EF Core 的联合返回无法转换集合操作，因为两个操作数具有不同的“包含”操作

Question

在使用 EF Core 5 的 .NET Core 5 WebAPI 项目中，我尝试对 LINQ 查询进行联合，但总是收到错误“无法翻译”。我试图连接的两个实体是相同的，并且字段定义的顺序也相同，所以我无法理解问题是什么以及为什么它不能转换为 SQL UNION：

IQueryable <MonthlyAggregatedPrice> monthlyAggregatedPrices = 
(from map in db.MonthlyAggregatedPrices
  where map.Adm0Code == adm0Code
  orderby map.CommodityPriceDate descending
  select map).Union(
                    from f in db.ST_PewiPriceForecasts
                    join cm in db.Commodities on f.CommodityID equals cm.CommodityID
                    join m in db.Markets on f.MarketID equals m.MarketId
                    join u in db.CommodityUnits on f.CommodityUnitID equals u.CommodityUnitID
                    join pt in db.PriceTypes on f.PriceTypeID equals pt.PriceTypeID
                    join cu in db.Currencies on f.CurrencyID equals cu.CurrencyID
                    where f.Adm0Code == adm0Code
                    select new MonthlyAggregatedPrice
                    {
                        CommodityId = f.CommodityID,
                        MarketId = f.MarketID,
                        PriceTypeId = f.PriceTypeID,
                        CommodityUnitId = f.CommodityUnitID,
                        CurrencyId = f.CurrencyID,
                        CommodityName = cm.CommodityName,
                        MarketName = m.MarketName,
                        PriceTypeName = pt.PriceTypeName,
                        CommodityUnitName = u.CommodityUnitName,
                        CurrencyName = cu.CurrencyName,
                        Adm0Code = adm0Code,
                        CountryISO3 = countryInfo.Iso3Alpha3,
                        CountryName = countryInfo.Name,
                        CommodityPrice = 0,
                        OriginalFrequency = "monthly",
                        CommodityPriceSourceName = "",
                        CommodityPriceObservations = null,
                        CommodityDateMonth = f.PriceForecastMonth,
                        CommodityDateYear = f.PriceForecastYear,
                        CommodityPriceDate= f.PriceDate,
                        CommodityPriceFlag = "forecast"
                    });

MonthlyAggregatePrice 实体是：

public partial class MonthlyAggregatedPrice
{
    public int CommodityId { get; set; }
    public int MarketId { get; set; }
    public int PriceTypeId { get; set; }
    public int CommodityUnitId { get; set; }
    public int CurrencyId { get; set; }
    public string CommodityName { get; set; }
    public string MarketName { get; set; }
    public string PriceTypeName { get; set; }
    public string CommodityUnitName { get; set; }
    public string CurrencyName { get; set; }
    public int Adm0Code { get; set; }
    public string CountryISO3 { get; set; }
    public string CountryName { get; set; }
    public decimal CommodityPrice { get; set; }
    public string OriginalFrequency { get; set; }
    public string CommodityPriceSourceName { get; set; }
    public int? CommodityPriceObservations { get; set; }
    public int CommodityDateMonth { get; set; }
    public int CommodityDateYear { get; set; }
    public DateTime CommodityPriceDate { get; set; }
    public string CommodityPriceFlag { get; set; }
}

它必须是 IQueryable，因为稍后我应该对数据应用更多过滤器

*** 更新 *** 即使我尝试在第一个查询中显式创建对象，我也会收到以下错误：

“当两侧的匹配列具有不同的存储类型时，无法转换集合操作。”

IQueryable < MonthlyAggregatedPrice > monthlyAggregatedPrices = 
(from map in db.MonthlyAggregatedPrices
where map.Adm0Code == adm0Code
orderby map.CommodityPriceDate descending
select new MonthlyAggregatedPrice
{
    CommodityId = map.CommodityId,
    MarketId = map.MarketId,
    PriceTypeId = map.PriceTypeId,
    CommodityUnitId = map.CommodityUnitId,
    CurrencyId = map.CurrencyId,
    CommodityName = map.CommodityName,
    MarketName = map.MarketName,
    PriceTypeName = map.PriceTypeName,
    CommodityUnitName = map.CommodityUnitName,
    CurrencyName = map.CurrencyName,
    Adm0Code = adm0Code,
    CountryISO3 = countryInfo.Iso3Alpha3,
    CountryName = countryInfo.Name,
    CommodityPrice = map.CommodityPrice,
    OriginalFrequency = map.OriginalFrequency,
    CommodityPriceSourceName = map.CommodityPriceSourceName,
    CommodityPriceObservations = map.CommodityPriceObservations,
    CommodityDateMonth = map.CommodityDateMonth,
    CommodityDateYear = map.CommodityDateYear,
    CommodityPriceDate = map.CommodityPriceDate,
    CommodityPriceFlag = map.CommodityPriceFlag
}).Union(
                    from f in db.ST_PewiPriceForecasts
                    join cm in db.Commodities on f.CommodityID equals cm.CommodityID
                    join m in db.Markets on f.MarketID equals m.MarketId
                    join u in db.CommodityUnits on f.CommodityUnitID equals u.CommodityUnitID
                    join pt in db.PriceTypes on f.PriceTypeID equals pt.PriceTypeID
                    join cu in db.Currencies on f.CurrencyID equals cu.CurrencyID
                    where f.Adm0Code == adm0Code
                    select new MonthlyAggregatedPrice
                    {
                        CommodityId = f.CommodityID,
                        MarketId = f.MarketID,
                        PriceTypeId = f.PriceTypeID,
                        CommodityUnitId = f.CommodityUnitID,
                        CurrencyId = f.CurrencyID,
                        CommodityName = cm.CommodityName,
                        MarketName = m.MarketName,
                        PriceTypeName = pt.PriceTypeName,
                        CommodityUnitName = u.CommodityUnitName,
                        CurrencyName = cu.CurrencyName,
                        Adm0Code = adm0Code,
                        CountryISO3 = countryInfo.Iso3Alpha3,
                        CountryName = countryInfo.Name,
                        CommodityPrice = 0,
                        OriginalFrequency = "monthly",
                        CommodityPriceSourceName = "",
                        CommodityPriceObservations = null,
                        CommodityDateMonth = f.PriceForecastMonth,
                        CommodityDateYear = f.PriceForecastYear,
                        CommodityPriceDate=dt,
                        CommodityPriceFlag = "forecast"
                    });

Answer 1

当我使用实体框架和 Oracle 遇到同样的问题时，我找到了一个简单的解决方案。为了方便起见，我复制了 @Phil A. 的部分答案。

(
    from item in _context.Table1
    select new SomeDto
    {
        // Some other fields trimmed for readability
        UserName = Convert.ToString(item.UserName)
    }
)
.Union
(
    from item in _context.Table2
    select new SomeDto
    {
        // Some other fields trimmed for readability
        UserName = Convert.ToString(item.UserName)
    }
)

即使我的实体已经是一个字符串，我也必须在两侧进行转换，以便它们查看相同的数据类型。

Answer 2

我知道这是一篇较旧的帖子，但我遇到了类似的问题并决定发布解决方法。有问题的查询是：

(
    from item in _context.Table1
    select new SomeDto
    {
        // Some other fields trimmed for readability
        UserName = item.UserName
    }
)
.Union
(
    from item in _context.Table2
    select new SomeDto
    {
        // Some other fields trimmed for readability
        UserName = item.UserName
    }
)

类型问题在于 UserName 列，它们是 varchar。解决方案是创建一个函数，将这些函数转换为指定的 varchar。因此，我们创建这样的强制转换函数：

public static class SqlFunctions
{
    public static string CastToVarchar(string value, int? varcharLength) => value;

    public static void Register(ModelBuilder modelBuilder)
    {
        MethodInfo method = typeof(SqlFunctions).GetMethod(nameof(CastToVarchar));
        modelBuilder.HasDbFunction(method).HasTranslation(TranslateCastToVarchar);
    }

    private static SqlExpression TranslateCastToVarchar(IReadOnlyList<SqlExpression> args)
    {
        var fieldArgument = args[0] as ColumnExpression;
        var lengthArgument = args[1] as SqlConstantExpression;

        var length = lengthArgument?.Value ?? "max";

        var result = new SqlFunctionExpression
        (
            "CAST",
            new SqlExpression[]
            {
                new SqlFragmentExpression($"{fieldArgument.TableAlias}.{fieldArgument.Name} AS varchar({length})"),
            },
            nullable: false,
            argumentsPropagateNullability: new[] { true, true },
            typeof(string), // typeof(string)?
            new StringTypeMapping($"varchar({length})", DbType.String) // as varchar(length)
        );

        return result;
    }
}

然后在OnModelCreating中注册它：

SqlFunctions.Register(modelBuilder);

并将查询重写为：

(
    from item in _context.Table1
    select new SomeDto
    {
        // Some other fields
        UserName = SqlFunctions.CastToVarchar(item.UserName, 32)
    }
)
.Union
(
    from item in _context.Table2
    select new SomeDto
    {
        // Some other fields
        UserName = SqlFunctions.CastToVarchar(item.UserName, 32)
    }
)

这最终会将 UserName 列翻译为：

CAST(x.UserName as varchar(32))

工会将会发挥作用。指定 null 作为参数将使用 varchar(max)。如果您需要 nvarchar 或其他类型，您可以轻松修改提供的函数以适应这种情况。

Answer 3

经过几次尝试，我发现实体框架在 UNION 运算符上存在错误，如果添加多个字段，它会变得混乱。

例如，以下查询基于整数和字符串的字段集（全部正确填写在数据库中）不起作用，并且返回“当匹配的列两侧具有不同的存储类型时，无法转换集合操作。 ”：

var tmp = ((from map in db.MonthlyAggregatedPrices
           where map.Adm0Code == adm0Code
           select new UnionTestDto
           {
               CommodityId = map.CommodityId,
               MarketId = map.MarketId,
               PriceTypeId = map.PriceTypeId,
               CommodityUnitId = map.CommodityUnitId,
               CurrencyId = map.CurrencyId,
               CommodityName = map.CommodityName,
               MarketName = map.MarketName,
               PriceTypeName = map.PriceTypeName,
               CommodityUnitName = map.CommodityUnitName,
               CurrencyName = map.CurrencyName
           }).Union(from f in db.ST_PewiPriceForecasts
                    join cm in db.Commodities on f.CommodityID equals cm.CommodityID
                    join m in db.Markets on f.MarketID equals m.MarketId
                    join u in db.CommodityUnits on f.CommodityUnitID equals u.CommodityUnitID
                    join pt in db.PriceTypes on f.PriceTypeID equals pt.PriceTypeID
                    join cu in db.Currencies on f.CurrencyID equals cu.CurrencyID
                    where f.Adm0Code == adm0Code
                    select new UnionTestDto
                    {
                        CommodityId = f.CommodityID,
                        MarketId = f.MarketID,
                        PriceTypeId = f.PriceTypeID,
                        CommodityUnitId = f.CommodityUnitID,
                        CurrencyId = f.CurrencyID,
                        CommodityName = cm.CommodityName,
                        MarketName = m.MarketName,
                        PriceTypeName = pt.PriceTypeName,
                        CommodityUnitName = u.CommodityUnitName,
                        CurrencyName = cu.CurrencyName
                    })).ToList();

但如果您尝试减少字段数量，它就会开始正常工作。我发现基于大约 10k 行的数据结果，在 5 个字段之后 EF 开始在运行 UNION 查询时引发错误。如果您使用 .toList() 单独执行查询，然后应用 UNION，则效果很好。

此外，如果您尝试执行 EF 生成的 SQL（这是正确的），您不会在 SQL Server 或 PostgreSQL 中收到任何错误。

安全使用 UNION 的唯一方法是在数据库中创建视图。