多列数据转换

Question

我正在接收数据源中的数据,在将信息发送到UI进行显示之前,我需要进行数据转换. I am new to concept of pivoting & I am not sure how to go about it.

该问题分为两部分:

1. 形成标题
2. 透视数据以匹配标题

要记住的事情:

3. 我有一些我不想转动的专栏.我打电话给他们static columns.

4. 我需要转动某些列以形成多级头信息.我打电话给他们dynamic columns

5. 某些列需要旋转,其中包含实际值.我打电话给他们value columns.

6. 有没有限制的数目dynamic, static and value columns可以有.

7. 假设数据到来时,我们将首先获得静态列的数据,然后是动态列,然后是值列.

   有关更多信息,请参见附图.

虚拟数据:

class Program
    {
        static void Main(string[] args)
        {
            var _staticColumnCount = 2; //Columns that should not be pivoted        
            var _dynamicColumnCount = 2; // Columns which needs to be pivoted to form header            
            var _valueColumnCount = 1; //Columns that represent Actual value        
            var valueColumnIndex = 4; //Assuming index starts with 0;

            List<List<string>> headerInfo = new List<List<string>>();
            headerInfo.Add(new List<string> {"Product Three", "Item Ten"});
            headerInfo.Add(new List<string> {"Product Two", "Item Five"});
            headerInfo.Add(new List<string> {"Product Two", "Item Seven"});
            headerInfo.Add(new List<string> {"Product Two", "Item Nine"});
            headerInfo.Add(new List<string> {"Product One", "Item One"});
            headerInfo.Add(new List<string> {"Product One", "Item Two"});
            headerInfo.Add(new List<string> {"Product One", "Item Four"});
            headerInfo.Add(new List<string> {"Product One", "Item Six"});
            headerInfo.Add(new List<string> {"Product One", "Item Eight"});
            headerInfo.Add(new List<string> {"Product One", "Item Eleven"});


            List<List<string>> data = new List<List<string>>();
            data.Add(new List<string> {"Global", "Europe", "Product One", "Item One", "579984.59"});
            data.Add(new List<string> {"Global", "North America", "Product One", "Item Two", "314586.73"});
            data.Add(new List<string> {"Global", "Asia", "Product One", "Item One", "62735.13"});
            data.Add(new List<string> {"Global", "Asia", "Product Two", "Item Five", "12619234.69"});
            data.Add(new List<string> {"Global", "North America", "Product Two", "Item Five", "8953713.39"});
            data.Add(new List<string> {"Global", "Europe", "Product One", "Item Two", "124267.4"});
            data.Add(new List<string> {"Global", "Asia", "Product One", "Item Four", "482338.49"});
            data.Add(new List<string> {"Global", "North America", "Product One", "Item Four", "809185.13"});
            data.Add(new List<string> {"Global", "Europe", "Product One", "Item Four", "233101"});
            data.Add(new List<string> {"Global", "Asia", "Product One", "Item Two", "120561.65"});
            data.Add(new List<string> {"Global", "North America", "Product One", "Item Six", "1517359.37"});
            data.Add(new List<string> {"Global", "Europe", "Product One", "Item Six", "382590.45"});
            data.Add(new List<string> {"Global", "North America", "Product One", "Item Eight", "661835.64"});
            data.Add(new List<string> {"Global", "Europe", "Product Three", "Item Three", "0"});
            data.Add(new List<string> {"Global", "Europe", "Product One", "Item Eight", "0"});
            data.Add(new List<string> {"Global", "Europe", "Product Two", "Item Five", "3478145.38"});
            data.Add(new List<string> {"Global", "Asia", "Product One", "Item Six", "0"});
            data.Add(new List<string> {"Global", "North America", "Product Two", "Item Seven", "4247059.97"});
            data.Add(new List<string> {"Global", "Asia", "Product Two", "Item Seven", "2163718.01"});
            data.Add(new List<string> {"Global", "Europe", "Product Two", "Item Seven", "2158782.48"});
            data.Add(new List<string> {"Global", "North America", "Product Two", "Item Nine", "72634.46"});
            data.Add(new List<string> {"Global", "Europe", "Product Two", "Item Nine", "127500"});
            data.Add(new List<string> {"Global", "North America", "Product One", "Item One", "110964.44"});
            data.Add(new List<string> {"Global", "Asia", "Product Three", "Item Ten", "2064.99"});
            data.Add(new List<string> {"Global", "Europe", "Product One", "Item Eleven", "0"});
            data.Add(new List<string> {"Global", "Asia", "Product Two", "Item Nine", "1250"});


        }
    }

Answer 1

您调用static columns的通常称为行组,dynamic columns- 列组和value columns- 值聚合或简单值.

为了实现这个目标,我建议使用以下简单的数据结构:

public class PivotData
{
    public IReadOnlyList<PivotValues> Columns { get; set; }
    public IReadOnlyList<PivotDataRow> Rows { get; set; }
}

public class PivotDataRow
{
    public PivotValues Data { get; set; }
    public IReadOnlyList<PivotValues> Values { get; set; }
}

将Columns成员PivotData表示您调用标题的Row成员,而成员 - PivotDataRow具有Data包含行组值的成员的对象列表和Values- 相应索引的值Columns(PivotDataRow.Values将始终具有相同Count的值PivotData.Columns.Count).

上述数据结构可序列化/可反序列化为JSON(使用Newtosoft.Json测试),可用于使用所需格式填充UI.

用于表示行组值,列组值和聚合值的核心数据结构如下:

public class PivotValues : IReadOnlyList<string>, IEquatable<PivotValues>, IComparable<PivotValues>
{
    readonly IReadOnlyList<string> source;
    readonly int offset, count;
    public PivotValues(IReadOnlyList<string> source) : this(source, 0, source.Count) { }
    public PivotValues(IReadOnlyList<string> source, int offset, int count)
    {
        this.source = source;
        this.offset = offset;
        this.count = count;
    }
    public string this[int index] => source[offset + index];
    public int Count => count;
    public IEnumerator<string> GetEnumerator()
    {
        for (int i = 0; i < count; i++)
            yield return this[i];
    }
    IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
    public override int GetHashCode()
    {
        unchecked
        {
            var comparer = EqualityComparer<string>.Default;
            int hash = 17;
            for (int i = 0; i < count; i++)
                hash = hash * 31 + comparer.GetHashCode(this[i]);
            return hash;
        }
    }
    public override bool Equals(object obj) => Equals(obj as PivotValues);
    public bool Equals(PivotValues other)
    {
        if (this == other) return true;
        if (other == null) return false;
        var comparer = EqualityComparer<string>.Default;
        for (int i = 0; i < count; i++)
            if (!comparer.Equals(this[i], other[i])) return false;
        return true;
    }
    public int CompareTo(PivotValues other)
    {
        if (this == other) return 0;
        if (other == null) return 1;
        var comparer = Comparer<string>.Default;
        for (int i = 0; i < count; i++)
        {
            var compare = comparer.Compare(this[i], other[i]);
            if (compare != 0) return compare;
        }
        return 0;
    }
    public override string ToString() => string.Join(", ", this); // For debugging
}

基本上它表示string具有相等和顺序比较语义的列表的范围(切片).前者允许在数据透视转换期间使用基于哈希的高效LINQ运算符,而后者允许可选的排序.此外,此数据结构允许有效转换,无需分配新列表,同时在从JSON反序列化时保留实际列表.

(通过实现IEquatable<PivotValues>接口GetHashCode和Equals方法来提供相等比较.通过这样做,它允许PivotValues根据List<string>输入元素内指定范围内的值将两个类实例视为相等List<List<string>>.类似,通过实现IComparable<PivotValues>接口提供排序- CompareTo方法))

转型本身很简单:

public static PivotData ToPivot(this List<List<string>> data, int rowDataCount, int columnDataCount, int valueDataCount)
{
    int rowDataStart = 0;
    int columnDataStart = rowDataStart + rowDataCount;
    int valueDataStart = columnDataStart + columnDataCount;

    var columns = data
        .Select(r => new PivotValues(r, columnDataStart, columnDataCount))
        .Distinct()
        .OrderBy(c => c) // Optional
        .ToList();

    var emptyValues = new PivotValues(new string[valueDataCount]); // For missing (row, column) intersection

    var rows = data
        .GroupBy(r => new PivotValues(r, rowDataStart, rowDataCount))
        .Select(rg => new PivotDataRow
        {
            Data = rg.Key,
            Values = columns.GroupJoin(rg,
                c => c,
                r => new PivotValues(r, columnDataStart, columnDataCount),
                (c, vg) => vg.Any() ? new PivotValues(vg.First(), valueDataStart, valueDataCount) : emptyValues
            ).ToList()
        })
        .OrderBy(r => r.Data) // Optional
        .ToList();

    return new PivotData { Columns = columns, Rows = rows };
}

首先,使用简单的LINQ Distinct运算符确定列(标题).然后通过对行列的源集进行分组来确定行.每个行分组内的值由外部加入Columns分组内容确定.

由于我们的数据结构实现,LINQ转换非常有效(空间和时间).列和行排序是可选的,如果您不需要,可以将其删除.

使用虚拟数据进行样本测试:

var pivotData = data.ToPivot(2, 2, 1);
var json = JsonConvert.SerializeObject(pivotData);
var pivotData2 = JsonConvert.DeserializeObject<PivotData>(json);